17 messages in ru.sysoev.nginxRe: High memory consumption when prox...| From | Sent On | Attachments |
|---|---|---|
| Rogério Schneider | Apr 11, 2010 5:16 pm | |
| Maxim Dounin | Apr 11, 2010 5:48 pm | |
| Rogério Schneider | Apr 11, 2010 6:07 pm | |
| Maxim Dounin | Apr 11, 2010 6:56 pm | |
| Rogério Schneider | Apr 11, 2010 8:42 pm | |
| Maxim Dounin | Apr 12, 2010 12:51 pm | |
| Rogério Schneider | Apr 12, 2010 1:16 pm | |
| Maxim Dounin | Apr 12, 2010 2:26 pm | |
| Rogério Schneider | Apr 15, 2010 12:42 am | |
| zanettea | May 28, 2010 6:54 am | |
| Rogério Schneider | Jun 7, 2010 7:14 pm | |
| Maxim Dounin | Jun 8, 2010 1:34 am | |
| zanettea | Jun 8, 2010 4:45 am | |
| Rogério Schneider | Jun 8, 2010 7:45 am | |
| Rogério Schneider | Jun 8, 2010 7:54 am | |
| Igor Sysoev | Jun 8, 2010 8:05 am | |
| Rogério Schneider | Jun 8, 2010 4:57 pm |
| Subject: | Re: High memory consumption when proxying to a Comet server | |
|---|---|---|
| From: | Maxim Dounin (mdou...@mdounin.ru) | |
| Date: | Apr 12, 2010 12:51:55 pm | |
| List: | ru.sysoev.nginx | |
Hello!
On Mon, Apr 12, 2010 at 12:43:23AM -0300, Rogério Schneider wrote:
[...]
You may switch on debug log, it will be possible to trace big allocations there. See here:
I posted this log of some tests, and if you can please take a look at it, I would appreciate.
It was separated in sections, the start, the small msg, the big msg, the close. Then another run, the small msg after many big msgs, and the close (with lots of "strange" free()s):
Yep, I see what's going on. It's chunked filter which eats memory - it allocates small buffers for chunk-size and crlf markers. They aren't big, but they are allocated for each buffer sent, and they aren't reused.
Here is snipped from log where it causes another 4k allocation from system (note "malloc" line):
2010/04/12 00:06:04 [debug] 32748#0: *55 recv: fd:16 1024 of 1024
2010/04/12 00:06:04 [debug] 32748#0: *55 http output filter
"/push/user1/xhrinteractive/canal2.b1?"
2010/04/12 00:06:04 [debug] 32748#0: *55 copy filter:
"/push/user1/xhrinteractive/canal2.b1?"
2010/04/12 00:06:04 [debug] 32748#0: *55 http chunk: 1024
2010/04/12 00:06:04 [debug] 32748#0: *55 malloc: 096E9878:4096
2010/04/12 00:06:04 [debug] 32748#0: *55 write new buf t:1 f:0 00000000, pos
096E985C, size: 5 file: 0, size: 0
2010/04/12 00:06:04 [debug] 32748#0: *55 write new buf t:0 f:0 00000000, pos
096E8140, size: 1024 file: 0, size: 0
2010/04/12 00:06:04 [debug] 32748#0: *55 write new buf t:0 f:0 00000000, pos
080A6EE5, size: 2 file: 0, size: 0
2010/04/12 00:06:04 [debug] 32748#0: *55 http write filter: l:0 f:1 s:1031
2010/04/12 00:06:04 [debug] 32748#0: *55 http write filter limit 0
2010/04/12 00:06:04 [debug] 32748#0: *55 writev: 1031
As a workaround you may want to increase proxy_buffer_size to reduce number of such allocation (and/or just drop connections periodically). Correct fix would be to make these buffers reusable once sent to client.
Maxim Dounin
_______________________________________________ nginx mailing list ngi...@nginx.org http://nginx.org/mailman/listinfo/nginx