7 messages in ru.sysoev.nginxRe: nginx large POST body under Solaris
FromSent OnAttachments
Wil TanMar 25, 2008 12:45 pm 
Igor SysoevMar 25, 2008 1:41 pm 
Wil TanMar 25, 2008 3:41 pm 
Igor SysoevMar 25, 2008 11:45 pm 
Wil TanMar 26, 2008 1:44 am 
Igor SysoevMar 26, 2008 2:16 am.segfault
Wil TanMar 26, 2008 9:23 am 
Actions with this message:
Paste this link in email or IM:
Paste this link in email or IM:
Atom feed for this thread
Paste this URL into your reader:
Subject:Re: nginx large POST body under SolarisActions...
From:Wil Tan (wil-@public.gmane.org)
Date:Mar 25, 2008 3:41:42 pm
List:ru.sysoev.nginx

Hello Igor,

Thank you for your quick response. I've installed gdb and enabled global setuid core and reproduced the scenario for 0.6.29. Here's the backtrace:

[moji@d9b60282 /opt/local/src]$ sudo gdb /opt/local/nginx/sbin/nginx / var/core/core.nginx.12427 GNU gdb 6.6 Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-pc-solaris2.10"... Reading symbols from /lib/libsocket.so.1...done. Loaded symbols for /lib/libsocket.so.1 Reading symbols from /lib/libnsl.so.1...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /lib/librt.so.1... warning: Lowest section in /lib/librt.so.1 is .dynamic at 00000074 done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libsendfile.so.1...done. Loaded symbols for /lib/libsendfile.so.1 Reading symbols from /opt/local/lib/libpcre.so.0...done. Loaded symbols for /usr/local/lib/libpcre.so.0 Reading symbols from /opt/local/lib/libz.so.1...done. Loaded symbols for /usr/local/lib/libz.so.1 Reading symbols from /lib/libc.so.1...done. Loaded symbols for /lib/libc.so.1 Reading symbols from /usr/sfw/lib/libgcc_s.so.1...done. Loaded symbols for /usr/sfw/lib/libgcc_s.so.1 Reading symbols from /lib/ld.so.1...done. Loaded symbols for /lib/ld.so.1 Core was generated by `/opt/local/nginx/sbin/nginx'. Program terminated with signal 11, Segmentation fault. #0 ngx_writev_chain (c=0x810e830, in=0x80d7d74, limit=2147479551) at src/os/unix/ngx_writev_chain.c:92 92 iov->iov_len += size; (gdb) where #0 ngx_writev_chain (c=0x810e830, in=0x80d7d74, limit=2147479551) at src/os/unix/ngx_writev_chain.c:92 #1 0x0807753e in ngx_solaris_sendfilev_chain (c=0x810e830, in=0x80d7cfc, limit=0) at src/os/unix/ngx_solaris_sendfilev_chain.c:64 #2 0x08062a05 in ngx_chain_writer (data=0x80ee7c8, in=0x0) at src/ core/ngx_output_chain.c:483 #3 0x08062518 in ngx_output_chain (ctx=0x80ee790, in=0xe) at src/ core/ngx_output_chain.c:217 #4 0x0808b859 in ngx_http_upstream_send_request (r=0x80f9798, u=0x80ee75c) at src/http/ngx_http_upstream.c:923 #5 0x0808b670 in ngx_http_upstream_connect (r=0x80f9798, u=0x80ee75c) at src/http/ngx_http_upstream.c:766 #6 0x0808af77 in ngx_http_upstream_init (r=0x80f9798) at src/http/ ngx_http_upstream.c:442 #7 0x0808786a in ngx_http_do_read_client_request_body (r=0x80f9798) at src/http/ngx_http_request_body.c:373 #8 0x08087495 in ngx_http_read_client_request_body_handler (r=0x80f9798) at src/http/ngx_http_request_body.c:246 #9 0x080818f8 in ngx_http_request_handler (ev=0x0) at src/http/ ngx_http_request.c:1566 #10 0x08070beb in ngx_event_process_posted (cycle=0x80d9da0, posted=0x80d1d94) at src/event/ngx_event_posted.c:39 #11 0x0806fb2a in ngx_process_events_and_timers (cycle=0x80d9da0) at src/event/ngx_event.c:272 #12 0x08076b56 in ngx_worker_process_cycle (cycle=0x80d9da0, data=0x0) at src/os/unix/ngx_process_cycle.c:767 #13 0x08074e11 in ngx_spawn_process (cycle=0x80d9da0, proc=0x8076a96 <ngx_worker_process_cycle>, data=0x0, name=0x80a85b1 "worker process", respawn=-3) at src/os/ unix/ngx_process.c:187 #14 0x0807636e in ngx_start_worker_processes (cycle=0x80d9da0, n=2, type=-3) at src/os/unix/ngx_process_cycle.c:332 #15 0x08076083 in ngx_master_process_cycle (cycle=0x80d9da0) at src/ os/unix/ngx_process_cycle.c:224 #16 0x0805f34d in main (argc=135037216, argv=0x8047d08) at src/core/ nginx.c:355 (gdb)

Thanks!

=wil

On Mar 25, 2008, at 4:42 PM, Igor Sysoev wrote:

On Tue, Mar 25, 2008 at 03:45:46PM -0400, Wil Tan wrote:

Hello Nginx-ers,

I had a problem with uploading large files to a FastCGI (Django) process behind nginx. Nginx crashes when running under Solaris 10 (SunOS 5.11 snv_67 i86pc i386 i86pc) and nginx 0.5.35 compiled without any special modules. I also tried nginx 0.6.29 with the same problem.

The problem is that when I upload a small file, it works fine. But when uploading a large file (> 1Mb) I got 413 request too large and so I upped the client_max_body_size parameter in the config. When I upload, I get a blank page back, with a message like this:

2008/03/25 08:03:15 [warn] 3176#0: *5 a client request body is buffered to a temporary file /tmp/nginx/0/00/0000000000, client: 75.75.25.21, server: mojipage.com, request: "POST /account/avatar/? t=1206428905 HTTP/1.1", host: "mojipage.com", referrer: "http:// mojipage.com/account/avatar/?t=1206428905" 2008/03/25 08:03:24 [notice] 3175#0: signal 18 (SIGCHLD) received 2008/03/25 08:03:24 [alert] 3175#0: worker process 3176 exited on signal 11 2008/03/25 08:03:24 [notice] 3175#0: start worker process 3184

Here's a debug log that I correspond to one of these attempts:

2008/03/25 07:29:21 [debug] 28590#0: *5 connect to unix:/home/moji/ web/floweb/floweb-fcgi.sock, fd:9 #6 2008/03/25 07:29:21 [debug] 28590#0: *5 devpoll add event: fd:9 ev: 0001 2008/03/25 07:29:21 [debug] 28590#0: *5 devpoll fd:9 ev:0001 fl:0000 2008/03/25 07:29:21 [debug] 28590#0: *5 connected 2008/03/25 07:29:21 [debug] 28590#0: *5 http upstream connect: 0 2008/03/25 07:29:21 [debug] 28590#0: *5 http upstream send request 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:992 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:354 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:14 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 malloc: 080F0E00:4096 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:27809 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:15 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:32768 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:8 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:27809 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer buf fl:0 s:15 2008/03/25 07:29:21 [debug] 28590#0: *5 chain writer in: 080F0D4C 2008/03/25 07:29:21 [debug] 28590#0: *5 writev: 1408 2008/03/25 07:29:21 [notice] 28575#0: signal 18 (SIGCHLD) received 2008/03/25 07:29:21 [alert] 28575#0: worker process 28590 exited on signal 11

Any pointers would be appreciated.

You should enable core dumps for setuid processes (nginx calls setuid() to change unprivileged user) and then run

gdb /path/to/nginx /path/to/core

bt

quit

-- Igor Sysoev http://sysoev.ru/en/