Re: crashing: /proc/613/exe: Permission denied - Martin MOKREJŠ - com.mysql.lists.bugs

9 messages in com.mysql.lists.bugsRe: crashing: /proc/613/exe: Permissi...

From	Sent On	Attachments
Martin MOKREJŠ	18 Oct 2002 04:36
Michael Widenius	20 Oct 2002 10:03
Martin MOKREJŠ	21 Oct 2002 04:03
Martin MOKREJŠ	04 Nov 2002 06:23
Michael Widenius	11 Nov 2002 05:53
Michael Widenius	11 Nov 2002 07:52
Martin MOKREJŠ	20 Nov 2002 08:25
Martin MOKREJŠ	18 Dec 2002 02:26
Martin MOKREJŠ	07 Jan 2003 07:30

Subject:	Re: crashing: /proc/613/exe: Permission denied
From:	Martin MOKREJŠ (mmok...@natur.cuni.cz)
Date:	11/20/2002 08:25:41 AM
List:	com.mysql.lists.bugs

On Mon, 11 Nov 2002, Michael Widenius wrote:

Hi!

Sorry for the delay, I have been very busy lately... I will now answer all your open questions in this thread.

"Martin" == Martin MOKREJ <Martin> writes:

<cut>

Martin> I see now in the $datadir 2 files:

Martin> REPORT: CURRENT PROCESS:459, THREAD:2051 Martin> --------- STACK TRACE FOR PROCESS 459, THREAD 2051 ---------------

Martin> FAULT ADDR: 00000000 Martin> EIP: 0023:[<081c31c9>] ESP: 002b:bf5ff938 Martin> EAX: fffffffc EBX: bf5ff984 ECX: 00000008 EDX: 0000001f Martin> ESI: bf5ffc00 EDI: bf5ff984 EBP: bf5ff954 DS: 002b ES: 002b Martin> STACK TRACE: Martin> 0x081c31c9: ???? (bf5ff984, 0, bf5ff984, 81ac1e6, 0, 0) + 90 Martin> 0x081ad269: ???? (bf5ffc00, 0, bf5ffa44, 81aa345, 8300fc0, 8300fcc) + 20 Martin> 0x081aa3fc: ???? (82c0bf4, 82c05fc, bf5ffa84, 8080234, 8300fc0, 8300fc0)
+ 10 Martin> 0x08080312: ???? (8300fc0, 1, bf5ffb34, 808966b, 8300fcc, 0) + a0 Martin> 0x08089680: ???? (8300fc0, 0, bf5ffd28, 81ab113, 81ab100, 0) + a0 Martin> 0x081ab1d8: ???? (bf5ffc00, bf5ffc00, 0, 0, 0, 0) + 40a003ec

<cut>

Martin> I've never resolved linux kernel/thread crashes, any quick help?

I have never got a stack trace like the above, so I can't help you much with this.

Actually, to resolve system stack traces, this seems to be the right way:

From: Randy.Dunlap <rddu...@osdlab.org> Date: 2001-09-20 15:02:45 Subject: Re: __alloc_pages: 0-order allocation failed still in -pre12

Andrea Arcangeli wrote:

Can you also resolve "c012e052" so we know who's allocating those pages just in case?

It's trivial to do that, of course, but if someone needs an automated way to do it (several times, easy lookup), you can try http://www.osdlab.org/sw_resources/scripts/ksysmap .

Usage is: ksysmap [system_map_file] offset

and it spits out address/symbol before offset, exact match if present, and address/symbol after offset.

Example:

[rddunlap@dragon linux]$ ksysmap ./System.map-249acpi c012e052 ksysmap: searching './System.map-249acpi' for 'c012e052'

c012df20 T sys_truncate c012e052 ..... <<<<< c012e0a0 T sys_ftruncate

If the above comes from mysql, you can get some information about this by pasting the stack trace result to 'resolve_stack_trace' that you can find in the extra directory:

nm -n mysqld > mysqld.sym resolve_stack_trace mysqld.sym 0x081c31c9

etc...

Yes, it works. I've resolved stack trace created today:

root@kulan:~# cat /data/mysql/mysqld-7935-87107-15643.backtrace --------- SEGV in PROCESS 15643, THREAD 87107 ---------------

FAULT ADDR: fffffffc EIP: 0023:[<081d8f97>] ESP: 002b:b75ff738 EAX: fffffffc EBX: b75ff934 ECX: 00000008 EDX: b75ffc00 ESI: fffffffc EDI: b75ffc00 EBP: b75ff760 DS: 002b ES: 002b STACK TRACE: 0x081d8f97: ???? (b75ff934, b75ffc00, b75ff934, 0, 0, 0) + 10 0x081d4ab5: ???? (fffffffc, 82fa6d8, bd5263e8, cc50, b3, 1f80) + 130 0xb75ff7a5: ???? (b75ff934, 82fa6d8, 82fa6d8, b75ff9b4, 81d79c1, b75ff934) + 4 0x081ed939: ???? (b75ff934, 0, b75ff934, 81d91d7, 0, 0) + 90 0x081d79d9: ???? (b75ffc00, ffffff8f, b75ff9f4, 81d4ab5, 82fa6c0, b75ffa68) + 20 0x081d4b6c: ???? (82faf58, 82fa6d8, b75ffa34, 81c8c7d, 82fa6c0, 15443) + 20 0x081c8c8d: ???? (82faf58, 82fa6c0, 824975c, 4a6, 832b090, 93736c4) + 20 0x08082c99: ???? (93736b8, 1, b75ffb34, 808ce40, 93736c4, 0) + a0 0x0808ce55: ???? (96cb310, 0, b75ffd28, 81d5883, 81d5870, 0) + a0 0x081d5948: ???? (b75ffc00, b75ffc00, 0, 0, 0, 0) + 48a003ec root@kulan:~# ls -la /data/mysql/mysqld-7935-87107-15643.backtrace -rw------- 1 mysql mysql 949 Nov 20 10:48
/data/mysql/mysqld-7935-87107-15643.backtrace root@kulan:~# nm -n /usr/local/mysql/libexec/mysqld > mysqld.sym root@kulan:~# /usr/local/mysql/bin/resolve_stack_dump -s mysqld.sym -n
/data/mysql/mysqld-7935-87107-15643.backtrace --------- SEGV in PROCESS 15643, THREAD 87107 ---------------

FAULT ADDR: fffffffc EIP: 0023:[<081d8f97>] ESP: 002b:b75ff738 EAX: fffffffc EBX: b75ff934 ECX: 00000008 EDX: b75ffc00 ESI: fffffffc EDI: b75ffc00 EBP: b75ff760 DS: 002b ES: 002b STACK TRACE: 0x81d8f97 __pthread_lock + 39 0x81d4ab5 pthread_cond_wait + 173 0xb75ff7a5 _end + -1355978691 0x81ed939 sigsuspend + 25 0x81d79d9 __pthread_wait_for_restart_signal + 69 0x81d4b6c pthread_cond_wait + 356 0x81c8c8d safe_cond_wait + 241 0x8082c99 end_thread__FP3THDb + 265 0x808ce55 handle_one_connection__FPv + 973 0x81d5948 pthread_start_thread + 216 root@kulan:~#

The system was running linux-2.4.20-pre10-ac2 on i686. mysql-BK20021104-debug