RE: Problem starting ndbd on some RHEL3 images - Morris Mark - mamorr - com.mysql.lists.cluster

3 messages in com.mysql.lists.clusterRE: Problem starting ndbd on some RHE...

From	Sent On	Attachments
Morris Mark - mamorr	10 Mar 2008 13:22
Morris Mark - mamorr	10 Mar 2008 13:25
Morris Mark - mamorr	14 Mar 2008 10:48

Subject:	RE: Problem starting ndbd on some RHEL3 images
From:	Morris Mark - mamorr (Mark...@acxiom.com)
Date:	03/14/2008 10:48:06 AM
List:	com.mysql.lists.cluster

I compiled from source on one of the nodes where ndbd fails to start. I used --with-debug option of configure and was able to get the following message in the log.

ndbd: NdbThread.c:121: NdbThread_Create: Assertion `result==0' failed.

It appears that something in the newer patches for RHEL3 regarding the pthreads library have caused this code to fail.

Has anyone else experienced this issue or is this a bug? Is there a compile option I need to set or something to get this working?

Note: If I reverse the implementation and just run the manager on the patched images, I can run ndbd on the "older" non-patched RHEL3 machine and it works correctly. So the only code affected is ndbd (I haven't tried mysqld on these images).

-----Original Message----- From: Morris Mark - mamorr Sent: Monday, March 10, 2008 3:26 PM To: clus...@lists.mysql.com Subject: RE: Problem starting ndbd on some RHEL3 images

Here is the /usr/tmp/ndb_1_cluster.log

2008-03-08 19:22:14 [MgmSrvr] INFO -- NDB Cluster Management Server. Version 5.0.51

2008-03-08 19:22:14 [MgmSrvr] INFO -- Id: 1, Command port: 1186

2008-03-08 19:22:55 [MgmSrvr] INFO -- Mgmt server state: nodeid 2 reserved for ip 10.103.10.130, m_reserved_nodes 0000000000000006.

2008-03-08 19:22:55 [MgmSrvr] INFO -- Node 1: Node 2 Connected

2008-03-08 19:22:57 [MgmSrvr] INFO -- Mgmt server state: nodeid 2 freed, m_reserved_nodes 0000000000000002.

2008-03-08 19:23:19 [MgmSrvr] ALERT -- Node 1: Node 2 Disconnected

Here is the /usr/tmp/ndb_1_out.log

NDB Cluster Management Server. Version 5.0.51

Id: 1, Command port: 1186

setEventReportingLevelImpl(2): failed

Here is the /usr/tmp/ndb_2_error.log

Current byte-offset of file-pointer is: 568

Time: Saturday 8 March 2008 - 19:23:19

Status: Temporary error, restart node

Message: WatchDog terminate, internal error or massive overload on the machine running this node (Internal error, programming error or missing error message, please report a bug)

Error: 6050

Error data: Job Handling

Error object: WatchDog.cpp

Program: ./ndbd

Pid: 19887

Trace: /usr/tmp/ndb_2_trace.log.1

Version: Version 5.0.51

***EOM***

Here is the /usr/tmp/ndb_2_trace.log.1

JAM CONTENTS up->down left->right ?=not block entry

BLOCK ADDR ADDR ADDR ADDR ADDR ADDR ADDR ADDR

?001549 001549 001549 001549 001549 001549 001549 001549

001549 001549 001549 001549 001549 001549 001549 001549

001549 001549 001549 001553

NDBCNTR 002692 002741

DBDIH 001084 011640 012149 011790

DBDIH 000297 000538 011645 011652 011790

DBDIH 000297 000538 011657 011790

DBDIH 000297 000538 011677 011790

DBDIH 000297 000538 011693 011790

DBDIH 000297 000538 011698 011790

DBDIH 000297 000538 011722 011790

DBDIH 000297 000538 011738 011790

DBDIH 000297 000538 011758 011790

DBDIH 000297 000538 011770

NDBCNTR 002692 002741