RE: table full error? - Crouch, Luke H. - com.mysql.lists.cluster

26 messages in com.mysql.lists.clusterRE: table full error?

From	Sent On	Attachments
Crouch, Luke H.	29 Jul 2004 13:36
Mikael Ronström	29 Jul 2004 13:57
Crouch, Luke H.	29 Jul 2004 14:29
Mikael Ronström	29 Jul 2004 14:46
Crouch, Luke H.	29 Jul 2004 15:07
Mikael Ronström	29 Jul 2004 15:28
Crouch, Luke H.	30 Jul 2004 06:40
Mikael Ronström	30 Jul 2004 07:08
Crouch, Luke H.	30 Jul 2004 07:25
Mikael Ronström	30 Jul 2004 08:23
Crouch, Luke H.	30 Jul 2004 08:36
pek...@mysql.com	31 Jul 2004 02:41
Crouch, Luke H.	31 Jul 2004 08:27
pek...@mysql.com	01 Aug 2004 03:09
Sundeep Narravula	01 Aug 2004 03:32
pek...@mysql.com	01 Aug 2004 03:50
Crouch, Luke H.	02 Aug 2004 06:29
pek...@mysql.com	02 Aug 2004 06:42
Crouch, Luke H.	03 Aug 2004 08:23
Mikael Ronström	04 Aug 2004 03:38
Crouch, Luke H.	04 Aug 2004 06:15
Mikael Ronström	04 Aug 2004 06:34
Crouch, Luke H.	04 Aug 2004 08:54
Mikael Ronström	04 Aug 2004 10:31
Crouch, Luke H.	04 Aug 2004 10:38
Mikael Ronström	04 Aug 2004 10:57

Subject:	RE: table full error?
From:	Crouch, Luke H. (Luke...@red-man.com)
Date:	07/30/2004 07:25:01 AM
List:	com.mysql.lists.cluster

I don't get the Ndb kernel stuck message, but I do still get an error trying to
start up...

2004-07-30 04:19:52 [NDB] INFO -- Angel pid: 2097 ndb pid: 2098 2004-07-30 04:19:52 [NDB] INFO -- NDB Cluster -- DB node 5 2004-07-30 04:19:52 [NDB] INFO -- Version 3.5.0 (beta) -- 2004-07-30 04:20:01 [NDB] INFO -- Start initiated (version 3.5.0) Error handler shutting down system Error handler shutdown completed - exiting

error.log:

Date/Time: Friday 30 July 2004 - 04:20:16 Type of error: error Message: Memory allocation failure Fault ID: 2327 Problem data: DBTUP could not allocate memory for Page Object of reference: Requested: 32768x16000 = 524288000 bytes ProgramName: NDB Kernel ProcessID: 2098 TraceFile: NDB_TraceFile_13.trace

that's after I changed the DataMemory to be 500M. and when I check the memory on
the boxes, I see...

Mem: 2061516k av, 55052k used, 2006464k free, 0k shrd, 7324k buff 21728k actv, 1936k in_d, 8k in_c

that's 2 GB free memory! and it's having problems allocating just 500M ?

is there a portion of the trace file you'd like to see for this, or do you have
an idea of what is happening now?

thanks again for all the help, -L

-----Original Message----- From: Mikael Ronström [mailto:mik...@mysql.com] Sent: Friday, July 30, 2004 9:09 AM To: Crouch, Luke H. Cc: clus...@lists.mysql.com Subject: Re: table full error?

Hi Luke,

There is an outstanding bug report I filed on this problem. The workaround is to increase Watch Dog Timeout. TimeBetweenWatchDogCheck: 30000 as an example.

2004-07-30 kl. 15.41 skrev Crouch, Luke H.:

since we have 4 machines with 2GB memory in each, I went ahead and set...

DataMemory: 1600M IndexMemory: 48M

Would recommend more balance between DataMemory and IndexMemory, also providing some memory for other processes running on the machine (if I remember correctly you also run a mysqld on the machine.

So something like: DataMemory: 800M IndexMemory: 250M

in the DB DEFAULT section. but when I try to start up now, I get...

[root@rh-mysql-1 2-db2]# ndbd -i 2004-07-30 03:21:33 [NDB] INFO -- Angel pid: 2904 ndb pid: 2906 2004-07-30 03:21:33 [NDB] INFO -- NDB Cluster -- DB node 2 2004-07-30 03:21:33 [NDB] INFO -- Version 3.5.0 (beta) -- 2004-07-30 03:21:33 [NDB] INFO -- Start initiated (version 3.5.0) Ndb kernel is stuck in: Job Handling Error handler shutting down system Error handler shutdown completed - exiting

another question regarding the memory requirement here...if we have 4 nodes, and 2 replicas, and only 1 node on each machine, then each machine will only need .5*DBSIZE of memory, correct? and for this particular table, if it is, say, 400MB, then each machine would actually only be using around 200MB for this particular table, right?

Right.

Rgrds Mikael

thanks, -L

-----Original Message----- From: Mikael Ronström [mailto:mik...@mysql.com] Sent: Thursday, July 29, 2004 5:29 PM To: Crouch, Luke H. Cc: clus...@lists.mysql.com Subject: Re: table full error?

Hi Luke,

2004-07-30 kl. 00.07 skrev Crouch, Luke H.:

Mikael,

the table has 4 million records, each about 100 bytes, and no primary key (doh!)

If no primary key exists then an 8 byte primary key will be generated. 100 bytes and default DataMemory setting should not provide space for more than around 800k records so 996k records is even more than that. IndexMemory default setting with 8 byte PK should also provide space for around 800k records. If you have a 4 node cluster then it can handle twice the amount of data.

could that be part of our problem? this doesn't seem like an unusually large table...do you think the default values would be able to handle a table this size? could it be something else other than these memory settings?

You need to increase default settings. Default settings is mostly for easy start on any machine, also a small machine. Sounds like you need to set DataMemory to something like 500M and IndexMemory to something like 150M. Check the manual at http://dev.mysql.com/doc/mysql/en/MySQL_Cluster_DB_Definition.html and the entries for DataMemory and IndexMemory for more info on how to calculate the proper parameters.

Also I am not exactly sure what your error code was. I can't think of any other table full variant than what here has been discussed but providing the error message will clear the doubts away.

Rgrds Mikael

thanks, -L

-----Original Message----- From: Mikael Ronström [mailto:mik...@mysql.com] Sent: Thursday, July 29, 2004 4:47 PM To: Crouch, Luke H. Cc: Vinay Joosery; clus...@lists.mysql.com Subject: Re: table full error?

Hi Luke, I am referring to the MySQL Manual available on http://dev.mysql.com/doc/mysql/en/index.html where chapter 17 is dedicated to MySQL Cluster. This page is updated immediately as changes are done in the manual.

To set a parameter to 300 MByte you write 300M as an example.

The cluster administration guide is a useful resource still but it is not updated so the manual contains the "latest and the greatest".

Rgrds Mikael

2004-07-29 kl. 23.30 skrev Crouch, Luke H.:

Mikael,

what manual are you referencing? the mysql cluster administration guide that I have has the parameters NoOfIndexPages and NoOfDataPages for DB definitions, with a unit of 8KB pages. what unit is the DataMemory and IndexMemory parameters expressed in? kB?

is there a kind of 'live' manual available online? where can we get the most recent manual each day?

thanks, -L

-----Original Message----- From: Mikael Ronström [mailto:mik...@mysql.com] Sent: Thursday, July 29, 2004 3:58 PM To: Crouch, Luke H. Cc: Vinay Joosery; clus...@lists.mysql.com Subject: Re: table full error?

Hi Luke, Most likely DataMemory and IndexMemory from your short description. Check the manual for more details on how to set it. Default is DataMemory size of 80000 kBytes sounds reasonable if your records are about 60-70 bytes. IndexMemory is by default set to 24000 kBytes so it is reasonable that this parameter is your problem if you have a 4 byte primary key.

Rgrds Mikael

2004-07-29 kl. 22.36 skrev Crouch, Luke H.:

we are trying to load up an NDB table thru one of the API nodes. we have a file (generated by a mysqldump) that inserts 4 million records, but when we run it, it stops at about 996k and says table is full! where is the setting that controls this?

thanks, -L

Mikael Ronström, Senior Software Architect MySQL AB, www.mysql.com

Clustering: http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html

http://www.eweek.com/article2/0,1759,1567546,00.asp

Mikael Ronström, Senior Software Architect MySQL AB, www.mysql.com

Clustering: http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html

http://www.eweek.com/article2/0,1759,1567546,00.asp

Mikael Ronström, Senior Software Architect MySQL AB, www.mysql.com

Clustering: http://www.infoworld.com/article/04/04/14/HNmysqlcluster_1.html

http://www.eweek.com/article2/0,1759,1567546,00.asp