3 messages in com.mysql.lists.clusterRe: node failure on load data
FromSent OnAttachments
Sand...@dtn.com16 Nov 2004 11:07 
Tomas Ulin17 Nov 2004 05:54.patch
Sand...@dtn.com19 Nov 2004 06:31 
Subject:Re: node failure on load data
From:Sand...@dtn.com (Sand@dtn.com)
Date:11/19/2004 06:31:53 AM
List:com.mysql.lists.cluster

I tried your suggestion, building v4.1.7 with your patch. It seems a slow disk is not my problem, at least the patch isn't solving it. I am still getting the same 'Pointer too large.' error. I tried a config with only one of the machines, running the mgmd and the ndbd on the localhost. I still got the same error when trying to LOAD DATA.

It seems like people are having success using a modified myisam mysqldump to load data. I will try to get some data by using the insert statements from a mysqldump.

Or do you think I should try the LOAD DATA with 4.1.8?

Thanks,

Sandi

Tomas Ulin <tom@mysql.com> 11/17/2004 07:54 AM

To Sand@dtn.com cc clus@lists.mysql.com Subject Re: node failure on load data

We think this is a problem of a slow disk. Resulting in a resource in ndb running out. We have a patch for this that is in the appended file.

If you have compiled from source, you should be able to apply this 1-line patch and see if it fixes you problem. We have problems verifying if this is really the problem from the setups that we have available.

Do you think you can try? Or do you want to try fresh with a 4.1.8? (we have not pushed the fix there yet)

BR,

T

Sand@dtn.com wrote:

I am attempting to load ~3,000,000 records into table using a LOAD DATA statement. Record size is 48 bytes, 32 byte primary key, and an additional non-unique ordered index. ~19,000 records are loaded then I get a node failure. The error in the log file is: Date/Time: Tuesday 16 November 2004 - 12:09:51 Type of error: error Message: Pointer too large Fault ID: 2306 Problem data: DblqhMain.cpp Object of reference: DBLQH (Line: 12820) 0x0000000a ProgramName: NDB Kernel ProcessID: 15609 TraceFile: /usr/local/mysql-cluster/ndb_2_trace.log.11 ***EOM***

What does the error "Pointer too large" mean?

After the node failure, I restart the cluster nodes, and the MYSQLD nodes

will not reconnect right away, takes 5-10 minutes. What is the reason for the delay? Is there a way to force mysqld to reconnect to the mgmd [a mysqld restart doesn't do it]?

I am running the MySQL-Max v4.1.7 cluster on 2 Linux machines with 1 replica. Here's my config: [COMPUTER] Id=1 HostName=10.0.8.46

[COMPUTER] Id=2 HostName=ticker5-test

[NDBD DEFAULT] NoOfReplicas= 1 RedoBuffer=128M

[MYSQLD DEFAULT] [NDB_MGMD DEFAULT] [TCP DEFAULT]

[NDB_MGMD] ExecuteOnComputer=1 PortNumber=2200

[NDBD] ExecuteOnComputer=1 DataDir= /usr/local/mysql-cluster DataMemory=800M IndexMemory=640M

[NDBD] ExecuteOnComputer=2 DataDir= /usr/local/mysql-cluster DataMemory=800M IndexMemory=640M

[MYSQLD] ExecuteOnComputer=1

[MYSQLD] ExecuteOnComputer=2

If the reason for the failure is dues to attempting to load so much data in one transaction, what parameters can I adjust to allow it? It is not feasible for me to break up the data files into smaller chunks for loading, as I have 6 more tables to load with 3-6 million records each.

Also, my application reading the data will need to query as much as 10,000 records at a time? Are there parameters I need to adjust to allow querying this much data?

Thanks for any help you can provide.

Sandi

===== ndb/src/kernel/blocks/dblqh/DblqhInit.cpp 1.11 vs edited ===== --- 1.11/ndb/src/kernel/blocks/dblqh/DblqhInit.cpp Fri Aug 27 22:31:13 2004 +++ ndb/src/kernel/blocks/dblqh/DblqhInit.cpp Wed Nov 17 13:38:15 2004 @@ -344,6 +344,9 @@

initData();

+ clfoFileSize = + (clfoFileSize < clogPageFileSize ? clogPageFileSize : clfoFileSize); + #ifdef VM_TRACE { void* tmp[] = {