Re: Help with large database import - Matthew Montgomery - com.mysql.lists.cluster

9 messages in com.mysql.lists.clusterRe: Help with large database import

From	Sent On	Attachments
Anders D. Hansen	01 Mar 2007 07:35
Anders D.Hansen	01 Mar 2007 12:50
Anders D. Hansen	01 Mar 2007 13:38
Pradeep Chandru	15 Oct 2007 00:56
Matthew Montgomery	15 Oct 2007 11:24
Pradeep Chandru	16 Oct 2007 22:56
Matthew Montgomery	19 Oct 2007 10:36
Matthew Montgomery	20 Oct 2007 13:40
Matthew Montgomery	22 Oct 2007 22:34

Subject:	Re: Help with large database import
From:	Matthew Montgomery (mmon...@mysql.com)
Date:	10/15/2007 11:24:03 AM
List:	com.mysql.lists.cluster

Hello Pradeep,

On Mon, 2007-10-15 at 13:26 +0530, Pradeep Chandru wrote:

Hi, I have the similar kind of issue in MySQL 5.0.45.

ERROR 1114 (HY000) at line 1702: The table 'qp_weightage_1' is full

I am trying the installation of mysql cluster for the first time. I am trying to import a db of size 900MB in to the cluster. The server has got 8GB RAM. I have not configured any of the parameters like data memory, index memory... etc. I have the following doubts:

1. How can i load a huge table. What are the variables to be tuned? what are the parameters to be considered for the import to be successful?

You *must* first define your DataMemory and IndexMemory. The default values for DataMemory is 80M and 18M for IndexMemory.

Optionally you should make sure you have sufficient MaxNoOfTables, MaxNoOfAttributes, MaxNoOfOrderedIndexes and MaxNoOfUniqueHashIndexes

You should also know that tables in cluster can require far more room to store the tables than in other engines. This is especially true in 5.0 because VARCHAR columns are stored in memory with a fixed length. So a VARCHAR(255) column will require a minimum of 255 bytes + 16 bytes overhead of DataMemory per row. VARCHAR columns can require as much as three times this amount for UTF-8 characters.

2. Is there any link where i can find more data on tuning the variables.

You can use the ndb_size.pl script to determine the recommended values for IndexMemory and DataMemory.