4 messages in com.mysql.lists.clusterRe: Memory never gets freed. ever!
FromSent OnAttachments
Chuck Cochems21 Feb 2008 11:16 
Geert Vanderkelen25 Feb 2008 07:20 
Geert Vanderkelen26 Feb 2008 01:31 
Geert Vanderkelen26 Feb 2008 22:56 
Subject:Re: Memory never gets freed. ever!
From:Geert Vanderkelen (gee@mysql.com)
Date:02/26/2008 10:56:18 PM
List:com.mysql.lists.cluster

Hi Chuck,

(putting list back in)

On Feb 26, 2008, at 18:52 , Chuck Cochems wrote: ..

Only if you tell it to lock pages in memory. I have datamemory at 12 gigs.

Not true. LockPagesInMainMemory is to make user data nodes are not swapping. If a data node can't allocate the asked memory, it will fail to start.

As shown with top below, which shows a diffrence between allocated and used, there is indeed a difference. I kept trying to tell the other people that it's not really used, and will get recycled as needed, but they refuse to believe it and say that the machine slows to a crawl when it fills up.

If the memory is allocated by the ndbd process it can't be used anymore by the OS. It doesn't mean that MySQL Cluster is actually using the memory, but it made sure it can when it has too. LockPagesInMainMemory just makes sure it doesn't Swap out. You best put this in no matter what.

Everything should be faster. How long did it take? Lots of data means longer startup since it has to either to read data from disk or sync it's data from the other data node when it was offline to long.

It easily takes an hour or more, during which time it seems to be using almost no cpu.

This can happen with lots of data. There is continuous improvement in this area (in 5.1/CGE).

If I try to drop and re-import the tables, it FAILS if I don't restart the cluster after dropping.

How does it fail? What error? How is the memory usage before and after dropping the table, check using ALL DUMP 1000.

I'm not even told how it's failing. All i know is what they keep telling me. I checked their syntax, and the docs say it should free the memory, but they insist they are running out.

I will try and get the error.

SHOW WARNINGS might be useful too, but it depends how you do the import..

When Mysql 5.1 becomes a generally available release, i will be allowed to put it on. until then, i MUST stick with 5.0.

This is intended to be a production site when it works.

To bad you can't check out MySQL 5.1.. Can't tell you when it gets out, but it would have made sense using it.

Cheers,

Geert

-- Geert Vanderkelen, Senior Support Engineer MySQL GmbH, Germany, www.mysql.com