Re: Writing to MYD/MYI on low level - Kim en Max - com.mysql.lists.java

8 messages in com.mysql.lists.javaRe: Writing to MYD/MYI on low level

From	Sent On	Attachments
Sergei Rodionov	15 Jun 2005 07:34
Kim en Max	15 Jun 2005 11:26
Kim en Max	15 Jun 2005 11:34
Sergei Rodionov	15 Jun 2005 18:10
Jeff Mathis	16 Jun 2005 08:49
JR Ruggentaler	16 Jun 2005 08:59
Ronald Klop	21 Jun 2005 05:42
Kim en Max	21 Jun 2005 06:02

Subject:	Re: Writing to MYD/MYI on low level
From:	Kim en Max (kime...@xs4all.nl)
Date:	06/21/2005 06:02:41 AM
List:	com.mysql.lists.java

Such an 'infile' is as easy as filling the file with queries. I use the infile method primary for migrating database structure and/or data. I am not sure this sollutions is providing any improvements related to normal inserts; the benefit of writing your inserts into a infile first would be that the filestream process for writing the file would demand less recourses and time; but then when you have to import the infile into your mysql dbms I think it should take just as much time as excecuting the inserts one by one.

The original idea of writing the myisam file binary could improve performance though I am confedent the developers of MySQL are focusing on the same subject; besides as it was said in the former message: 'this just sounds scary'. You are risking ceausing bugs in the filestructure ceausing MySQL to get in trouble.

kind regards,

Max van de Wiel

Op dinsdag 21 juni 2005 14:43, schreef Ronald Klop:

And such an 'infile' should be easy to generate.

On Thu Jun 16 18:00:03 CEST 2005 JR Ruggentaler <JR.R...@mpv.com>

wrote:

The fastest way I've found to insert bulk data into MySQL is LOAD DATA INFILE... see http://dev.mysql.com/doc/mysql/en/load-data.html. It's much faster than SQL inserts. Works best if the data is local to the MySQL server.

J.R.

-----Original Message----- From: Jeff Mathis [mailto:jmat...@predict.com] Sent: Thursday, June 16, 2005 10:49 AM To: Serg...@xxi.com Cc: ja...@lists.mysql.com Subject: Re: Writing to MYD/MYI on low level

this just sounds scary. i don't think i would be by-passing the database and creating your own database files. i think that if you are having performance problems with inserts you should work to optimize your insert code.

just my $0.02

On Jun 15, 2005, at 7:10 PM, Sergei Rodionov wrote:

Hi , Max

Since I was in the dillusion RDBMS systems where invented tho take care of the low level aspects entill your message, you have made me very currious about this topic. May I ask you what's behind the thought a Java object for this task would save you a lot of hasle?

Well, you see, its all nice, and MySQL is fast db and all that, no doubt. But when i need to transform million or few of Java objects into db records - speed of insertion becomes somewhat of issue.

So when i can output binary file in 15 mins or less, and insertion of same amount of data takes at least 4 times more - I am starting to look for other meanings of getting data in db (like writing separate table on low level, and then do batch merge).

That would be the reason to look at the low level output.

Cheers Sergei

---------------------------------------------------------------- Jeff Mathis, Ph.D. 505-955-1434 (w) Prediction Company 505-660-2901 (c) 525 Camino de los Marquez, Ste 6 http://www.predict.com Santa Fe, NM 87505 jmat...@predict.com

Max van de Wiel