17 messages in org.freebsd.freebsd-fsRe: FreeBSD random I/O performance is...| From | Sent On | Attachments |
|---|---|---|
| Richard Wendland | Mar 21, 2000 4:22 pm | |
| Matthew Dillon | Mar 21, 2000 4:59 pm | |
| Paul Richards | Mar 21, 2000 6:44 pm | |
| Matthew Dillon | Mar 21, 2000 10:17 pm | |
| Richard Wendland | Mar 22, 2000 7:43 am | |
| Mike Smith | Mar 22, 2000 12:39 pm | |
| Matthew Dillon | Mar 22, 2000 4:10 pm | |
| FREENIX IS OVERRATED | Mar 22, 2000 4:16 pm | |
| Matthew Dillon | Mar 22, 2000 4:48 pm | |
| Julian Elischer | Mar 22, 2000 11:33 pm | |
| Brad Knowles | Mar 23, 2000 1:34 am | |
| Richard Wendland | Mar 26, 2000 1:24 pm | |
| Matthew Dillon | Apr 1, 2000 5:03 pm | |
| tele danmarQ kvindeservice | Apr 4, 2000 3:02 pm | |
| Kun Limfjordsporter | Apr 4, 2000 6:28 pm | |
| Matthew Dillon | Apr 4, 2000 7:18 pm | |
| Kun Limfjordsporter | Apr 4, 2000 10:23 pm |
| Subject: | Re: FreeBSD random I/O performance issues | |
|---|---|---|
| From: | tele danmarQ kvindeservice (kvin...@netscum.dk) | |
| Date: | Apr 4, 2000 3:02:27 pm | |
| List: | org.freebsd.freebsd-fs | |
On Thu, 23 Mar 19100 (not like I'm slow or anything), Matthew Dillon wrote:
[regarding random-access database files, such as are used for the INN history files, and FreeBSD 4.0...]
For INN there are several things you can tune in 4.0. First and foremost you can try turning off the write-behind code, sysctl -w vfs.write_behind=0. Secondly you can mess around with the vfs.hidirtybuffers sysctl (generally lower it) in order to force out dirty pages earlier and thus reduce the number that fsync has to deal with.
Thanks for the tips. A few days ago, I rebuilt one of my transit peering boxen to use the 4.0-RELENG k0deZ and then proceeded to tune the above, after letting it run for a while to get a feel for how it was acting, as well as to let accumulated backlogs clear out.
Now, the history file I'm dealing with contains some 14 million entries, so there's about 200 megabytes of goods to be scribbled over the disks (or at least the pages for some 12-18 entries per second over the 30- second intervals). It's just a single disk, so it takes a while to dump the data, and much more so when taking in backlogs at several dozen message IDs per second (limited mostly by the time blocked waiting for the disk writes to complete).
After first firing things up, it almost seemed to be taking anywhere from 25 to 55 seconds to dump to the disks every so often, with an available cycle time of perhaps 10 to 15%. After a bit of time, as the backlogs were cleared out, things seemed to settle down to somewhere around 15 seconds blocked, plus or minus a bit, followed by around 25 seconds of activity.
Toggling the sysctl knob and tweaking the limits might have trimmed a couple seconds from the unresponsive times, although it's difficult to say just what was an improvement, and what was due to natural fluctuation in the flow of incoming news. It was nothing to write home about. So, while I did tune things, the results weren't quite what I hoped for.
I believe that INN also messes around with shared/R+W mmap()'s - it may be possible to add MAP_NOSYNC to those maps to turn off the 30 second fsync on pages dirtied through the VM system (for those maps), though this may increase the amount of stale (unwritten) data after a crash.
If this is the MMAP_SYNC in the (INN 1.5) config.data file, this is set to `DONT' on these boxes.
I have noticed that our 2.2-RELENG box, which had the time between sync disk dumps tweaked up to an hour, always seemed to sync the disks when it would crash (except when the truck delivering a new UPS ran into the electric pole), and as long as the text file (only appended to) is in a consistent state, one can recreate the database files, so the risk of losing data hasn't concerned me too much.
Now, I did notice with great joy that you committed some k0deZ a few days ago that might help this problem, and as soon as I unearth that announcement, I'll followup to that and describe the results on our history database files.
thanks, barry (not a fluffy, oh no) bouwsma, teledanmorkinternet
--
*** This was posted with the express permission of *** ****************************************************** ** HIS HIGHNESS KAAZMANN LORD AND MASTER OF USENET ** ****************************************************** ********* We are simple servants of his will *********
To Unsubscribe: send mail to majo...@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message