We've an application that logs data on one very large raid6 array
and updates/accesses a database on another smaller raid5 array.
You would be better off with RAID10 for a database (or anything which
does random IO).
Both arrays are connected to the same PCIe 3ware RAID controller. The
system has 2 six core 3Ghz processors and 24 GB of RAM. The system is
running FreeBSD 8.1.
Did you do any additional OS tuning? Do you use UFS or ZFS?
The problem we're encountering is that the disk subsystem appears to
'pause' periodically. It looks as if this is a result of disk read/write
operations from the database array taking a very long time to complete
(up to 8 sec).
You should be able to monitor this with "iostat -x 1" (or whatever
number of seconds instead of "1") - the last three columns should tell
you if the device(s) are extraordinarily busy, and the r/s and w/s
columns should tell you what the real IOPS rate is. You should probably
post a sample output from this command when the problem appears.