82 messages in com.xensource.lists.xen-develRe: RE: RE: [Xen-devel] poor domU VBD...
FromSent OnAttachments
Peter Bier26 Mar 2005 10:14 
Ian Pratt27 Mar 2005 09:41 
peter bier28 Mar 2005 00:48 
peter bier28 Mar 2005 04:44 
Ian Pratt28 Mar 2005 10:54 
Andrew Theurer28 Mar 2005 11:32 
Ian Pratt28 Mar 2005 12:13 
Andrew Theurer28 Mar 2005 12:17 
Andrew Theurer28 Mar 2005 13:48 
Ian Pratt28 Mar 2005 14:16 
Peter Bier28 Mar 2005 15:38 
Andrew Theurer28 Mar 2005 16:27 
Pasi Kärkkäinen28 Mar 2005 22:19 
Ian Pratt29 Mar 2005 00:13 
peter bier29 Mar 2005 00:43 
peter bier29 Mar 2005 03:39 
Kurt Garloff29 Mar 2005 05:33 
Ian Pratt29 Mar 2005 05:38 
Ian Pratt29 Mar 2005 06:19 
B.G. Bruce29 Mar 2005 06:28 
peter bier29 Mar 2005 07:27 
Andrew Theurer29 Mar 2005 10:39.Other
Steven Hand29 Mar 2005 11:13 
Kurt Garloff29 Mar 2005 14:44.diff
Andrew Theurer29 Mar 2005 14:58 
Kurt Garloff29 Mar 2005 15:19 
Andrew Theurer29 Mar 2005 15:26 
Jens Axboe30 Mar 2005 00:53 
Kurt Garloff30 Mar 2005 02:00 
Ian Pratt30 Mar 2005 03:15 
peter bier30 Mar 2005 09:00 
Andrew Theurer30 Mar 2005 10:04 
Jens Axboe30 Mar 2005 23:04 
Jens Axboe30 Mar 2005 23:10 
Keir Fraser31 Mar 2005 00:17 
Jens Axboe31 Mar 2005 00:18 
Philip R Auld31 Mar 2005 06:32 
Kurt Garloff31 Mar 2005 07:34 
Jens Axboe31 Mar 2005 07:39 
Jens Axboe31 Mar 2005 07:41 
Keir Fraser31 Mar 2005 07:49 
Andrew Theurer31 Mar 2005 08:01 
Nivedita Singhvi31 Mar 2005 08:27 
Philip R Auld31 Mar 2005 08:52 
Philip R Auld31 Mar 2005 08:55 
Jens Axboe31 Mar 2005 09:42 
Jens Axboe31 Mar 2005 09:44 
Ian Pratt31 Mar 2005 09:54 
Jens Axboe31 Mar 2005 10:01 
Jens Axboe31 Mar 2005 10:03 
Kurt Garloff31 Mar 2005 10:27 
Philip R Auld31 Mar 2005 10:42 
Keir Fraser31 Mar 2005 10:57 
Keir Fraser31 Mar 2005 11:07 
Keir Fraser31 Mar 2005 11:10 
Jens Axboe31 Mar 2005 11:19 
Jens Axboe31 Mar 2005 11:21 
Jens Axboe31 Mar 2005 11:21 
Andrew Theurer31 Mar 2005 12:48 
Keir Fraser31 Mar 2005 13:14 
Andrew Theurer31 Mar 2005 13:26 
Ian Pratt31 Mar 2005 13:32 
Nivedita Singhvi31 Mar 2005 13:59 
Andrew Theurer31 Mar 2005 14:13 
Ian Pratt31 Mar 2005 14:36 
Andrew Theurer31 Mar 2005 15:05 
Jens Axboe31 Mar 2005 21:43 
peter bier01 Apr 2005 08:35 
Ian Pratt01 Apr 2005 09:46 
Cédric Schieli01 Apr 2005 13:40 
Ian Pratt01 Apr 2005 15:21 
Cédric Schieli02 Apr 2005 02:36 
Ian Pratt02 Apr 2005 02:56 
Cédric Schieli02 Apr 2005 04:10 
peter bier02 Apr 2005 11:53 
Cédric Schieli03 Apr 2005 08:26 
Ian Pratt03 Apr 2005 09:38 
Cédric Schieli04 Apr 2005 12:13 
Ian Pratt04 Apr 2005 12:36 
Nicholas Lee04 Apr 2005 15:35 
peter bier12 Apr 2005 03:28 
Ian Pratt12 Apr 2005 03:51 
Subject:Re: RE: RE: [Xen-devel] poor domU VBD performance.
From:Jens Axboe (axb@suse.de)
Date:03/30/2005 11:10:21 PM
List:com.xensource.lists.xen-devel

On Thu, Mar 31 2005, Jens Axboe wrote:

On Wed, Mar 30 2005, Ian Pratt wrote:

I'll check the xen block driver to see if there's anything else that sticks out.

Jens Axboe

Jens, I'd really appreciate this.

The blkfront/blkback drivers have rather evolved over time, and I don't think any of the core team fully understand the block-layer differences between 2.4 and 2.6.

There's also some junk left in there from when the backend was in Xen itself back in the days of 1.2, though Vincent has prepared a patch to clean this up and also make 'refreshing' of vbd's work (for size changes), and also allow the blkfront driver to import whole disks rather than paritions. We had this functionality on 2.4, but lost it in the move to 2.6.

My bet is that it's the 2.6 backend that is where the true perofrmance bug lies. Using a 2.6 domU blkfront talking to a 2.4 dom0 blkback seems to give good performance under a wide variety of circumstances. Using a 2.6 dom0 is far more pernickety. I agree with Andrew that I suspect it's the work queue changes are biting us when we don't have many outstanding requests.

You never schedule the queues you submit the io against for the 2.6 kernel, you only have a tq_disk run for 2.4 kernels. This basically puts you at the mercy of the timeout unplugging, which is really suboptimal unless you can keep the io queue of the target busy at all times.

You need to either mark the last bio going to that device as BIO_SYNC, or do a blk_run_queue() on the target queue after having submitted all io in this batch for it.

Here is a temporary work-around, this should bring you close to 100% performance at the cost of some extra unplugs. Uncompiled.

--- blkback.c~ 2005-03-31 09:06:16.000000000 +0200 +++ blkback.c 2005-03-31 09:09:27.000000000 +0200 @@ -481,7 +481,6 @@ for ( i = 0; i < nr_psegs; i++ ) { struct bio *bio; - struct bio_vec *bv;

bio = bio_alloc(GFP_ATOMIC, 1); if ( unlikely(bio == NULL) ) @@ -494,17 +493,12 @@ bio->bi_private = pending_req; bio->bi_end_io = end_block_io_op; bio->bi_sector = phys_seg[i].sector_number; - bio->bi_rw = operation;

- bv = bio_iovec_idx(bio, 0); - bv->bv_page = virt_to_page(MMAP_VADDR(pending_idx, i)); - bv->bv_len = phys_seg[i].nr_sects << 9; - bv->bv_offset = phys_seg[i].buffer & ~PAGE_MASK; + bio_add_page(bio, virt_to_page(MMAP_VADDR(pending_idx, i)), + phys_seg[i].nr_sects << 9, + phys_seg[i].buffer & ~PAGE_MASK);

- bio->bi_size = bv->bv_len; - bio->bi_vcnt++; - - submit_bio(operation, bio); + submit_bio(operation | (1 << BIO_RW_SYNC), bio); } #endif

-- Jens Axboe

_______________________________________________ Xen-devel mailing list Xen-@lists.xensource.com http://lists.xensource.com/xen-devel