[Xen-devel] RE: VP problematic for backend drivers on IA64? - Magenheimer, Dan (HP Labs Fort Collins) - com.xensource.lists.xen-ia64-devel

2 messages in com.xensource.lists.xen-ia64-devel[Xen-devel] RE: VP problematic for ba...

From	Sent On	Attachments
Magenheimer, Dan (HP Labs Fort Collins)	22 Jan 2006 14:44
Alex Williamson	23 Jan 2006 07:52

Subject:	[Xen-devel] RE: VP problematic for backend drivers on IA64?
From:	Magenheimer, Dan (HP Labs Fort Collins) (dan....@hp.com)
Date:	01/22/2006 02:44:55 PM
List:	com.xensource.lists.xen-ia64-devel

-----Original Message----- From: Muli Ben-Yehuda [mailto:mul...@mulix.org] Sent: Friday, January 20, 2006 3:17 PM To: Magenheimer, Dan (HP Labs Fort Collins) Cc: xen-devel; okr...@us.ibm.com; Jon Mason Subject: Re: VP problematic for backend drivers on IA64?

On Fri, Jan 20, 2006 at 09:08:21AM -0800, Magenheimer, Dan (HP Labs Fort Collins) wrote:

Hi Muli --

I'm cc'ing the xen-ia64-devel list as many of the Xen/ia64 team don't keep up with xen-devel...

Actually, you didn't :-)

Oops! For anyone on xen-ia64-devel wanting to catch up on this thread:

http://lists.xensource.com/archives/html/xen-devel/2006-01/msg00492.html http://lists.xensource.com/archives/html/xen-devel/2006-01/msg00507.html

The backend drivers have a lot of code that assume P2M. Blkback has been "ported" to handle P==M but netback never was. Neither has been "ported" to VP yet so there is some work to do. It may turn out to be easy (e.g. #define'ing a few macros to be no-ops). However, there's likely to be some subtle changes too as there was for P==M.

Where can I find the diff for blkback to work P==M? is this integrated into xen-unstable or is it in the IA64 tree?

It is all checked in to xen-unstable. (The xen-ia64-unstable tree is sync'ed roughly weekly with xen-unstable.)

But the real problem is not really in the backend drivers, it is in the lower layers of the driver stack that the backend drivers sit on top of. VP means that the machine addresses are hidden to the domain. But domain0 (and future driver domains) still need to program DMA-capable devices, both for any domain0 I/O and for I/O on behalf of domU's (via blkfront/blkback). Thus, domain0 cannot really be fully VP.

Linux provides the DMA-API abstraction, so that drivers do not need to be aware of the deails of translating from a guest-physical address to a bus address (akak machine address). Theoretically, a DMA-API implementation is the only part of the dom0 Linux kernel that would need to know to read the P2M table (P2M) or do nothing (P=M) or call into Xen to get the tanslation (VP without IOMMU) or call into Xen to establish an IOMMU mapping (VP w/ IOMMU).

Yes, unless there are legacy drivers/devices that circumvent the DMA interface. I don't know if this is the case on some/many/all Linux/ia64 configurations... perhaps someone with more familiarity with a broad range of Linux/ia64 configurations can comment? I would be concerned with, for example, IDE, GART, VGA, console...?

I think what we discussed at the summit was a modified form of VP which is somewhere between VP and P2M. All RAM addressing is VP, but all device addressing needs to be P2M. It was observed that since an IOMMU intercepts all device addressing (and only device addressing), by ensuring that domain0 (and any driver domain) only has device addressing via a "software IOMMU", the problem should be solved.

Unless the machine has a real HW IOMMU, the device must see bus addresses, which means the driver must pass it bus addresses. The "virtual IOMMU" therefore becomes a DMA-API implementation which calls into Xen for P->Bus translation.

OK.

That just about exhausts my expertise in this area, so others can feel free to jump in (and please correct my mistakes).

I think it makes sense. Does IA64 already implement VP dom0? are there any plans for x86(-64) VP dom0?

No, Xen/ia64 domain0 has always been P==M, though some hypervisor code written prior to booting on hardware (back when it only ran on a simulator) under an ifdef may be resurrected that supports VP dom0.

Cheers, Muli -- Muli Ben-Yehuda

Thanks! Dan