On Sat, 20 Aug 2011, Lev Serebryakov wrote:

Part of the key here will be reworking things like ipfw(4) and pf(4) to scale better than they do currently. For pf(4), it's particularly important that we align hardware work distribution via RSS with state management for TCP connections. I've been working on this for the base system TCP implementation over the last few years, and got most of it into 9.x (but not the actual RSS driver interface, as I wasn't convinced it was a stable KPI in the form I prototyped it in). Post-9.0, I'll try to get the RSS KPI cleaned up so that we can merge it and get our device drivers updated.

There's also a related work-in-progress I have that teaches the network stack how to program NIC filters, usually implemented as TCAMs (Chelsio) or hardware hash tables (Solarflare) about network stack connection affinity. My plan is to work on making this substantially more real once the RSS patches are in. (Those are, themselves, fairly minor: we have connection groups already in 9.0, and the RSS changes simply cause existing software-side hash tables to align with hardware-side hashing: the tricky bit is a sustainable KPI for device driver writers).

These are closely related to the issue of userspace networking, which Luigi is starting to explore with netmap. Ideally, you could use the same NIC for both kernel network stack stuff and userspace applications, using hardware filters to decide whether individual packets go to a descriptor ring in the kernel or userspace. Solarflare's Open Onload is an interesting potential model there, although perhaps not the exact model we want (they rely on shared network stacks between kernel and userspace, and for most of our purposes, less sharing is not only sufficient, but perhaps better).

Robert