|Jason Evans||Jun 24, 2000 11:56 pm|
|Daniel Eischen||Jun 25, 2000 6:58 am|
|Terry Lambert||Jun 25, 2000 10:12 am|
|Terry Lambert||Jun 25, 2000 10:36 am|
|Julian Elischer||Jun 25, 2000 10:41 am|
|Poul-Henning Kamp||Jun 25, 2000 11:07 am|
|Nate Williams||Jun 25, 2000 9:41 pm|
|Frank Mayhar||Jun 25, 2000 11:27 pm|
|Frank Mayhar||Jun 25, 2000 11:31 pm|
|Luoqi Chen||Jun 26, 2000 9:46 am|
|Arun Sharma||Jun 26, 2000 9:47 am|
|Jason Evans||Jun 26, 2000 11:06 am|
|Matthew Dillon||Jun 26, 2000 12:26 pm|
|Matthew Dillon||Jun 26, 2000 12:48 pm|
|John Sconiers||Jun 26, 2000 12:56 pm|
|Matthew Dillon||Jun 26, 2000 1:07 pm|
|Luoqi Chen||Jun 26, 2000 1:13 pm|
|Doug Rabson||Jun 26, 2000 1:26 pm|
|Jason Evans||Jun 26, 2000 2:56 pm|
|Jason Evans||Jun 26, 2000 3:14 pm|
|Daniel Eischen||Jun 26, 2000 4:59 pm|
|Luoqi Chen||Jun 26, 2000 7:14 pm|
|Jason Evans||Jun 26, 2000 7:55 pm|
|Joe Eykholt||Jun 26, 2000 8:09 pm|
|Greg Lehey||Jun 27, 2000 8:00 pm|
|Jason Evans||Jun 27, 2000 8:25 pm|
|Daniel Eischen||Jun 27, 2000 8:26 pm|
|Greg Lehey||Jun 27, 2000 9:59 pm|
|Greg Lehey||Jun 27, 2000 10:11 pm|
|Terry Lambert||Jun 28, 2000 4:15 pm|
|Terry Lambert||Jun 28, 2000 4:18 pm|
|Terry Lambert||Jun 28, 2000 4:37 pm|
|Terry Lambert||Jun 28, 2000 4:51 pm|
|Arun Sharma||Jun 28, 2000 9:43 pm|
|Greg Lehey||Jul 2, 2000 7:15 pm|
|Daniel Eischen||Jul 3, 2000 3:23 am|
|Greg Lehey||Jul 3, 2000 3:30 am|
|Jeroen C. van Gelderen||Jul 3, 2000 7:55 am|
|Chuck Paterson||Jul 3, 2000 8:28 am|
|Chuck Paterson||Jul 3, 2000 8:47 am|
|Frank Mayhar||Jul 3, 2000 8:49 am|
|Greg Lehey||Jul 3, 2000 4:08 pm|
|David Scheidt||Jul 3, 2000 4:35 pm|
|Joe Eykholt||Jul 3, 2000 4:47 pm|
|Greg Lehey||Jul 3, 2000 4:52 pm|
|Joe Eykholt||Jul 3, 2000 4:58 pm|
|Greg Lehey||Jul 3, 2000 5:26 pm|
|Joe Eykholt||Jul 3, 2000 5:41 pm|
|Chuck Paterson||Jul 3, 2000 7:17 pm|
|Daniel Eischen||Jul 3, 2000 7:25 pm|
|Daniel Eischen||Jul 3, 2000 7:35 pm|
|Greg Lehey||Jul 3, 2000 7:39 pm|
|Daniel Eischen||Jul 3, 2000 7:41 pm|
|Chuck Paterson||Jul 3, 2000 8:40 pm|
|Alfred Perlstein||Jul 3, 2000 10:08 pm|
|Greg Lehey||Jul 3, 2000 10:37 pm|
|Peter Wemm||Jul 4, 2000 2:43 pm|
|Greg Lehey||Jul 4, 2000 3:58 pm|
|Peter Wemm||Jul 4, 2000 4:06 pm|
|Terry Lambert||Jul 5, 2000 3:38 pm|
|Terry Lambert||Jul 5, 2000 4:00 pm|
|Terry Lambert||Jul 5, 2000 4:06 pm|
|Terry Lambert||Jul 5, 2000 4:10 pm|
|Alfred Perlstein||Jul 5, 2000 4:29 pm|
|Terry Lambert||Jul 6, 2000 4:50 pm|
|Subject:||Re: SMP meeting summary|
|From:||Daniel Eischen (eisc...@vigrid.com)|
|Date:||Jun 25, 2000 6:58:05 am|
On 24 Jun 2000, Jason Evans wrote:
3. The BSD/OS solution.
- The BGL remains, but becomes increasingly meaningless. In particular, it is not always necessary to obtain it in order to enter the kernel.
- Instead the system protects shared data structures with mutexes. These mutexes replace calls to tsleep() when waiting on shared resources (the involuntary process synchronization mentioned above). In contrast to traditional UNIX, mutexes will be used much more frequently in order to protect data structures which were previously implicitly protected by the non-preemptive nature of the kernel. This mechanism will replace calls to tsleep() for involuntary context switches.
Compared with the use of tsleep(), mutexes have a number of advantages:
- Each mutex has its own wait (sleep) queue. When a process releases a mutex, it automatically schedules the next process waiting on the queue. This is more efficient than searching a possibly very long, linear sleep queue. It also avoids the flooding when multiple processes get scheduled, and most of them have to go back to sleep again.
- Mutexes can be a combination of spin and sleep mutexes: for a resource which may be held only for a very short period of time, even the overhead of sleeping and rescheduling may be higher than waiting in a tight loop. A spin/sleep lock might first wait in a tight loop for 2 microseconds and then sleep if the lock is still not available at that time. This is an issue which Sun has investigated in great detail with Solaris. BSDi has not pursued this yet, though the BSD/OS threading primitives make this an easy extention to add. It's possibly an area for us to investigate once the system is up and limping again.
If anyone is interested...
All high-level interrupts (levels 11-15, mostly PIO serial interrupts) in Solaris use spin mutexes and don't use an interrupt thread. They execute in the context of the thread that was currently running. All other interrupts below level 11 (clock, network, disk, etc) use interrupt threads.
A Solaris (non-spinning) mutex will only spin while the owning thread is running. Since BSDi mutexes have owners (correct me if I'm wrong), this seems to be better than arbitrarily spinning.
- Interrupt lockout (spl()s) go away completely. Instead, interrupt functions use mutexes for synchronization. This means that an interrupt function must be capable of blocking, which is currently impossible. In order to block, the function must have a "process" context (a stack and a process control block). In particular, this could include kernel threads.
BSD/OS on Intel currently uses light-weight interrupt threads to process interrupts, while on SPARC uses normal ("heavyweight") processes. Chuck indicated that the decision to implement light-weight threads initially was probably the wrong one, since it gave rise to a large number of problems, and although the heavyweight process model would give lousy performance, it would probably make it easier to develop the kernel while the light-weight processes were being debugged. There is also the possibility of building a kernel with one or the other support, so that in case of problems during development it would be possible to revert to the heavy-weight processes while searching for the bug.
Other details we discussed included:
- BSD/OS has not implemented condition variables. We didn't go into details. The opinion was expressed that they would be very useful for synchronization, but that they require coding discipline somewhat different than the tsleep() mechanism. Converting all use of tsleep() is a lot of work, and of dubious value. However, condition variables can live along with tsleep(), so a full changeover is not necessary.
For a lot of drivers, it seems pretty straight forward to convert splXXX() ... tsleep() ... splx() to mtx_enter() ... cv_wait()/cv_wait_sig() ... mtx_exit().
- BSD/OS also does not implement read/write locks, a thing that Linux has recently introduced. We didn't discuss this further, but the concepts are well understood, and it should be relatively simple to implement them if we find a need.
Mutexes are only used in Solaris when they will be held for very small amounts of time. Read/write locks and semaphores are used for all other instances. While we are modifying the kernel to add mutexes, it would probably be worthwhile to comment those sections of code that could hold mutexes for something other than a very short period of time. Or even use a different naming convention for those mutexes.
-- Dan Eischen
To Unsubscribe: send mail to majo...@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message