|Matthew Dillon||Jun 23, 2000 11:36 am|
|David Greenman||Jun 24, 2000 9:10 pm|
|Matthew Dillon||Jun 25, 2000 10:24 am|
|Jake Burkholder||Jun 25, 2000 11:19 am|
|Chuck Paterson||Jun 26, 2000 9:50 am|
|Matthew Dillon||Jun 27, 2000 10:41 am|
|Chuck Paterson||Jun 27, 2000 11:19 am|
|Kevin Van Maren||Jun 29, 2000 1:52 pm|
|Greg Lehey||Jul 2, 2000 6:51 pm|
|Jason Evans||Jul 3, 2000 8:41 am|
|Jake Burkholder||Jul 3, 2000 1:14 pm|
|Jake Burkholder||Jul 5, 2000 2:23 am|
|Matthew Dillon||Jul 5, 2000 9:43 am|
|Chuck Paterson||Jul 5, 2000 9:52 am|
|Chuck Paterson||Jul 5, 2000 9:57 am|
|Luoqi Chen||Jul 5, 2000 10:29 am|
|Matthew Dillon||Jul 5, 2000 11:28 am|
|Greg Lehey||Jul 5, 2000 4:20 pm|
|Doug Rabson||Jul 6, 2000 1:26 am|
|Matthew Dillon||Jul 6, 2000 10:43 am|
|Greg Lehey||Jul 22, 2000 2:26 am|
|Matthew Dillon||Jul 22, 2000 9:20 am|
|Chuck Paterson||Jul 22, 2000 9:57 am|
|Bruce Evans||Jul 22, 2000 10:36 am|
|Greg Lehey||Jul 23, 2000 8:14 pm|
|Bruce Evans||Jul 24, 2000 1:08 am|
|Chuck Paterson||Jul 24, 2000 6:43 am|
|Matthew Dillon||Jul 24, 2000 9:16 am|
|Subject:||Re: : ipending (was: SMP progress (was: Stepping on Toes))|
|From:||Bruce Evans (bd...@zeta.org.au)|
|Date:||Jul 24, 2000 1:08:56 am|
On Mon, 24 Jul 2000, Greg Lehey wrote:
On Sunday, 23 July 2000 at 3:36:39 +1000, Bruce Evans wrote:
On Sat, 22 Jul 2000, Matthew Dillon wrote:
I think you still have to mask level interrupts, otherwise you won't be able to sti. Some subsystems may generate a phenominal number of interrupts while the interrupt routine is running -- for example, ^^^ another the serial ports.
E.g., one serial port interrupting every 87 usec gives about 50 interrupts while the keyboard interrupt handler is busy-waiting to program the keyboard LEDs.
Ugh. Are keyboards that slow? But is that such a big problem? The
Yes, keyboards are slow, and keyboard LED programming is sloppy. All PIO devices doing large i/o's or busy waiting have the same problem (perhaps for only hundreds of usec instead of several msec). Other examples: IDE drives in PIO mode and the printer driver.
interrupts occur at the same rate when no keyboard interrupt is running. The interrupt only schedules the handler thread. If it's already scheduled, it's very fast.
The point is that we must do lots of fiddling with the PIC to allow the keyboard interrupt handler/thread to be interrupted at all.
On Saturday, 22 July 2000 at 10:57:40 -0600, Chuck Paterson wrote:
When I read you mail I didn't answer because what you said sounded right. But, what was I thinking. The short answer is Matt is correct. ... Matt's comment about bunches of extra level triggered interrupts being a problem is something that is going to have to be looked into with BSD/OS. The reason we don't mask the interrupts now is that doing the actual masking operation is soooo expensive. I suspect we will want to mark which edge triggered interrupts we level? which to mask.
FreeBSD does some lazy masking of interrupts in the SMP case. I think this amounts to detecting which interrupts should be masked, masking them after they repeat, and then forgetting that they should have been masked before they repeat.
My understanding is that the threaded interrupt stuff splits interrupts (potentially) into two separate processing steps:
1. Receiving the interrupt, handling the PIC/APIC, and scheduling an interrupt thread.
2. Processing the interrupt thread.
I understand that we would do (1) immediately and (2) when we have time for it, but as quickly as possible. Sure, while we're handling a slow operation like the keyboard LEDs, a lot of other interrupts can come in. But the interrupt rate won't change, the serial interrupts still come in at 11.5 kHz. All that happens there is that step (1) will be executed each time, just like it would be at any other time.
Except for fast interrupts, up to 99% of the overhead for step (1) is currently avoided by not touching the PIC. This shouldn't change (on i386's). This is not very portable. I'm not sure about the APIC case, and i386-PIC style fast interrupts are not implementable in general. E.g., suppose interrupts masks are actually levels and the clock interrupt is hardwired to a level higher than that of serial interrupts. Then fast (serial) interrupts can't interrupt slow (clock) interrupts. This is essentially the default state for the PIC, with the hardwiring especially braindamaged, and we fiddle with the PIC masks to get away from it.
The other issue is that we should potentially be able to handle one interrupt in one processor and another in a different processor. I know that there are problems in this attitude, and that we have deferred it, but I can't see how this can work at all if we mask the interrupts.
Yes, concurrency is limited by the number of hardware interrupt priority levels or bits in the hardware interrupt masks.
Looking at the code we have at the moment, the "slow" interrupts write two values to the PIC, one to set the mask and one to EOI, before
More precisely: - one to set the mask in the slave PIC if irq >= 8 - one to set the mask in the master PIC - one to send EOI to the slave PIC if irq >= 8 and not AUTO_EOI_2 - one to send EOI to the master PIC if not AUTO_EOI_1
processing the interrupt, and another after the interrupt to unmask the interrupt:
More precisely: - one to clear the mask in the slave PIC if irq >= 8 - one to clear the mask in the master PIC
This happens on every interrupt, and as Bruce points out, the writes take a long time. By comparison, the code I have now is much shorter and contains only the minimum 1 outb instruction:
Xintr3: pushl $0 pushl $0 pushal pushl %ds pushl %es pushl %fs mov $0x10 ,%ax mov %ax,%ds mov %ax,%es mov %ax,%fs movb $0x20 ,%al outb %al,$0x020 incb intr_nesting_level Xresume3: pushl 3 IRQ number sti call sched_ithd jmp doreti
But this doesn't actually work. It fails in most cases for level triggered interrupts and works in most cases for edge triggered interrupts. With level triggered interrupts that stay on, it gives endless recursion after the "sti". With edge triggered interrupts, the recursion may be limited by the hardware not toggling the irq too often. The lazy masking that I mentioned above avoids the setting the (A)PIC masks initially by detecting the recursion and recovering from it by setting the masks later, etc. It can almost be used to simplify the code as above. Unfortunately, this ends up being more complicated, of course.
To Unsubscribe: send mail to majo...@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message