There are problems of cpu starvation with a Many to Many model in that usually (unless you manually step up the concurrency) the Many to Many ends up being Many to Number of CPUs. If Number of CPUs is equal to 1, then usually you will only get one LWP beyond the ones that are sleeping. Using the Solaris libpthread, a user-level thread will not give up the LWP it's using until it voluntarily makes a call into libthread which gives libthread an opportunity to deschedule that thread and schedule a different one.

To illustrate this, I talked to a customer the other day whose application had around 200 threads. He had around 40 LWPs created, but 39 of them were blocked in the kernel, and 1 appeared to be running when looking at the threads in dbx. The application appeared to be hung, though. Finally he figured out that the one thread that was actually running on the one free LWP was in fact running a hugely inefficient code chunk that was simply taking forever to complete, and since it wasn't calling into libthread, it was hogging that cpu and the other 160 or so threads in the program simply were not getting *any* time whatsoever.

The solution to this was either to attempt to set the concurrency manually to something like "a few" LWPs higher than the number he expected to block, or throw in some sched_yield()s into all of his threads to make sure others got some time, or else simply make them bound. I advised him to just make them bound, run some tests to see how this affected his performance, and then decide from there. In all likelihood the majority of his threads can and do block at some point, and I see very little reason if that is the case why he shouldn't just make his threads bound.

I am starting to believe, based on the number of customers I have talked to writing threaded code, that most real-world applications that use threads use threads that sometimes block in the kernel.

That being the case, I honestly see no real reason not to make them bound.

Fact is, once they block, they cause a new LWP to be created anyways

and then basically if all your threads block at least once in a relatively short amount of time you end up with most of your threads having their "own" LWP anyhow, plus you have the additional overhead of the more complicated thread library. In short, for many if not most real-world applications, user-level threads don't really buy you anything.