The problem, IMHO, is none of this is in any way:

* documented; * modellable by a user; * explorable by a user (eg by an easy version of schedgraph to explore things in a useful way.

Arnaud raises a valid point - he's given a synthetic benchmark whose numbers are unpredictable. He's asking why. There are plenty of "complex systems interact complexly!" style answers, none of which are in any way useful to an end-user.

Arnaud, have you ever used ktr/sched_graph to look at what's going on? I think it'd be a worthwhile step to begin documenting what's going on here. I'd also suggest (in a completely non-inflammatory way, so you may not be the right person to write it :-) perhaps keeping some kind of blog listing the tests you're doing and what the results of system inspection are. I think that kind of thing would be very very helpful for engineers and users who are looking to get better behaviour in their use case.

This kind of thing is sorely lacking at the moment.

Adrian