On Thu, Sep 4, 2008 at 1:09 PM, chromatic <chro...@wgz.org> wrote:

I think it's a mistake to set this up as just an author-vs-tester zero-sum situation. I see this as a team game where (hopefully) we're all pretty much on the same side in that we want stuff to work.

It shouldn't be any big deal to report a failure -- once -- to an author. That's just the normal bug-report cycle as an author might get from any human user. Author can look into it (if they care to), decide if it's a legitimate bug of theirs or if it's upstream. In some cases "upstream" is the toolchain. In other cases, it's dependencies.

For example, RJBS noted how a CPAN Testers report helped him find a dependency issue: http://use.perl.org/~rjbs/journal/37336

The difference with toolchain-driven failures is that authors often can't really do anything about them directly. They can add workarounds, yes, but not all authors want to spend their time doing that. (Nor should we expect them too, really.) Usually, the easiest answer is for the end-user to upgrade their toolchain.

That's why we need to partition the failure types, so that authors can distinguish between test failures -- which we presume they are likely to be interested in addressing -- instead of PL/make failures, which they may not be interested in addressing. It's not to say that one is less of a problem for end-users. Using UNKNOWN is just convenient at the moment because it exists already and is minimally used.

That said, it should still be responsibility of testers to ensure they have a reasonably sane configuration that could potentially be successful at building and testing a distribution. It does very little good to have a broken CPAN that causes "Build -j3" errors -- no Build.PL could ever succeed and so the fact that a Build.PL dist failed isn't telling us anything valuable about the distribution.

-- David