32 messages in org.perl.cpan-testers-discussRe: The relation between CPAN Testers...
FromSent OnAttachments
David GoldenSep 3, 2008 2:09 pm 
Andrew MooreSep 3, 2008 2:28 pm 
David CantrellSep 4, 2008 6:39 am 
David CantrellSep 4, 2008 6:42 am 
David GoldenSep 4, 2008 7:28 am 
Gabor SzaboSep 4, 2008 7:35 am 
Greg Sabino MullaneSep 4, 2008 8:29 am 
Andy LesterSep 4, 2008 8:41 am 
chromaticSep 4, 2008 10:08 am 
Greg Sabino MullaneSep 4, 2008 11:19 am 
David GoldenSep 4, 2008 11:30 am 
Steffen SchwigonSep 4, 2008 11:42 am 
Andy LesterSep 4, 2008 11:52 am 
Andrew MooreSep 4, 2008 12:11 pm 
Andy LesterSep 4, 2008 12:21 pm 
Andrew MooreSep 4, 2008 12:29 pm 
BarbieSep 4, 2008 1:13 pm 
BramSep 4, 2008 1:13 pm 
David GoldenSep 4, 2008 1:56 pm 
Eric WilhelmSep 4, 2008 2:16 pm 
David E. WheelerSep 5, 2008 8:48 am 
David E. WheelerSep 5, 2008 9:07 am 
Andy LesterSep 5, 2008 9:13 am 
David E. WheelerSep 5, 2008 9:17 am 
chromaticSep 5, 2008 10:31 am 
David E. WheelerSep 5, 2008 11:22 am 
chromaticSep 5, 2008 11:35 am 
David E. WheelerSep 5, 2008 11:45 am 
Eric WilhelmSep 5, 2008 11:55 am 
Eric WilhelmSep 5, 2008 2:48 pm 
David GoldenSep 5, 2008 3:22 pm 
Eric WilhelmSep 5, 2008 4:41 pm 
Actions with this message:
Paste this link in email or IM:
Paste this link in email or IM:
Atom feed for this thread
Paste this URL into your reader:
Subject:Re: The relation between CPAN Testers and quality (or why CPAN Testers sucks if you don't need it)Actions...
From:Greg Sabino Mullane (gr@endpoint.com)
Date:Sep 4, 2008 8:29:56 am
List:org.perl.cpan-testers-discuss

Two cents from someone who appreciates the hell out of the CPAN testing service and eagerly awaits new reports every time I release a new version of a module.

However, from author's perspective, if a report is legitimate (and assuming they care), they really only need to hear it once. Having more and more testers sending the same FAIL report on platform X is overkill and gives yet more encouragement for authors to tune out.

So the more successful CPAN Testers is in attracting new testers, the more duplicate FAIL reports authors are likely to receive, which makes them less likely to pay attention to them.

Sorry, but paying attention is the author's job. A fail is something that should be fixed, period, regardless of the number of them. As mentioned elsewhere, the idea of author's "receiving" FAIL reports is outdated anyway: they should be pulling them via a RSS feed.

First, we can lower our collective tolerance of false positives -- for example, stop telling authors to just ignore bogus reports if they don't like it and find ways to filter them.

+1

Second, we can reclassify PL/make/Build fails to UNKNOWN.

I don't like this: failure by any other name would smell just as bad. In other words, if an end user is not going to have a happy, functional module after typing install Foo::Bar at the CPAN prompt, this is a failure that should be noted as such and fixed by the author. Makefiles have a surprising amount of power and flexibility in this regard.

However, as long as the CPAN Testers system has individual testers emailing authors, there is little we can do to address the problem of repetition.

Yep. Use RSS or deal with the duplicates, I say.

For those who read this to the end, thank you for your attention to what is surely becoming a tedious subject.

Thanks for raising it. I honestly feel the problem is not with the testers or the testing service, but the authors. But perhaps I'm still grumpy from the slew of modules I've come across on CPAN lately that are popular yet obviously unmaintained, with bug reports, questions, and unapplied patches that linger in the RT queues for years. It would be nice if we had some sort of system that tracked and reported on that.