On Dec 20, 2004, at 6:44 PM, David Wheeler wrote:

I got him this morning. Here's the discussion:

09:50am] Theory: seen autrijus [09:50am] purl: autrijus was last seen on #p5p 1 hour and 32 minutes ago, saying: rofl [09:50am] autrijus: you seek me? [09:50am] modred: you didn't even need to say his name three times [09:50am] xantus: seek(0) [09:50am] Theory: autrijus: I do! [09:51am] autrijus: pray tell(), why? [09:51am] Theory: autrijus: Are you on perl-qa? [09:51am] autrijus: nay, I am not [09:51am] Theory: autrijus: Let me find an archive link for a unicode & Test::Builder discussion there. [09:51am] Theory: I have a feeling you'll already be familiar with the issue... [09:51am] autrijus: http://www.nntp.perl.org/group/per​l.qa/3404 [09:52am] Theory: autrijus: D'oh! [09:52am] Theory: You're way ahead of me. [09:52am] Theory: autrijus: What do you think? [09:52am] purl: I think Theory should try flossing more often! [09:52am] autrijus: give me a test case that involves big5? [09:52am] Theory vomits on purl [09:53am] Theory: autrijus: Hrm. I'd have to dig up some Big5. [09:53am] Theory: autrijus: The problem won't come up with Big5 unless you binmode Test::Builder's FHs to utf8. [09:54am] Theory: autrijus: Schwern was suggesting that they just always be utf8, but I thought that'd break things when you used non-utf8 characters. [09:54am] autrijus: if it's always in utf8 and you send random binary data there [09:54am] autrijus: it could be unhappy. [09:54am] Theory: autrijus: Exactly, and that's when Schwern threw up his hands in disgust. [09:54am] Theory: seen Schwern [09:54am] purl: Schwern was last seen on #perl 24 minutes ago, saying: crab:  I've read "Naked" [09:55am] Schwern left the chat room. (Ping timeout: 240 seconds) [09:56am] Theory: autrijus: So it seems to me that there needs to be some way to tell Test::Builder what binmode to use on its file handles. [09:56am] autrijus: so T::B uses its own fh [09:56am] autrijus: apart from STDOUT [09:56am] autrijus: and it does not inherid STDOUT's layers. [09:56am] autrijus: is it that? [09:56am] Theory: autrijus: Yes, it dupes STDOUT and STDERR, but the duping doesn't preserve binmode. [09:56am] Theory: right [09:57am] autrijus: so it seems to me that the right fix is for the duping to fix binmode. [09:57am] Theory: It does the duping so it can know what it's outputting to STDOUT (and STDERR) as opposed to what the scripts are outputting. [09:57am] Theory: autrijus: Me too, but I don't know if there's a way to detect the layer assigned to a file handle. Do you? [09:57am] autrijus: I do. [09:57am] uri: Theory: i bet dupping occurs at the system level so binmode (which is perl i/o level) is lost [09:58am] autrijus: I don't want to go there. [09:58am] Theory: uri: Good point. [09:58am] autrijus thinks. [09:58am] Theory: autrijus: Uh, why not? What is it? [09:58am] uri: Theory: perl prolly calls dup() or variant to do it [09:58am] Theory: uri: Yeah [09:59am] uri: and just returns a plain handle. it should be smart about copying i/o flags [09:59am] autrijus: you can use a layer to read io layers. [09:59am] uri: but you can write a smart dup sub [09:59am] autrijus: I don't think there is an api for that. [09:59am] hachi: dup sub! [09:59am] autrijus: and even if we write one, T::B could not ship it [09:59am] autrijus: because it's, well, XS [10:01am] Theory: autrijus: I was afraid of that. [10:01am] autrijus: and sometimes it does not make sense. [10:01am] Theory: autrijus: So the workaround is to have some way to tell Test::Builder what mode to use. [10:01am] autrijus: yes. [10:02am] autrijus: and in this regard I agree your binmode proposal. [10:02am] Theory: autrijus: Okay. [10:02am] Theory: pokes schwern Ovid: Autrijus:  what about using Ingy's Devel::Pointer to get down to that flag info?  It's pure Perl and relatively cross-platform. [10:11am] hachi: you can't actually twiddle the flag with that, I thought [10:12am] Ovid: But you should be able to read them and create a new handle with appropriate flags, yes? [10:12am] Theory: uri: I agree that it should be smarter about copying i/o flags. [10:12am] Theory: Maybe I should mention it on #p5p [10:12am] crab: what are we talking about again? [10:12am] hachi: Ovid: I think the problem is that you can't actually touch the FH that is in question here... otherwise they would just do a binmode() on it [10:13am] Ovid: Oh, I take it back.  Devel::Pointer is Cozens' module.  It's http://search.cpan.org/~ingy/Poin​ter-0.10/ that I meant. [10:13am] Ovid: Yes, but aren't the filehandles getting duped in T::B?  If so, you can read the flags on the handles they're getting duped from. [10:14am] Theory: ovid: Yes, the ideal solution would be for Test::B to detect the flags on the STDERR and STDOUT file handles and copy them to its duped versions. [10:14am] Theory: ovid: Even better would be if Perl did it for you, as uri mentioned. Theory: crab: http://www.nntp.perl.org/group/pe​rl.qa/3404 [10:20am] autrijus: Ovid: nono, it's the layers not the flags [10:20am] autrijus: layers is not flags. [10:20am] autrijus: very different. [10:20am] Ovid: autrijus, I have no idea what that means [10:21am] autrijus: ok. so say you have a ':via(something)' layer [10:21am] autrijus: that layer can't reliably be restored into the ':via(something)' string. [10:21am] autrijus: it's not a simple string [10:21am] autrijus: like a file mode [10:21am] autrijus: it's an object that got associated with the fh. [10:22am] clkao: especially with via::dynamic [10:22am] autrijus: especially with clkao's via::dynamic, yes. [10:22am] Ovid: Aha, I'm reading perldoc -f open and (as usual) learning a few things. [10:22am] autrijus: perldoc perliol as well [10:23am] autrijus: also, each layer keeps state. [10:23am] autrijus: so simply duping the layers may result in state clash [10:24am] autrijus: so that's why perlio layers has a "Dup" method [10:24am] autrijus: which should theoretically be used as part of dup()... [10:24am] autrijus: weird. it says that open with "&" to dup will dup the layers. [10:25am] Theory: Uh, so should open's duping be using that dupe? [10:25am] Theory: autrijus: That doesn't seem to work. [10:25am] autrijus: perlbug!

Anyway, you get the idea. I think what needs to happen is one of the following:

1. Perl gets smarter about duping file handles, so that the dupes get the same i/o layer settings as the handles they dupe.

2. Test::Builder somehow detects the i/o flags on STDERR and STDOUT and applies them to the duped file handles. I have no idea how to do this.

3. Test::Builder (and the modules that depend on it) provides some method for users to tell it what binmodes they're going to use.

I rather like the third suggestion, since the first two would require code like this:

#!/usr/bin/perl -w

use strict; BEGIN { binmode STDERR, ":utf8"; binmode STDOUT ":utf8"; } use Test::More;

That is, I'd have to remember to set the binmode before loading Test::More. Better would be, IMO:

use Test::More tests => 5, binmode => ":utf8";

Is this do-able?

Regards,

David