12 messages in org.w3.public-evangelistRe: japanese encoding nightmare: conc...
FromSent OnAttachments
Paul ArensonNov 12, 2006 5:50 pm 
Karl DubostNov 13, 2006 5:21 am 
Paul ArensonNov 13, 2006 6:25 am 
Paul ArensonNov 13, 2006 6:58 am 
Daniel BarclayNov 13, 2006 10:14 am 
Mike SchinkelNov 13, 2006 3:10 pm 
David DorwardNov 13, 2006 3:17 pm 
Paul ArensonNov 13, 2006 6:06 pm 
Daniel BarclayNov 16, 2006 11:13 am 
Richard IshidaNov 23, 2006 2:15 am 
Tex TexinNov 23, 2006 9:03 am 
Paul ArensonDec 5, 2006 8:07 am 
Actions with this message:
Paste this link in email or IM:
Paste this link in email or IM:
Atom feed for this thread
Paste this URL into your reader:
Subject:Re: japanese encoding nightmare: conclusionActions...
From:Paul Arenson (pa@tokyoprogressive.org)
Date:Dec 5, 2006 8:07:54 am
List:org.w3.public-evangelist

Hi

Way back a month ago I asked a question about why I was able to create a functioning web page on my MAC desktop that showed up wrong on my server,

You might be interested in this report. It shows something very wieird with one machine or program or set of programs ...

It was created as UTF-8 yet one of you mentioned that it was JIS (anyway Japanese).

Well, I found that uploaded to another company's server it was also the same problem. Duplicating the same thing on another Mac as well as a Windows machine I found subsequently that creating a similar file worked on the desktops as well as the servers.

So I again went back to the offending Mac, created a new file, and again the same problem . When i sent that file to myself and pic ked it up on the other Mac (using Email), then uploaded to the web, it was fine.

Conclusion: something on that one mac is corrupting the file.

It is mysterious and never happened before. I tried to download a new version of Mozilla midwat between last time and now and as I recall it did not change things. Shall i conclude that something on my one Mac is corrupting things?

Anyway, your guess is as good as mine, but this does seem to be the problem with one Mac. Would you guess I should reformat the thing, or do you have any idea what might cause the Mac/Mozilla/FTP program (one or all?) to mess up a file?

I can do any tests if anyone is interested.

Thanks

__/__/__/__/__/__/__/__/__/__/ Paul Arenson

EMAIL pa@tokyoprogressive.org

PHONE &VOICE MAIL 1-617-379-0761 (U.S.) 090-4173-3873 (Japan) paularenson (Skype) __/__/__/__/__/__/__/__/__/__/

On Nov 13, 2006, at 11:25 PM, Paul Arenson wrote:

__/__/__/__/__/__/__/__/__/__/ Paul Arenson

EMAIL pa@tokyoprogressive.org

PHONE &VOICE MAIL 1-617-379-0761 (U.S.) 090-4173-3873 (Japan) paularenson (Skype) __/__/__/__/__/__/__/__/__/__/

On Nov 13, 2006, at 10:22 PM, Karl Dubost wrote:

Le 13 nov. 2006 à 10:50, Paul Arenson a écrit :

UNSUCCESSFUL EXAMPLE (Looks ok on desktop but not on server) http://tokyoprogressive.org/why.html

CODE <meta content="text/html; charset=UTF-8" http-equiv="content-type">

but this page is not in utf-8 but in shift-jis

Either you have to save your page as utf-8 or to change the encoding information to <META HTTP-EQUIV="Content-Type" CONTENT="text/ html;charset=Shift_JIS">

It is? I don't recall using that. hmmm. And when i save to desktop, changing to shift jis doesn't help, nor does looking at it on the web. Oh well....

SUCCESSFUL EXAMPLE ONE (JAPANESE COMES OUT RIGHT) http://www.tokyoprogressive.org/index/weblog/print/april-entries/

Yes the page is rightly utf-8. not valid but utf-8 http://validator.w3.org/check?uri=http%3A%2F% 2Fwww.tokyoprogressive.org%2Findex%2Fweblog%2Fprint%2Fapril-entries %2F

Ok.....way back when i used the predecessor to Expression Engine, the encoding was something other than unicode. Then when I upgraded to unicode, I asked the guy who helped me and he changed something in the program or on my server (using the database???). When he did that the new pages, like above, came out good, though old pages did not. perhaps what he did to make Expression Engine work has to do with the server?

As i said, pages look good on my desktop but not on the server....

This was made via EXPRESSION ENGINE

I note I have both xml: lang and uft-8.

xml:lang doesn't influence the display of the page. It is there for example for triggering the right accent when passing the text through a vocal browser. Or to help translation engines (not sure they implement it though). Or to help spelling cheker to choose the right dictionary.

I would recommend that you stick to utf-8, it would help to keep consistency in the way you serve the pages.

I THOUGHT I did this in UFT-8, but no. Mozilla even says it is UFT-8, but as you can see the code is western. In other words, why does it work?

because so browsers try to display wrong pages (invalid, wrong encoding, etc.) then people who develop Web pages do not know that they have done something wrong, and they do not fix it. IMHO it is a mistake from browsers. It is cool to try to recover and display the page, but it is wrong to do silent recovery, as we do not enter in a cycle which help everyone to fix things and have a better experience.

SUCCESSUL EXAMPLE FOUR (most bizarre?) I even forgot to add the meta tag!!! http://tokyoprogressive.org/

The server is sending by default an information which has usually priority other the information contained in the file. The encoding in a file is a guess, and the browser _should_ follow what the servers says.

Yes, I guess in the css? http://tokyoprogressive.org/style.css

But I do not see anything there....hmmmm?

Anyway, I am a bit lost. Is this something that the person who adjusted my database did when he set for Expression Engine and it affects all pages on server?

How do I fix the server (it is a commercial company)...

Thanks!

Make a page in several encodings http://tokyoprogressive.org/a.html <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <meta content="text/html; charset=ISO-2022-JP" LOOKS OK ONLINE

doesn't look ok for me.

but your server is configured in a strange way

GET /a.html HTTP/1.1[CRLF] Host: tokyoprogressive.org[CRLF] Connection: close[CRLF] Accept-Encoding: gzip[CRLF] Accept: text/xml,application/xml,application/xhtml+xml,text/ html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5[CRLF] Accept-Language: fr,en;q=0.9,ja;q=0.9,de;q=0.8,es;q=0.7,it;q=0.7,nl;q=0.6,sv;q=0.5,nb; q=0.5,da;q=0.4,fi;q=0.3,pt;q=0.3,zh-Hans;q=0.2,zh- Hant;q=0.1,ko;q=0.1[CRLF] Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7[CRLF] User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv: 1.8.0.7) Gecko/20060911 Camino/1.0.3 Web-Sniffer/1.0.24[CRLF] Referer: http://web-sniffer.net/[CRLF] [CRLF]

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7[CRLF]

You serve first iso-8859-1 and then utf-8 and then anything. Maybe one of the sources of your problems is there.

1. Change all your pages in one encoding only. utf-8 2. Change the configuration of your server to send only utf-8.

-- Karl Dubost - http://www.w3.org/People/karl/ W3C Conformance Manager, QA Activity Lead QA Weblog - http://www.w3.org/QA/ *** Be Strict To Be Cool ***