atom feed28 messages in org.xwiki.users[xwiki-users] problem with Chinese
FromSent OnAttachments
ning wangMar 16, 2007 7:32 am.html
Sergiu DumitriuMar 16, 2007 8:00 am.htm, .sql
Vincent MassolMar 16, 2007 8:24 am.htm
ning wangMar 16, 2007 8:27 am.html
Sergiu DumitriuMar 16, 2007 8:42 am.htm
Vincent MassolMar 16, 2007 8:44 am.html
ning wangMar 16, 2007 9:10 am.htm
ning wangMar 16, 2007 9:12 am.html
Gilles SerassetMar 16, 2007 9:24 am.htm
Sergiu DumitriuMar 16, 2007 9:30 am.html
Sergiu DumitriuMar 16, 2007 10:03 am.htm
jimmy at comkraft.comMar 16, 2007 10:12 am 
ning wangMar 16, 2007 10:17 am.html
Gilles SerassetMar 16, 2007 10:18 am.html
ning wangMar 16, 2007 10:38 am.htm
ning wangMar 16, 2007 11:27 am.html
ning wangMar 19, 2007 9:32 am.html
beyMar 19, 2007 12:28 pm.htm
Youcef BEYMar 19, 2007 4:23 pm.html
Jackson WangMar 20, 2007 7:55 pm.htm, .properties, .png
ning wangMar 20, 2007 8:03 pm.html
Vincent MassolMar 21, 2007 12:08 am.htm
Sergiu DumitriuMar 21, 2007 7:27 am.html
ning wangMar 21, 2007 10:15 am.htm
Sergiu DumitriuMar 21, 2007 10:25 am.htm
Jackson WangMar 22, 2007 2:36 am.html
ning wangMar 22, 2007 8:38 am.htm, .jpg
ning wangMar 22, 2007 8:51 am.htm
Subject:[xwiki-users] problem with Chinese
From:Sergiu Dumitriu (
Date:Mar 16, 2007 10:03:46 am
attachment.htm - 4k

On 3/16/07, Gilles Serasset <Gilles.Serasset at> wrote:

Hi Ning, I also have a utf8 instance and I cannot reproduce your bug even when mixing French and Japanese characters. I'm using postgres though, but mysql qhould work also.

However, I can reproduce this, on a fully UTF-8 system: OS, disk, database, XWiki settings.

It seems that when working with the document archive the characters are broken. In the database everything is OK, so it's something internal.

When did you set up your wiki to use UTF8 characters ? Maybe you created the

database with an ISO Latin 1 encoding and then change your wiki encoding after the DB creation ?

What happens when you ask to see the modifications between the current version and an older one ? Do you correctly see the chinese chars of the older version ?

And by the way, Sergiu, I think the parameter are just useless (as far as I could test) as the velocity macros are called using Strings and not using InputStreams, in fact, it could hurt a little when reading the xxx.vm files that are not encoded in UTF-8 (see discussion on the encoding of the code base...).

Indeed, I did not notice anything when changing this. But I don't know what internal stuff might break if I don't set this.

I think that this only affect the way .vm files stored on the disk are read, but these files are plain ASCII.

Regards, Gilles,

On 16 mars 07, at 16:27, ning wang wrote:

I do everything you said. And I make sure that all encoding is UTF-8 or utf8. But I'm sad to find that it does not work, still "?????" when roll back.

2007/3/16, Sergiu Dumitriu < sergiu.dumitriu at>:

Some ideas:

* Check that the tables are really using UTF-8: get a mysql shell, open the xwiki database and type "select charset(xwd_content from xwikidoc;". If it does not say UTF-8, then you need to change the encoding. I've attached a script that tries to do that (and creates indexes, too)

* Check that the mysql connections are set to UTF-8. "show variables like 'character_set%';". If it says something else, you can either globally set the mysql encoding to utf-8 (in the mysql configuration file, search the mysql documentation), or configure hibernate to request this: <xwiki-dir>/WEB-INF/hibernate.cfg.xml, add <property name="connection.characterEncoding">utf8</property> after: <property name="connection.driver_class">com.mysql.jdbc.Driver</property>

* Check that you set all the encoding parameters to UTF-8: ** in WEB-INF/xwiki.cfg: xwiki.encoding=UTF-8 ** in WEB-INF/web.xml: <param-name>encoding</param-name> <param-value>UTF-8</param-value> ** in WEB-INF/ input.encoding=UTF-8 output.encoding=UTF-8

After this, it should work just fine.