Remember that my MySQL skills are at the beginner level; and this whole Unicode / utf8 business always gives me a headache.

Any Unicode or utf8 characters with diacritical marks will look funky in DOS.

Normally what I do is take my data, convert it from uft8 to utf8, and see if the results match what I originally had. I'm not sure how reliable this technique is, but it will at least recognize files which have characters that are not utf8-encoded. This is counting on the fact that PHP doesn't recognize this as a "null" conversion.

My particular problem was that I had files that were created in Windows applications, using CP1252 encoding. I needed to get these into utf8, and wanted to test my results. I have a PHP script to do this. My translator seems to work: at least, my results look right after the translation. What disturbs me is that the translated (utf8) files also seem to be CP1252, which seems counter-intuitive. I did this awhile ago, so it may be that CP1252 has alternate encodings that are a superset of utf8 (and I've forgotten).

How are you looking at the data? Try directing some of it from MySQL into a text file, and open the text file with Notepad. It will give you a choice of opening the file as ANSI, Unicode, or utf8. Try opening the file in Unicode: if it is not Unicode, then the letters with diacritical marks should look wrong (or be missing).

If MySQL's engine is as blind as PHP, you can do this in SQL. The problem, as I see it, is that the engine already believes the data is UTF8 so it might not work.

SELECT COUNT(*) FROM table1 WHERE CONVERT(field1 USING utf8) != field1;

Regards,

860.674.8796 / FAX: 860.674.8341