mysql - Php/ODBC encoding problem

Question

I use ODBC to connect to SQL Server from PHP. In PHP I read some string (nvarchar column) data from SQL Server and then want to insert it to mysql database. When I try to insert such value to mysql database table I get this mysql error:

Incorrect string value: 'xB3xB9ow...' for column 'name' at row 1

For string with all ASCII characters everything is fine, the problem occurs when non-ASCII characters (from some European languages) exist.

So, in more general terms: there is a Unicode string in MS SQL Server database, which is retrieved by PHP trough ODBC. Then it is put in sql insert query (as value for utf-8 varchar column) which is executed for mysql database.

Can someone explain to me what is happening in this situation in terms of encoding? At which step what character encoding convertions may take place?

I use: PHP 5.2.5, MySQL5.0.45-community-nt, MS Sql Server 2005.

PHP have to run on Linux platform.

UPDATE: The error doesn't occur when I call utf8_encode($s) on this string and use that value in mysql insert query, but then the inserted string doesn't display correctly in mysql database (so that utf8 encoding only worked for enforcing proper utf8 string, but it loses correct characters).

Answer 1

First you have the encoding of the DB. Then you have the encoding used by the ODBC client.

If the encoding of your ODBC client connection does not match the one of the DB, the ODBC layer will automatically transcode your data, in some cases.

The trick here is to force the encoding of the ODBC client connection.

For an "all UTF-8" setup :

$conn=odbc_connect(DB_DSN,DB_USR,DB_PWD);
odbc_exec($conn, "SET NAMES 'UTF8'");
odbc_exec($conn, "SET client_encoding='UTF-8'");

// processing here

This works perfectly with PostgreSQL + Php 5.x. The exact syntax and options depends on the DB vendor.

You can find very useful and clear additional info for MySql here : http://dev.mysql.com/doc/refman/5.0/fr/charset-connection.html

hope this helps.