[MXS-2076] utf8mb4 chars getting mangaled Created: 2018-10-02 Updated: 2018-11-16 Resolved: 2018-11-16 |
|
| Status: | Closed |
| Project: | MariaDB MaxScale |
| Component/s: | N/A |
| Affects Version/s: | 2.2.15 |
| Fix Version/s: | N/A |
| Type: | Bug | Priority: | Major |
| Reporter: | Steven Lewis | Assignee: | markus makela |
| Resolution: | Not a Bug | Votes: | 0 |
| Labels: | need_feedback | ||
| Environment: |
CentOS 7 |
||
| Attachments: |
|
| Sprint: | MXS-SPRINT-70 |
| Description |
|
Hi When I am storing any emoji in the database they are being stored as 0x3F3F3F3F, when we are using maxscale, We have had to switch back to haproxy as it was service affecting, I still have our staging setup using maxscale for testing. |
| Comments |
| Comment by Steven Lewis [ 2018-10-02 ] |
|
As a update, I am noticing that it is mangling the data in both directions, If stored data via haproxy and then fetch via maxscale it mangles the data, but the data is fine in the DB, |
| Comment by markus makela [ 2018-11-08 ] |
|
Possibly a charset issue? Does it happen with all router modules or just readwritesplit? |
| Comment by Steven Lewis [ 2018-11-09 ] |
|
I will have to set some tests up over the weekend to find out. It might be a internal charset issue, as all works well with a direct connection or going through haproxy. |
| Comment by Steven Lewis [ 2018-11-09 ] |
|
The problem is also happening in the readconnroute as well, I thick I have isolated it the the writes only, I need to write a much smaller test case to test to isolated the problem and move other components out of the way within the app to make a better test case. I did tryed and paste the emoji's in jira, but jira does not like it either. |
| Comment by Steven Lewis [ 2018-11-12 ] |
|
I have attached a test written in php here is a result of me running the test. [root@e1b10 tests]# php ./DButf8mb4Test.php I have run the test successfully directly and via haproxy. |
| Comment by markus makela [ 2018-11-15 ] |
|
Tested with MaxScale 2.3.1 and MariaDB 10.2.17 and it seems to work locally. Attached my test script with minor updates to make it work with a replicated cluster. Tested with 2.2.16 as well and it seems to work. Even 2.2.15 seems to work. |
| Comment by Steven Lewis [ 2018-11-15 ] |
|
I am using it in a network environment, with 2.2.16 and MariaDB 10.1.36. I will adjust my setup so I can run your script directly without modification, and then message the results back to you. |
| Comment by markus makela [ 2018-11-15 ] |
|
I'll see if using an older MariaDB version affects it. |
| Comment by markus makela [ 2018-11-15 ] |
|
Note that my version of the script recreates the table every time. This fixed a problem I had where the table definitions weren't updated. |
| Comment by markus makela [ 2018-11-15 ] |
|
Even with MariaDB 10.1.36 it seems to work. I can't seem to be able to reproduce this issue locally with a set of fresh docker MariaDB instances. I only get corrupted output if I go in and manually change the charset for the column. |
| Comment by Steven Lewis [ 2018-11-15 ] |
|
I have run it on a fresh setup and it has worked locally, it worked when using a fetch DB structure within the cluster, but to a legacy structure that has been through a few DB upgrades it is upset, so I am going to be having a look at sorting them out has they are live multi GB tables. I will do some extra testing. |
| Comment by markus makela [ 2018-11-15 ] |
|
I suspect that the table is using the wrong charset which causes the data to get mangled. As to why this only happens through MaxScale, it's hard to say. If you can reproduce it on a fresh setup, please let us know as this would mean that there is something wrong with MaxScale. |
| Comment by markus makela [ 2018-11-16 ] |
|
I'll close this issue but if you manage to reproduce it on a fresh setup, please open it again. Right now it looks like the table definition uses the wrong charset and the results returned are incorrect. |