[MDEV-32277] Migration from MySQL 5.7 to MariaDB 10.6 with partitioned tables fails Created: 2023-09-28  Updated: 2023-09-30

Status: Open
Project: MariaDB Server
Component/s: Partitioning, Upgrades
Affects Version/s: 10.6.11, 10.6.15
Fix Version/s: 10.6

Type: Bug Priority: Major
Reporter: Oli Sennhauser Assignee: Michael Widenius
Resolution: Unresolved Votes: 0
Labels: Migration, json, mysql, partitioning, upgrade
Environment:

Ubuntu 22.04, n.a.


Issue Links:
Blocks
is blocked by MDEV-29253 Detect incompatible MySQL partition s... Closed

 Description   

We had to upgrade a System from MySQL 5.7 to MariaDB 10.6. In general it works but it miserably fails on a system with partitioned tables... Possibly this is NOT supported in the MySQL JSON plug-in. At least I have not seen any notice in the code. This should be at least documented here:
https://mariadb.com/kb/en/mysql_json/
https://mariadb.org/making-mariadb-understand-mysql-json/
https://mariadb.com/kb/en/json-data-type/

Test case:

SQL> CREATE TABLE jsonptntest (
  id INT UNSIGNED NOT NULL AUTO_INCREMENT
, data JSON
, ts TIMESTAMP
, PRIMARY KEY (id, ts)
, INDEX (ts)
) ENGINE = InnoDB
PARTITION BY RANGE ( UNIX_TIMESTAMP(ts) ) (
  PARTITION p_2021 VALUES LESS THAN ( UNIX_TIMESTAMP('2022-01-01 00:00:00') )
, PARTITION p_2022 VALUES LESS THAN ( UNIX_TIMESTAMP('2023-01-01 00:00:00') )
, PARTITION p_2023 VALUES LESS THAN ( UNIX_TIMESTAMP('2024-01-01 00:00:00') )
, PARTITION p_2024 VALUES LESS THAN ( UNIX_TIMESTAMP('2025-01-01 00:00:00') )
, PARTITION p_max  VALUES LESS THAN (MAXVALUE)
);
 
SQL> INSERT INTO jsonptntest VALUES (null, '{"ahv-nr": "1.1.1.1", "data": "Bla"}', '2021-12-31 12:00:00');
SQL> INSERT INTO jsonptntest VALUES (null, '{"ahv-nr": "1.1.1.2", "data": "Bli"}', '2022-12-31 12:00:00');
SQL> INSERT INTO jsonptntest VALUES (null, '{"ahv-nr": "2.1.1.2", "data": "Blub"}', '2023-12-31 12:00:00');
 
SQL> SELECT * FROM jsonptntest;
+----+---------------------------------------+---------------------+
| id | data                                  | ts                  |
+----+---------------------------------------+---------------------+
|  1 | {"data": "Bla", "ahv-nr": "1.1.1.1"}  | 2021-12-31 12:00:00 |
|  2 | {"data": "Bli", "ahv-nr": "1.1.1.2"}  | 2022-12-31 12:00:00 |
|  3 | {"data": "Blub", "ahv-nr": "2.1.1.2"} | 2023-12-31 12:00:00 |
+----+---------------------------------------+---------------------+
 
SQL> SELECT * FROM jsonptntest partition (p_2023);
+----+---------------------------------------+---------------------+
| id | data                                  | ts                  |
+----+---------------------------------------+---------------------+
|  3 | {"data": "Blub", "ahv-nr": "2.1.1.2"} | 2023-12-31 12:00:00 |
+----+---------------------------------------+---------------------+

  • Stop MySQL 5.7
  • Copy over $datadir to MariaDB 10.6
  • Start MariaDB 10.6

shell> mariadb-upgrade --force --user=root
...
test
test.jsonptntest
Error    : Table 'test.jsonptntest' doesn't exist in engine
status   : Operation failed
...
Repairing tables
test.jsonptntest
Error    : Table 'test.jsonptntest' doesn't exist in engine
status   : Operation failed
uninstalling plugin for 'type_mysql_json' data type
Phase 7/7: Running 'FLUSH PRIVILEGES'
OK
 
mariadb-upgrade --force --user=root
test
mariadb-check: Got error: 2013: Lost connection to server during query when executing 'CHECK TABLE ...  FOR UPGRADE'
FATAL ERROR: Upgrade failed

2023-09-28 11:37:23 45 [ERROR] mysqld: Table rebuild required. Please do "ALTER TABLE `test.jsonptntest` FORCE" or dump/reload to fix it!
2023-09-28 11:37:23 45 [ERROR] mysqld: Incorrect information in file: './test/t@00232.frm'
2023-09-28 11:37:32 47 [ERROR] InnoDB: Column last_update in table mysql.innodb_table_stats is BINARY(4) NOT NULL but should be INT UNSIGNED NOT NULL
2023-09-28 11:37:33 47 [ERROR] InnoDB: Column last_update in table mysql.innodb_table_stats is BINARY(4) NOT NULL but should be INT UNSIGNED NOT NULL
230928 11:37:36 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.6.11-MariaDB
key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=1
max_threads=153
thread_count=1
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 345086 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x7fa7e0000c68
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7fab3066acc0 thread_stack 0x49000
addr2line: 'bin/mysqld': No such file
Printing to addr2line failed
bin/mysqld(my_print_stacktrace+0x2e)[0x559e8302e29e]
bin/mysqld(handle_fatal_signal+0x307)[0x559e829f5947]
libc_sigaction.c:0(__restore_rt)[0x7fab3871d520]
addr2line: 'bin/mysqld': No such file
bin/mysqld(_ZN5Field5cloneEP11st_mem_rootP5TABLE+0x19)[0x559e829dd0f9]
bin/mysqld(_Z21open_table_from_shareP3THDP11TABLE_SHAREPK25st_mysql_const_lex_stringjjjP5TABLEbP4ListI6StringE+0x3ba)[0x559e828a248a]
bin/mysqld(_Z10open_tableP3THDP10TABLE_LISTP18Open_table_context+0xb76)[0x559e82767226]
bin/mysqld(_Z11open_tablesP3THDRK14DDL_options_stPP10TABLE_LISTPjjP19Prelocking_strategy+0xf0c)[0x559e8276a2cc]
bin/mysqld(_Z20open_and_lock_tablesP3THDRK14DDL_options_stP10TABLE_LISTbjP19Prelocking_strategy+0x36)[0x559e8276a886]
bin/mysqld(+0x8dbf08)[0x559e828dbf08]
bin/mysqld(+0x8dd36b)[0x559e828dd36b]
bin/mysqld(_ZN19Sql_cmd_check_table7executeEP3THD+0xae)[0x559e828df63e]
bin/mysqld(_Z21mysql_execute_commandP3THDb+0x1a9a)[0x559e827d318a]
bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x20b)[0x559e827d6f3b]
bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x122a)[0x559e827d90fa]
bin/mysqld(_Z10do_commandP3THDb+0x123)[0x559e827da7e3]
bin/mysqld(_Z24do_handle_one_connectionP7CONNECTb+0x187)[0x559e828d10a7]
bin/mysqld(handle_one_connection+0x34)[0x559e828d1344]
bin/mysqld(+0xc62bbc)[0x559e82c62bbc]
nptl/pthread_create.c:442(start_thread)[0x7fab3876fb43]
x86_64/clone3.S:83(__clone3)[0x7fab38801a00]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7fa7e0013d80): CHECK TABLE `jsonptntest`  FOR UPGRADE
 
Connection ID (thread ID): 49
Status: NOT_KILLED
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off
 
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
information that should help you find out what is causing the crash.
Writing a core file...
Working directory at /home/mysql/database/mariadb-106/data
Resource Limits:
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        0                    unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             62212                62212                processes 
Max open files            64186                64186                files     
Max locked memory         2054049792           2054049792           bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       62212                62212                signals   
Max msgqueue size         819200               819200               bytes     
Max nice priority         0                    0                    
Max realtime priority     0                    0                    
Max realtime timeout      unlimited            unlimited            us        
Core pattern: |/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E

Kernel version: Linux version 5.15.0-84-generic (buildd@lcy02-amd64-005) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #93-Ubuntu SMP Tue Sep 5 17:16:10 UTC 2023

SQL> select * from jsonptntest;
ERROR 4161 (HY000): Unknown data type: 'MYSQL_JSON'
SQL> install soname 'type_mysql_json';
Query OK, 0 rows affected (0.007 sec)
 
SQL> SELECT * FROM jsonptntest;
ERROR 1707 (HY000): Table rebuild required. Please do "ALTER TABLE `test.jsonptntest` FORCE" or dump/reload to fix it!
 
SQL> ALTER TABLE `test.jsonptntest` FORCE;
ERROR 1146 (42S02): Table 'test.test.jsonptntest' doesn't exist



 Comments   
Comment by Marko Mäkelä [ 2023-09-28 ]

I think that monty should be interested in this, because he recently implemented MDEV-29253.

If my understanding is correct, MySQL may use a different (binary) representation of JSON data. Or was that starting with MySQL 8.0 only?

Comment by Daniel Black [ 2023-09-29 ]

There is a JSON type plugin type_mysql_json, so this might just need --plugin-load-add=type_mysql_json

It seems to be stable in 10.6, however yes, something better than a crash is desirable.

Generated at Thu Feb 08 10:30:10 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.