[MDEV-28512] InnoDB: Assertion failure in file ./storage/innobase/dict/dict0mem.cc line 147 Created: 2022-05-08  Updated: 2023-02-13  Resolved: 2022-05-10

Status: Closed
Project: MariaDB Server
Component/s: Data Definition - Alter Table, Storage Engine - InnoDB
Affects Version/s: 10.5.15
Fix Version/s: 10.3.36, 10.4.26, 10.5.17, 10.6.9, 10.7.5, 10.8.4, 10.9.2, 10.10.1

Type: Bug Priority: Major
Reporter: 3x3 Assignee: Marko Mäkelä
Resolution: Duplicate Votes: 0
Labels: crash
Environment:

Debian GNU/Linux 11

Attachments: PNG File data.png    
Issue Links:
Duplicate
duplicates MDEV-27852 InnoDB: Failing assertion: dict_tf2_i... Closed
Relates
relates to MDEV-26577 InnoDB: Failing assertion: dict_tf2_i... Closed

 Description    

2022-05-08 20:06:22 0x7f95500e8700  InnoDB: Assertion failure in file ./storage/innobase/dict/dict0mem.cc line 147
 
InnoDB: Failing assertion: dict_tf2_is_valid(flags, flags2)
 
InnoDB: We intentionally generate a memory trap.
 
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
 
InnoDB: If you get repeated assertion failures or crashes, even
 
InnoDB: immediately after the mysqld startup, there may be
 
InnoDB: corruption in the InnoDB tablespace. Please refer to
 
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
 
InnoDB: about forcing recovery.
 
220508 20:06:22 [ERROR] mysqld got signal 6 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.5.15-MariaDB-0+deb11u1
 
key_buffer_size=134217728
 
read_buffer_size=131072
 
max_used_connections=3
 
max_threads=153
 
thread_count=3
 
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467872 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x7f94f8000c58
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7f95500e7d78 thread_stack 0x49000
 
??:0(my_print_stacktrace)[0x55e80efd554e]
 
??:0(handle_fatal_signal)[0x55e80ead4f65]
 
??:0(__restore_rt)[0x7f9551f1b140]
 
??:0(gsignal)[0x7f9551a64ce1]
 
??:0(abort)[0x7f9551a4e537]
 
??:0(Wsrep_server_service::log_dummy_write_set(wsrep::client_state&, wsrep::ws_meta const&))[0x55e80e7b6b4c]
 
??:0(Wsrep_server_service::log_dummy_write_set(wsrep::client_state&, wsrep::ws_meta const&))[0x55e80e7c79e0]
 
??:0(wsrep_notify_status(wsrep::server_state::state, wsrep::view const*))[0x55e80edcf927]
 
??:0(wsrep_notify_status(wsrep::server_state::state, wsrep::view const*))[0x55e80edd2c07]
 
??:0(mysql_alter_table(THD*, st_mysql_const_lex_string const*, st_mysql_const_lex_string const*, HA_CREATE_INFO*, TABLE_LIST*, Alter_info*, unsigned int, st_order*, bool, bool))[0x55e80e980737]
 
??:0(Sql_cmd_alter_table::execute(THD*))[0x55e80e9dac3c]
 
??:0(mysql_execute_command(THD*))[0x55e80e8dc356]
 
??:0(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x55e80e8e05db]
 
??:0(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x55e80e8e2a5d]
 
??:0(do_command(THD*))[0x55e80e8e42de]
 
??:0(do_handle_one_connection(CONNECT*, bool))[0x55e80e9d5fb2]
 
??:0(handle_one_connection)[0x55e80e9d622d]
 
??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x55e80ed1211b]
 
??:0(start_thread)[0x7f9551f0fea7]
 
??:0(clone)[0x7f9551b26def]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x7f94f8012990): ALTER TABLE oc_mounts ADD mount_provider_class VARCHAR(128) DEFAULT NULL
 
 
 
Connection ID (thread ID): 144
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off
 
 
 
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
 
information that should help you find out what is causing the crash.
 
Writing a core file...
 
Working directory at /var/lib/mysql
 
Resource Limits:
 
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        0                    unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             39715                39715                processes 
Max open files            32768                32768                files     
Max locked memory         65536                65536                bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       39715                39715                signals   
Max msgqueue size         819200               819200               bytes     
Max nice priority         0                    0                    
Max realtime priority     0                    0                    
Max realtime timeout      unlimited            unlimited            us        
Core pattern: core
 
 
 
2022-05-08 20:06:27 0 [Note] InnoDB: Uses event mutexes
 
2022-05-08 20:06:27 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
 
2022-05-08 20:06:27 0 [Note] InnoDB: Number of pools: 1
 
2022-05-08 20:06:27 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
 
2022-05-08 20:06:28 0 [Note] InnoDB: Using Linux native AIO
 
2022-05-08 20:06:28 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
 
2022-05-08 20:06:28 0 [Note] InnoDB: Completed initialization of buffer pool
 
2022-05-08 20:06:28 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=76260980249,76260980249
 
2022-05-08 20:06:28 0 [Note] InnoDB: 128 rollback segments are active.
 
2022-05-08 20:06:28 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
 
2022-05-08 20:06:28 0 [Note] InnoDB: Creating shared tablespace for temporary tables
 
2022-05-08 20:06:28 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
 
2022-05-08 20:06:28 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
 
2022-05-08 20:06:28 0 [Note] InnoDB: 10.5.15 started; log sequence number 76260980339; transaction id 90339450
 
2022-05-08 20:06:28 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
 
2022-05-08 20:06:28 0 [Note] Server socket created on IP: '127.0.0.1'.
 
2022-05-08 20:06:28 0 [Note] Reading of all Master_info entries succeeded
 
2022-05-08 20:06:28 0 [Note] Added new Master_info '' to hash table
 
2022-05-08 20:06:28 0 [Note] /usr/sbin/mariadbd: ready for connections.

This happens during the Update of Nextcloud.
It seems, there is at least one other User with the same Problem.
AFAIK no other Queries are affected.
I already repaired all tables which where marked as crashed.
Do you need any more information's?

SHOW CREATE TABLE oc_mounts;

CREATE TABLE `oc_mounts` (
 
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
 
  `storage_id` bigint(20) NOT NULL,
 
  `root_id` bigint(20) NOT NULL,
 
  `user_id` varchar(64) COLLATE utf8mb4_bin NOT NULL,
 
  `mount_point` varchar(4000) COLLATE utf8mb4_bin NOT NULL,
 
  `mount_id` bigint(20) DEFAULT NULL,
 
  PRIMARY KEY (`id`),
 
  UNIQUE KEY `mounts_user_root_index` (`user_id`,`root_id`),
 
  KEY `mounts_storage_index` (`storage_id`),
 
  KEY `mounts_root_index` (`root_id`),
 
  KEY `mounts_mount_id_index` (`mount_id`)
 
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin ROW_FORMAT=COMPRESSED

SELECT * FROM oc_mounts;

 Comments   
Comment by Daniel Black [ 2022-05-09 ]

Like MDEV-27852, this might be a mismatch between recording of the format of the oc_mount table.

Can you SHOW TABLE STATUS LIKE 'oc_mount'?

Do you know the history of the data directory that might have got your tables in this state?

ALTER TABLE oc_mount FORCE ALGORITHM=COPY might resolve this for you.

Comment by 3x3 [ 2022-05-09 ]

> Can you SHOW TABLE STATUS LIKE 'oc_mounts'?

MariaDB [c1owncloud]> SHOW TABLE STATUS LIKE 'oc_mounts';
 
+-----------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-------------+----------+-----------------------+---------+------------------+-----------+
 
| Name      | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time         | Update_time | Check_time | Collation   | Checksum | Create_options        | Comment | Max_index_length | Temporary |
 
+-----------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-------------+----------+-----------------------+---------+------------------+-----------+
 
| oc_mounts | InnoDB |      10 | Compact    |    3 |           5461 |       16384 |               0 |        81920 |         0 |              4 | 2021-09-25 05:51:58 | NULL        | NULL       | utf8mb4_bin |     NULL | row_format=COMPRESSED |         |                0 | N         |
 
+-----------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-------------+----------+-----------------------+---------+------------------+-----------+
 
1 row in set (0.001 sec)

> ALTER TABLE oc_mount FORCE ALGORITHM=COPY might resolve this for you.

MariaDB [c1owncloud]> ALTER TABLE oc_mounts FORCE ALGORITHM=COPY;
 
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'ALGORITHM=COPY' at line 1
 
MariaDB [c1owncloud]> ALTER TABLE oc_mounts ALGORITHM=COPY;
 
Query OK, 0 rows affected (0.001 sec)
 
Records: 0  Duplicates: 0  Warnings: 0

The original error persists.
The update of nextcloud fails on the query `ALTER TABLE oc_mounts ADD mount_provider_class VARCHAR(128) DEFAULT NULL`

Comment by 3x3 [ 2022-05-09 ]

What is strange :
`SHOW TABLE STATUS LIKE 'oc_mounts';` shows a count of 3 Rows, but `select * from oc_mounts;` shows a resultset of 2 rows.

Comment by 3x3 [ 2022-05-09 ]

I just created anoter table `oc_mounts_bkp` with the show create table statement of `oc_mounts`.
Copied all data from `oc_mounts` into `oc_mounts_bkp`.
Then: RENAME TABLE oc_mounts TO oc_mounts_old;
Then: RENAME TABLE oc_mounts_bkp TO oc_mounts;

The upgrade of nextcloud have finished successfully

Comment by Daniel Black [ 2022-05-10 ]

So like MDEV-27852, show create table shows ROW_FORMAT=COMPRESSED while show table status shows ROW_FORMAT=Compact.

Apologizes for the syntax error in ALTER TABLE, there's a , between FORCE and ALGORITHM.

The row count of table status is an estimate, so small differences are ok.

Glad you worked out another table re-creation technique.

Note the ROW_FORMAT from nextcloud was troublesome and some defaults changed back to ROW_FORMAT=Dynamic.

Comment by Marko Mäkelä [ 2022-06-27 ]

Fixed in MDEV-26577.

Generated at Thu Feb 08 10:01:19 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.