[MDEV-16150] Mariadb 10.3.6 Failing assertion on Docker Created: 2018-05-12  Updated: 2018-05-14  Resolved: 2018-05-14

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.3.6
Fix Version/s: 10.3.7

Type: Bug Priority: Major
Reporter: Alex Fustini Assignee: Marko Mäkelä
Resolution: Fixed Votes: 0
Labels: None
Environment:

Docker 18.03.1 on windows 10


Attachments: File 000-start-master.sql     File docker-compose.yml     File my.cnf    
Issue Links:
Problem/Incident
is caused by MDEV-14705 systemd: EXTEND_TIMEOUT_USEC= to avoi... Closed
Relates
relates to MDEV-16149 Failing assertion: node->modificatio... Closed

 Description   

I start a new container with Docker compose with Maraidb 10.3.6 and I see all strucutures created on file system (system table), after that the server shut down and this is the report :

2018-05-12  7:54:11 6 [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1146: Table 'mysql.gtid_slave_pos' doesn't exist
2018-05-12 07:54:13 0x7fb4e104f780  InnoDB: Assertion failure in file /home/buildbot/buildbot/build/mariadb-10.3.6/storage/innobase/fil/fil0fil.cc line 759
InnoDB: Failing assertion: node->modification_counter == node->flush_counter || node->space->purpose == FIL_TYPE_TEMPORARY || srv_fast_shutdown == 2 || !srv_was_started
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: https://mariadb.com/kb/en/library/xtradbinnodb-recovery-modes/
InnoDB: about forcing recovery.
180512  7:54:13 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
 
Server version: 10.3.6-MariaDB-1:10.3.6+maria~jessie-log
key_buffer_size=134217728
read_buffer_size=2097152
max_used_connections=0
max_threads=102
thread_count=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 348747 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x49000
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55dd1759cfbe]
/usr/sbin/mysqld(handle_fatal_signal+0x347)[0x55dd1703fe67]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xf890)[0x7fb4e0c30890]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37)[0x7fb4defba067]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x148)[0x7fb4defbb448]
/usr/sbin/mysqld(+0x4c0f76)[0x55dd16d83f76]
/usr/sbin/mysqld(+0xaeff94)[0x55dd173b2f94]
/usr/sbin/mysqld(+0xafcf0e)[0x55dd173bff0e]
/usr/sbin/mysqld(+0x944824)[0x55dd17207824]
/usr/sbin/mysqld(+0xa00d35)[0x55dd172c3d35]
/usr/sbin/mysqld(+0x8e4a76)[0x55dd171a7a76]
/usr/sbin/mysqld(_Z22ha_finalize_handlertonP13st_plugin_int+0x34)[0x55dd170424b4]
/usr/sbin/mysqld(+0x5a9474)[0x55dd16e6c474]
/usr/sbin/mysqld(+0x5ad27e)[0x55dd16e7027e]
/usr/sbin/mysqld(_Z15plugin_shutdownv+0xd5)[0x55dd16e70f85]
/usr/sbin/mysqld(+0x4df8b1)[0x55dd16da28b1]
/usr/sbin/mysqld(unireg_abort+0x103)[0x55dd16da4363]
/usr/sbin/mysqld(_Z11mysqld_mainiPPc+0xf62)[0x55dd16dacf32]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7fb4defa6b45]
/usr/sbin/mysqld(+0x4dbedd)[0x55dd16d9eedd]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Aborted

Installation of system tables failed! Examine the logs in
/var/lib/mysql/ for more information.

The problem could be conflicting information in an external
my.cnf files. You can ignore these by doing:

shell> /usr/bin/mysql_install_db --defaults-file=~/.my.cnf

You can also try to start the mysqld daemon with:

If I try to restart the same server (then without creating mysql data on file system) all works.



 Comments   
Comment by Elena Stepanova [ 2018-05-13 ]

Thanks for the report. It is most likely the same problem as MDEV-16149, only in that case the failure was happening after some heavy load.

At least your version, the problem with bootstrap, seems to have been fixed already in 10.3 tree, by this merge:

commit 99fa7c6c2f9c8d705481bec35f7c4af48848cf70
Merge: 2898c7e b4c5e4a
Author: Marko Mäkelä <marko.makela@mariadb.com>
Date:   Thu Apr 26 22:58:41 2018 +0300
 
    Merge 10.2 into 10.3

Although the exact commit which fixed it is yet unknown.

Comment by Marko Mäkelä [ 2018-05-14 ]

This is a duplicate of MDEV-16149. The only release affected is MariaDB 10.3.6, even though the bug was present in the 10.1 and 10.2 series as well.

This was broken in MDEV-14705 and fixed by me when I noticed it while testing MDEV-12218:

10.2 fix (merged to 10.3)

commit b4c5e4a717e3ce2c2d434106cc74417fe9a1d3dc
Author: Marko Mäkelä <marko.makela@mariadb.com>
Date:   Thu Apr 26 22:46:39 2018 +0300
 
    Follow-up fix to MDEV-14705: Flush log at shutdown
    
    In the merge of commit e7f4e61f6e013e8879a64dae3ab0370ec137a121
    the call fil_flush_file_spaces(FIL_TYPE_LOG) is necessary.
    Tablespaces will be flushed as part of the redo log
    checkpoint, but the redo log will not necessarily
    be flushed, depending on innodb_flush_method.

10.1 fix (present in 10.1.33)

commit a1ea8d6f8117179f412799bde1587c210a319b91
Author: Marko Mäkelä <marko.makela@mariadb.com>
Date:   Thu Apr 26 22:44:09 2018 +0300
 
    Follow-up fix to MDEV-14705: Flush log at shutdown
    
    In commit e7f4e61f6e013e8879a64dae3ab0370ec137a121
    the call fil_flush_file_spaces(FIL_LOG) is necessary.
    Tablespaces will be flushed as part of the redo log
    checkpoint, but the redo log will not necessarily
    be flushed, depending on innodb_flush_method.

There was no release of MariaDB 10.1 or 10.2 series before I fixed this.

Generated at Thu Feb 08 08:26:45 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.