[MDEV-16567] rpl.rpl_insert_id_pk failed in buildbot with Failing assertion: ret == 0 (pthread_create failed) Created: 2018-06-25  Updated: 2023-04-27

Status: Confirmed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB, Storage Engine - XtraDB, Tests
Affects Version/s: 10.0, 10.1, 10.2, 10.3, 10.4
Fix Version/s: 10.4

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: crash, startup

Issue Links:
Relates
relates to MDEV-16264 Implement a common work queue for Inn... Closed
relates to MDEV-16264 Implement a common work queue for Inn... Closed
relates to MDEV-18199 [draft] Failing assertion: ret == 0 i... Open

 Description   

http://buildbot.askmonty.org/buildbot/builders/kvm-rpm-fedora28-amd64/builds/141

rpl.rpl_insert_id_pk 'innodb,stmt'       w4 [ fail ]
        Test ended at 2018-06-12 11:44:28
 
CURRENT_TEST: rpl.rpl_insert_id_pk
 
 
Failed to start mysqld.2
mysqltest failed but provided no output

10.2 8f5f0575ab41bd03369ab1dc31425ef7ab1c6b10

CURRENT_TEST: rpl.rpl_insert_id_pk
2018-06-12 11:44:10 139640564107520 [Note] /usr/sbin/mysqld (mysqld 10.2.16-MariaDB-log) starting as process 853 ...
2018-06-12 11:44:10 139640564107520 [Warning] Could not increase number of max_open_files to more than 1024 (request: 4182)
2018-06-12 11:44:10 139640564107520 [Warning] Changed limits: max_open_files: 1024  max_connections: 151 (was 151)  table_cache: 421 (was 2000)
2018-06-12 11:44:10 139640564107520 [Note] Plugin 'SEQUENCE' is disabled.
2018-06-12 11:44:10 139640564107520 [Note] Plugin 'partition' is disabled.
2018-06-12 11:44:10 139640564107520 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-06-12 11:44:10 139640564107520 [Note] InnoDB: Uses event mutexes
2018-06-12 11:44:10 139640564107520 [Note] InnoDB: Compressed tables use zlib 1.2.11
2018-06-12 11:44:10 139640564107520 [Note] InnoDB: Using Linux native AIO
2018-06-12 11:44:10 139640564107520 [Note] InnoDB: Number of pools: 1
2018-06-12 11:44:10 139640564107520 [Note] InnoDB: Using generic crc32 instructions
2018-06-12 11:44:10 139640564107520 [Note] InnoDB: Initializing buffer pool, total size = 8M, instances = 1, chunk size = 8M
2018-06-12 11:44:10 139640564107520 [Note] InnoDB: Completed initialization of buffer pool
2018-06-12 11:44:10 139640283571968 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2018-06-12 11:44:10 139640564107520 [Note] InnoDB: Highest supported file format is Barracuda.
2018-06-12 11:44:10 139640564107520 [Note] InnoDB: 128 out of 128 rollback segments are active.
2018-06-12 11:44:10 139640564107520 [Note] InnoDB: Creating shared tablespace for temporary tables
2018-06-12 11:44:10 139640564107520 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2018-06-12 11:44:11 139640564107520 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2018-06-12 11:44:11 0x7f009a384900  InnoDB: Assertion failure in file /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.2.16/storage/innobase/os/os0thread.cc line 139
InnoDB: Failing assertion: ret == 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: https://mariadb.com/kb/en/library/xtradbinnodb-recovery-modes/
InnoDB: about forcing recovery.
180612 11:44:12 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.2.16-MariaDB-log
key_buffer_size=1048576
read_buffer_size=131072
max_used_connections=0
max_threads=153
thread_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63014 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x49000
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x561d863630ee]
/usr/sbin/mysqld(handle_fatal_signal+0x4eb)[0x561d85e78dfb]
/lib64/libpthread.so.0(+0x11fb0)[0x7f0099f6bfb0]
/lib64/libc.so.6(gsignal+0x10b)[0x7f0097e37f4b]
/lib64/libc.so.6(abort+0x12b)[0x7f0097e22591]
/usr/sbin/mysqld(+0x435335)[0x561d85c19335]
/usr/sbin/mysqld(+0x419fe2)[0x561d85bfdfe2]
/usr/sbin/mysqld(+0x42ee00)[0x561d85c12e00]
/usr/sbin/mysqld(+0x808547)[0x561d85fec547]
/usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x6c)[0x561d85e7b07c]
/usr/sbin/mysqld(+0x511775)[0x561d85cf5775]
/usr/sbin/mysqld(_Z11plugin_initPiPPci+0x97a)[0x561d85cf697a]
/usr/sbin/mysqld(+0x4703a4)[0x561d85c543a4]
/usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x1b16)[0x561d85c59006]
/lib64/libc.so.6(__libc_start_main+0xeb)[0x7f0097e241bb]
/usr/sbin/mysqld(_start+0x2a)[0x561d85c4c7ea]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Writing a core file at /dev/shm/var/4/mysqld.2/data/
CURRENT_TEST: rpl.rpl_insert_id_pk



 Comments   
Comment by Marko Mäkelä [ 2018-06-25 ]

The failing call is pthread_create():

	int	ret = pthread_create(&new_thread_id, &attr, func, arg);
 
	ut_a(ret == 0);

This looks like a resource outage. InnoDB expects to be able to create a number of system threads at startup. We could handle this a little more gracefully, by making os_thread_create_func() return an error, and by refusing InnoDB startup if some thread creation fails. But InnoDB would still crash due to malloc() failure. That would be harder to fix.

In MDEV-16264 we would stop creating so many threads in InnoDB, and rely on a work queue instead.

Comment by Marko Mäkelä [ 2018-06-25 ]

In MariaDB 5.5, InnoDB would call exit() if it fails to create a thread. There would be fewer InnoDB internal threads in that version.

Comment by Marko Mäkelä [ 2019-09-30 ]

This would likely be fixed by MDEV-16264, which will eliminate many InnoDB threads.

Comment by Elena Stepanova [ 2020-05-09 ]

Apparently it hasn't been fixed:
http://buildbot.askmonty.org/buildbot/builders/kvm-rpm-fedora30-amd64/builds/1131

10.4 4d61f1247a1b6a86570cb03d3450930a

rpl.rpl_upgrade_master_info 'stmt'       w2 [ fail ]
        Test ended at 2020-01-30 12:12:53
 
CURRENT_TEST: rpl.rpl_upgrade_master_info
 
 
Server [mysqld.2 - pid: 17179, winpid: 17179, exit: 256] failed during test run
Server log from this test:
----------SERVER LOG START-----------
2020-01-30 12:10:59 365 [Note] Deleted Master_info file '/dev/shm/var/2/mysqld.2/data/master.info'.
2020-01-30 12:10:59 365 [Note] Deleted Master_info file '/dev/shm/var/2/mysqld.2/data/relay-log.info'.
2020-01-30 12:10:59 365 [Note] Master connection name: ''  Master_info_file: 'master.info'  Relay_info_file: 'relay-log.info'
2020-01-30 12:10:59 365 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='16060', master_log_file='', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16060', master_log_file='master-bin.000001', master_log_pos='4'.
2020-01-30 12:10:59 367 [Note] Slave I/O thread: Start asynchronous replication to master 'root@127.0.0.1:16060' in log 'master-bin.000001' at position 4
2020-01-30 12:10:59 368 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 4, relay log './slave-relay-bin.000001' position: 4
2020-01-30 12:10:59 367 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16060',replication started in log 'master-bin.000001' at position 4
2020-01-30 12:10:59 368 [Note] Error reading relay log event: slave SQL thread was killed
2020-01-30 12:10:59 368 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 329
2020-01-30 12:10:59 367 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 329
2020-01-30 12:10:59 369 [Note] Master connection name: ''  Master_info_file: 'master.info'  Relay_info_file: 'relay-log.info'
2020-01-30 12:10:59 369 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='16060', master_log_file='master-bin.000001', master_log_pos='329'. New state master_host='127.0.0.1', master_port='16060', master_log_file='master-bin.000001', master_log_pos='329'.
2020-01-30 12:10:59 369 [Note] Previous Using_Gtid=No. New Using_Gtid=Current_Pos
2020-01-30 12:10:59 0 [Note] /usr/sbin/mysqld (initiated by: root[root] @ localhost [127.0.0.1]): Normal shutdown
2020-01-30 12:10:59 0 [Note] Event Scheduler: Purging the queue. 0 events
2020-01-30 12:10:59 0 [Note] InnoDB: FTS optimize thread exiting.
2020-01-30 12:10:59 0 [Note] InnoDB: Starting shutdown...
2020-01-30 12:10:59 0 [Note] InnoDB: Dumping buffer pool(s) to /dev/shm/var/2/mysqld.2/data/ib_buffer_pool
2020-01-30 12:10:59 0 [Note] InnoDB: Instance 0, restricted to 125 pages due to innodb_buf_pool_dump_pct=25
2020-01-30 12:10:59 0 [Note] InnoDB: Buffer pool(s) dump completed at 200130 12:10:59
2020-01-30 12:11:00 0 [Note] InnoDB: Shutdown completed; log sequence number 68342; transaction id 38
2020-01-30 12:11:00 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2020-01-30 12:11:00 0 [Note] /usr/sbin/mysqld: Shutdown complete
 
2020-01-30 12:11:01 0 [Note] /usr/sbin/mysqld (mysqld 10.4.13-MariaDB-log) starting as process 16655 ...
2020-01-30 12:11:01 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 4186)
2020-01-30 12:11:01 0 [Warning] Changed limits: max_open_files: 1024  max_connections: 151 (was 151)  table_cache: 421 (was 2000)
2020-01-30 12:11:01 0 [Note] Plugin 'partition' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'SEQUENCE' is disabled.
2020-01-30 12:11:01 0 [Note] InnoDB: Using Linux native AIO
2020-01-30 12:11:01 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2020-01-30 12:11:01 0 [Note] InnoDB: Uses event mutexes
2020-01-30 12:11:01 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2020-01-30 12:11:01 0 [Note] InnoDB: Number of pools: 1
2020-01-30 12:11:01 0 [Note] InnoDB: Using generic crc32 instructions
2020-01-30 12:11:01 0 [Note] mysqld: O_TMPFILE is not supported on /dev/shm/var/tmp/2/mysqld.2 (disabling future attempts)
2020-01-30 12:11:01 0 [Note] InnoDB: Initializing buffer pool, total size = 8M, instances = 1, chunk size = 8M
2020-01-30 12:11:01 0 [Note] InnoDB: Completed initialization of buffer pool
2020-01-30 12:11:01 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2020-01-30 12:11:01 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
2020-01-30 12:11:01 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2020-01-30 12:11:01 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2020-01-30 12:11:01 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2020-01-30 12:11:01 0 [Note] InnoDB: Waiting for purge to start
2020-01-30 12:11:01 0 [Note] InnoDB: 10.4.13 started; log sequence number 68342; transaction id 34
2020-01-30 12:11:01 0 [Note] InnoDB: Loading buffer pool(s) from /dev/shm/var/2/mysqld.2/data/ib_buffer_pool
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_SYS_DATAFILES' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_SYS_TABLESTATS' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_LOCKS' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_MUTEXES' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_CMPMEM' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_CMP_PER_INDEX' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_CMP' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_FT_DELETED' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_CMP_RESET' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_LOCK_WAITS' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_TABLESPACES_ENCRYPTION' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_BUFFER_PAGE_LRU' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_SYS_FIELDS' is disabled.
2020-01-30 12:11:01 0 [Note] InnoDB: Buffer pool(s) load completed at 200130 12:11:01
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_CMPMEM_RESET' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'FEEDBACK' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_SYS_COLUMNS' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_FT_INDEX_TABLE' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_CMP_PER_INDEX_RESET' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'user_variables' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_FT_INDEX_CACHE' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_SYS_FOREIGN_COLS' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_FT_BEING_DELETED' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_BUFFER_POOL_STATS' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_TRX' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_SYS_FOREIGN' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_SYS_TABLES' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_FT_DEFAULT_STOPWORD' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_FT_CONFIG' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_BUFFER_PAGE' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_SYS_TABLESPACES' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_METRICS' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_SYS_INDEXES' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_SYS_VIRTUAL' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_TABLESPACES_SCRUBBING' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'INNODB_SYS_SEMAPHORE_WAITS' is disabled.
2020-01-30 12:11:01 0 [Note] Plugin 'unix_socket' is disabled.
2020-01-30 12:11:01 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-feedback-debug-startup-interval=20'
2020-01-30 12:11:01 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-feedback-debug-first-interval=60'
2020-01-30 12:11:01 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-feedback-debug-interval=60'
2020-01-30 12:11:01 0 [Warning] /usr/sbin/mysqld: unknown option '--loose-pam-debug'
2020-01-30 12:11:01 0 [Warning] /usr/sbin/mysqld: unknown option '--loose-aria'
2020-01-30 12:11:01 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-debug-sync-timeout=300'
2020-01-30 12:11:01 0 [Note] Server socket created on IP: '127.0.0.1'.
2020-01-30 12:11:01 0 [Note] Reading of all Master_info entries succeeded
2020-01-30 12:11:01 0 [Note] Added new Master_info '' to hash table
2020-01-30 12:11:01 0 [Note] /usr/sbin/mysqld: ready for connections.
Version: '10.4.13-MariaDB-log'  socket: '/dev/shm/var/tmp/2/mysqld.2.sock'  port: 16061  MariaDB Server
2020-01-30 12:11:02 9 [Note] Master connection name: ''  Master_info_file: 'master.info'  Relay_info_file: 'relay-log.info'
2020-01-30 12:11:02 9 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='3310', master_log_file='mysql-bin.000001', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16060', master_log_file='', master_log_pos='4'.
2020-01-30 12:11:02 9 [Note] Previous Using_Gtid=Current_Pos. New Using_Gtid=Current_Pos
2020-01-30 12:11:02 13 [Note] Slave I/O thread: Start asynchronous replication to master 'root@127.0.0.1:16060' in log '' at position 4
2020-01-30 12:11:02 13 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16060',replication starts at GTID position ''
2020-01-30 12:11:02 14 [Note] Slave SQL thread initialized, starting replication in log 'FIRST' at position 4, relay log './slave-relay-bin.000001' position: 4; GTID position ''
2020-01-30 12:11:02 14 [Note] Error reading relay log event: slave SQL thread was killed
2020-01-30 12:11:02 14 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 680; GTID position '0-1-2'
2020-01-30 12:11:02 13 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 680; GTID position 0-1-2
2020-01-30 12:11:02 0 [Note] /usr/sbin/mysqld (initiated by: root[root] @ localhost [127.0.0.1]): Normal shutdown
2020-01-30 12:11:02 0 [Note] Event Scheduler: Purging the queue. 0 events
2020-01-30 12:11:02 0 [Note] InnoDB: FTS optimize thread exiting.
2020-01-30 12:11:02 0 [Note] InnoDB: Starting shutdown...
2020-01-30 12:11:02 0 [Note] InnoDB: Dumping buffer pool(s) to /dev/shm/var/2/mysqld.2/data/ib_buffer_pool
2020-01-30 12:11:02 0 [Note] InnoDB: Instance 0, restricted to 125 pages due to innodb_buf_pool_dump_pct=25
2020-01-30 12:11:02 0 [Note] InnoDB: Buffer pool(s) dump completed at 200130 12:11:02
2020-01-30 12:11:04 0 [Note] InnoDB: Shutdown completed; log sequence number 68351; transaction id 34
2020-01-30 12:11:04 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2020-01-30 12:11:04 0 [Note] /usr/sbin/mysqld: Shutdown complete
 
2020-01-30 12:11:05 0 [Note] /usr/sbin/mysqld (mysqld 10.4.13-MariaDB-log) starting as process 16773 ...
2020-01-30 12:11:05 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 4186)
2020-01-30 12:11:05 0 [Warning] Changed limits: max_open_files: 1024  max_connections: 151 (was 151)  table_cache: 421 (was 2000)
2020-01-30 12:11:05 0 [Note] Plugin 'partition' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'SEQUENCE' is disabled.
2020-01-30 12:11:05 0 [Note] InnoDB: Using Linux native AIO
2020-01-30 12:11:05 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2020-01-30 12:11:05 0 [Note] InnoDB: Uses event mutexes
2020-01-30 12:11:05 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2020-01-30 12:11:05 0 [Note] InnoDB: Number of pools: 1
2020-01-30 12:11:05 0 [Note] InnoDB: Using generic crc32 instructions
2020-01-30 12:11:05 0 [Note] mysqld: O_TMPFILE is not supported on /dev/shm/var/tmp/2/mysqld.2 (disabling future attempts)
2020-01-30 12:11:05 0 [Note] InnoDB: Initializing buffer pool, total size = 8M, instances = 1, chunk size = 8M
2020-01-30 12:11:05 0 [Note] InnoDB: Completed initialization of buffer pool
2020-01-30 12:11:05 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2020-01-30 12:11:05 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
2020-01-30 12:11:05 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2020-01-30 12:11:05 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2020-01-30 12:11:05 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2020-01-30 12:11:05 0 [Note] InnoDB: Waiting for purge to start
2020-01-30 12:11:05 0 [Note] InnoDB: 10.4.13 started; log sequence number 68351; transaction id 34
2020-01-30 12:11:05 0 [Note] InnoDB: Loading buffer pool(s) from /dev/shm/var/2/mysqld.2/data/ib_buffer_pool
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_SYS_DATAFILES' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_SYS_TABLESTATS' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_LOCKS' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_MUTEXES' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_CMPMEM' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_CMP_PER_INDEX' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_CMP' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_FT_DELETED' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_CMP_RESET' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_LOCK_WAITS' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_TABLESPACES_ENCRYPTION' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_BUFFER_PAGE_LRU' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_SYS_FIELDS' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_CMPMEM_RESET' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'FEEDBACK' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_SYS_COLUMNS' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_FT_INDEX_TABLE' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_CMP_PER_INDEX_RESET' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'user_variables' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_FT_INDEX_CACHE' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_SYS_FOREIGN_COLS' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_FT_BEING_DELETED' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_BUFFER_POOL_STATS' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_TRX' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_SYS_FOREIGN' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_SYS_TABLES' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_FT_DEFAULT_STOPWORD' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_FT_CONFIG' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_BUFFER_PAGE' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_SYS_TABLESPACES' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_METRICS' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_SYS_INDEXES' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_SYS_VIRTUAL' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_TABLESPACES_SCRUBBING' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'INNODB_SYS_SEMAPHORE_WAITS' is disabled.
2020-01-30 12:11:05 0 [Note] Plugin 'unix_socket' is disabled.
2020-01-30 12:11:05 0 [Note] InnoDB: Buffer pool(s) load completed at 200130 12:11:05
2020-01-30 12:11:05 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-feedback-debug-startup-interval=20'
2020-01-30 12:11:05 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-feedback-debug-first-interval=60'
2020-01-30 12:11:05 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-feedback-debug-interval=60'
2020-01-30 12:11:05 0 [Warning] /usr/sbin/mysqld: unknown option '--loose-pam-debug'
2020-01-30 12:11:05 0 [Warning] /usr/sbin/mysqld: unknown option '--loose-aria'
2020-01-30 12:11:05 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-debug-sync-timeout=300'
2020-01-30 12:11:05 0 [Note] Server socket created on IP: '127.0.0.1'.
2020-01-30 12:11:05 0 [Note] Reading of all Master_info entries succeeded
2020-01-30 12:11:05 0 [Note] Added new Master_info '' to hash table
2020-01-30 12:11:05 0 [Note] /usr/sbin/mysqld: ready for connections.
Version: '10.4.13-MariaDB-log'  socket: '/dev/shm/var/tmp/2/mysqld.2.sock'  port: 16061  MariaDB Server
2020-01-30 12:11:05 9 [Note] Master connection name: ''  Master_info_file: 'master.info'  Relay_info_file: 'relay-log.info'
2020-01-30 12:11:05 9 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='3310', master_log_file='mysql-bin.000001', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16060', master_log_file='', master_log_pos='4'.
2020-01-30 12:11:05 9 [Note] Previous Using_Gtid=Current_Pos. New Using_Gtid=Current_Pos
2020-01-30 12:11:05 13 [Note] Slave I/O thread: Start asynchronous replication to master 'root@127.0.0.1:16060' in log '' at position 4
2020-01-30 12:11:05 13 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16060',replication starts at GTID position '0-1-2'
2020-01-30 12:11:05 14 [Note] Slave SQL thread initialized, starting replication in log 'FIRST' at position 4, relay log './slave-relay-bin.000001' position: 4; GTID position '0-1-2'
2020-01-30 12:11:06 14 [Note] Error reading relay log event: slave SQL thread was killed
2020-01-30 12:11:06 14 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 887; GTID position '0-1-3'
2020-01-30 12:11:06 13 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 887; GTID position 0-1-3
2020-01-30 12:11:06 0 [Note] /usr/sbin/mysqld (initiated by: root[root] @ localhost [127.0.0.1]): Normal shutdown
2020-01-30 12:11:06 0 [Note] Event Scheduler: Purging the queue. 0 events
2020-01-30 12:11:06 0 [Note] InnoDB: FTS optimize thread exiting.
2020-01-30 12:11:06 0 [Note] InnoDB: Starting shutdown...
2020-01-30 12:11:06 0 [Note] InnoDB: Dumping buffer pool(s) to /dev/shm/var/2/mysqld.2/data/ib_buffer_pool
2020-01-30 12:11:06 0 [Note] InnoDB: Instance 0, restricted to 125 pages due to innodb_buf_pool_dump_pct=25
2020-01-30 12:11:06 0 [Note] InnoDB: Buffer pool(s) dump completed at 200130 12:11:06
2020-01-30 12:11:06 0 [Note] InnoDB: Shutdown completed; log sequence number 68360; transaction id 34
2020-01-30 12:11:06 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2020-01-30 12:11:06 0 [Note] /usr/sbin/mysqld: Shutdown complete
 
2020-01-30 12:11:07 0 [Note] /usr/sbin/mysqld (mysqld 10.4.13-MariaDB-log) starting as process 16862 ...
2020-01-30 12:11:07 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 4186)
2020-01-30 12:11:07 0 [Warning] Changed limits: max_open_files: 1024  max_connections: 151 (was 151)  table_cache: 421 (was 2000)
2020-01-30 12:11:07 0 [Note] Plugin 'partition' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'SEQUENCE' is disabled.
2020-01-30 12:11:07 0 [Note] InnoDB: Using Linux native AIO
2020-01-30 12:11:07 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2020-01-30 12:11:07 0 [Note] InnoDB: Uses event mutexes
2020-01-30 12:11:07 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2020-01-30 12:11:07 0 [Note] InnoDB: Number of pools: 1
2020-01-30 12:11:07 0 [Note] InnoDB: Using generic crc32 instructions
2020-01-30 12:11:07 0 [Note] mysqld: O_TMPFILE is not supported on /dev/shm/var/tmp/2/mysqld.2 (disabling future attempts)
2020-01-30 12:11:07 0 [Note] InnoDB: Initializing buffer pool, total size = 8M, instances = 1, chunk size = 8M
2020-01-30 12:11:07 0 [Note] InnoDB: Completed initialization of buffer pool
2020-01-30 12:11:07 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2020-01-30 12:11:07 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
2020-01-30 12:11:07 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2020-01-30 12:11:07 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2020-01-30 12:11:07 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2020-01-30 12:11:07 0 [Note] InnoDB: Waiting for purge to start
2020-01-30 12:11:07 0 [Note] InnoDB: 10.4.13 started; log sequence number 68360; transaction id 34
2020-01-30 12:11:07 0 [Note] InnoDB: Loading buffer pool(s) from /dev/shm/var/2/mysqld.2/data/ib_buffer_pool
2020-01-30 12:11:07 0 [Note] InnoDB: Buffer pool(s) load completed at 200130 12:11:07
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_SYS_DATAFILES' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_SYS_TABLESTATS' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_LOCKS' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_MUTEXES' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_CMPMEM' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_CMP_PER_INDEX' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_CMP' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_FT_DELETED' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_CMP_RESET' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_LOCK_WAITS' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_TABLESPACES_ENCRYPTION' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_BUFFER_PAGE_LRU' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_SYS_FIELDS' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_CMPMEM_RESET' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'FEEDBACK' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_SYS_COLUMNS' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_FT_INDEX_TABLE' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_CMP_PER_INDEX_RESET' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'user_variables' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_FT_INDEX_CACHE' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_SYS_FOREIGN_COLS' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_FT_BEING_DELETED' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_BUFFER_POOL_STATS' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_TRX' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_SYS_FOREIGN' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_SYS_TABLES' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_FT_DEFAULT_STOPWORD' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_FT_CONFIG' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_BUFFER_PAGE' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_SYS_TABLESPACES' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_METRICS' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_SYS_INDEXES' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_SYS_VIRTUAL' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_TABLESPACES_SCRUBBING' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'INNODB_SYS_SEMAPHORE_WAITS' is disabled.
2020-01-30 12:11:07 0 [Note] Plugin 'unix_socket' is disabled.
2020-01-30 12:11:07 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-feedback-debug-startup-interval=20'
2020-01-30 12:11:07 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-feedback-debug-first-interval=60'
2020-01-30 12:11:07 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-feedback-debug-interval=60'
2020-01-30 12:11:07 0 [Warning] /usr/sbin/mysqld: unknown option '--loose-pam-debug'
2020-01-30 12:11:07 0 [Warning] /usr/sbin/mysqld: unknown option '--loose-aria'
2020-01-30 12:11:07 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-debug-sync-timeout=300'
2020-01-30 12:11:07 0 [Note] Server socket created on IP: '127.0.0.1'.
2020-01-30 12:11:07 0 [Note] Reading of all Master_info entries succeeded
2020-01-30 12:11:07 0 [Note] Added new Master_info '' to hash table
2020-01-30 12:11:07 0 [Note] /usr/sbin/mysqld: ready for connections.
Version: '10.4.13-MariaDB-log'  socket: '/dev/shm/var/tmp/2/mysqld.2.sock'  port: 16061  MariaDB Server
2020-01-30 12:11:08 9 [Note] Master connection name: ''  Master_info_file: 'master.info'  Relay_info_file: 'relay-log.info'
2020-01-30 12:11:08 9 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='3310', master_log_file='mysql-bin.000001', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16060', master_log_file='', master_log_pos='4'.
2020-01-30 12:11:08 9 [Note] Previous Using_Gtid=Current_Pos. New Using_Gtid=Current_Pos
2020-01-30 12:11:08 13 [Note] Slave I/O thread: Start asynchronous replication to master 'root@127.0.0.1:16060' in log '' at position 4
2020-01-30 12:11:08 13 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16060',replication starts at GTID position '0-1-3'
2020-01-30 12:11:08 14 [Note] Slave SQL thread initialized, starting replication in log 'FIRST' at position 4, relay log './slave-relay-bin.000001' position: 4; GTID position '0-1-3'
2020-01-30 12:11:08 14 [Note] Error reading relay log event: slave SQL thread was killed
2020-01-30 12:11:08 14 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1094; GTID position '0-1-4'
2020-01-30 12:11:08 13 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 1094; GTID position 0-1-4
2020-01-30 12:11:08 0 [Note] /usr/sbin/mysqld (initiated by: root[root] @ localhost [127.0.0.1]): Normal shutdown
2020-01-30 12:11:08 0 [Note] Event Scheduler: Purging the queue. 0 events
2020-01-30 12:11:08 0 [Note] InnoDB: FTS optimize thread exiting.
2020-01-30 12:11:08 0 [Note] InnoDB: Starting shutdown...
2020-01-30 12:11:08 0 [Note] InnoDB: Dumping buffer pool(s) to /dev/shm/var/2/mysqld.2/data/ib_buffer_pool
2020-01-30 12:11:08 0 [Note] InnoDB: Instance 0, restricted to 125 pages due to innodb_buf_pool_dump_pct=25
2020-01-30 12:11:08 0 [Note] InnoDB: Buffer pool(s) dump completed at 200130 12:11:08
2020-01-30 12:11:10 0 [Note] InnoDB: Shutdown completed; log sequence number 68369; transaction id 34
2020-01-30 12:11:10 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2020-01-30 12:11:10 0 [Note] /usr/sbin/mysqld: Shutdown complete
 
2020-01-30 12:11:10 0 [Note] /usr/sbin/mysqld (mysqld 10.4.13-MariaDB-log) starting as process 16963 ...
2020-01-30 12:11:10 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 4186)
2020-01-30 12:11:10 0 [Warning] Changed limits: max_open_files: 1024  max_connections: 151 (was 151)  table_cache: 421 (was 2000)
2020-01-30 12:11:10 0 [Note] Plugin 'partition' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'SEQUENCE' is disabled.
2020-01-30 12:11:10 0 [Note] InnoDB: Using Linux native AIO
2020-01-30 12:11:10 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2020-01-30 12:11:10 0 [Note] InnoDB: Uses event mutexes
2020-01-30 12:11:10 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2020-01-30 12:11:10 0 [Note] InnoDB: Number of pools: 1
2020-01-30 12:11:10 0 [Note] InnoDB: Using generic crc32 instructions
2020-01-30 12:11:10 0 [Note] mysqld: O_TMPFILE is not supported on /dev/shm/var/tmp/2/mysqld.2 (disabling future attempts)
2020-01-30 12:11:10 0 [Note] InnoDB: Initializing buffer pool, total size = 8M, instances = 1, chunk size = 8M
2020-01-30 12:11:10 0 [Note] InnoDB: Completed initialization of buffer pool
2020-01-30 12:11:10 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2020-01-30 12:11:10 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
2020-01-30 12:11:10 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2020-01-30 12:11:10 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2020-01-30 12:11:10 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2020-01-30 12:11:10 0 [Note] InnoDB: Waiting for purge to start
2020-01-30 12:11:10 0 [Note] InnoDB: 10.4.13 started; log sequence number 68369; transaction id 34
2020-01-30 12:11:10 0 [Note] InnoDB: Loading buffer pool(s) from /dev/shm/var/2/mysqld.2/data/ib_buffer_pool
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_SYS_DATAFILES' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_SYS_TABLESTATS' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_LOCKS' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_MUTEXES' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_CMPMEM' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_CMP_PER_INDEX' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_CMP' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_FT_DELETED' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_CMP_RESET' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_LOCK_WAITS' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_TABLESPACES_ENCRYPTION' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_BUFFER_PAGE_LRU' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_SYS_FIELDS' is disabled.
2020-01-30 12:11:10 0 [Note] InnoDB: Buffer pool(s) load completed at 200130 12:11:10
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_CMPMEM_RESET' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'FEEDBACK' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_SYS_COLUMNS' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_FT_INDEX_TABLE' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_CMP_PER_INDEX_RESET' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'user_variables' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_FT_INDEX_CACHE' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_SYS_FOREIGN_COLS' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_FT_BEING_DELETED' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_BUFFER_POOL_STATS' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_TRX' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_SYS_FOREIGN' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_SYS_TABLES' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_FT_DEFAULT_STOPWORD' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_FT_CONFIG' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_BUFFER_PAGE' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_SYS_TABLESPACES' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_METRICS' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_SYS_INDEXES' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_SYS_VIRTUAL' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_TABLESPACES_SCRUBBING' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'INNODB_SYS_SEMAPHORE_WAITS' is disabled.
2020-01-30 12:11:10 0 [Note] Plugin 'unix_socket' is disabled.
2020-01-30 12:11:10 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-feedback-debug-startup-interval=20'
2020-01-30 12:11:10 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-feedback-debug-first-interval=60'
2020-01-30 12:11:10 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-feedback-debug-interval=60'
2020-01-30 12:11:10 0 [Warning] /usr/sbin/mysqld: unknown option '--loose-pam-debug'
2020-01-30 12:11:10 0 [Warning] /usr/sbin/mysqld: unknown option '--loose-aria'
2020-01-30 12:11:10 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-debug-sync-timeout=300'
2020-01-30 12:11:10 0 [Note] Server socket created on IP: '127.0.0.1'.
2020-01-30 12:11:10 0 [Note] Reading of all Master_info entries succeeded
2020-01-30 12:11:10 0 [Note] Added new Master_info '' to hash table
2020-01-30 12:11:10 0 [Note] /usr/sbin/mysqld: ready for connections.
Version: '10.4.13-MariaDB-log'  socket: '/dev/shm/var/tmp/2/mysqld.2.sock'  port: 16061  MariaDB Server
2020-01-30 12:11:12 9 [Note] Master connection name: ''  Master_info_file: 'master.info'  Relay_info_file: 'relay-log.info'
2020-01-30 12:11:12 9 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='3310', master_log_file='mysql-bin.000001', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16060', master_log_file='', master_log_pos='4'.
2020-01-30 12:11:12 9 [Note] Previous Using_Gtid=Current_Pos. New Using_Gtid=Current_Pos
2020-01-30 12:11:12 13 [Note] Slave I/O thread: Start asynchronous replication to master 'root@127.0.0.1:16060' in log '' at position 4
2020-01-30 12:11:12 14 [Note] Slave SQL thread initialized, starting replication in log 'FIRST' at position 4, relay log './slave-relay-bin.000001' position: 4; GTID position '0-1-4'
2020-01-30 12:11:12 13 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16060',replication starts at GTID position '0-1-4'
2020-01-30 12:11:12 14 [Note] Error reading relay log event: slave SQL thread was killed
2020-01-30 12:11:12 14 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1301; GTID position '0-1-5'
2020-01-30 12:11:12 13 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 1301; GTID position 0-1-5
2020-01-30 12:11:13 0 [Note] /usr/sbin/mysqld (initiated by: root[root] @ localhost [127.0.0.1]): Normal shutdown
2020-01-30 12:11:13 0 [Note] Event Scheduler: Purging the queue. 0 events
2020-01-30 12:11:13 0 [Note] InnoDB: FTS optimize thread exiting.
2020-01-30 12:11:13 0 [Note] InnoDB: Starting shutdown...
2020-01-30 12:11:13 0 [Note] InnoDB: Dumping buffer pool(s) to /dev/shm/var/2/mysqld.2/data/ib_buffer_pool
2020-01-30 12:11:13 0 [Note] InnoDB: Instance 0, restricted to 125 pages due to innodb_buf_pool_dump_pct=25
2020-01-30 12:11:13 0 [Note] InnoDB: Buffer pool(s) dump completed at 200130 12:11:13
2020-01-30 12:11:18 0 [Note] InnoDB: Shutdown completed; log sequence number 68378; transaction id 34
2020-01-30 12:11:18 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2020-01-30 12:11:19 0 [Note] /usr/sbin/mysqld: Shutdown complete
 
2020-01-30 12:11:26 0 [Note] /usr/sbin/mysqld (mysqld 10.4.13-MariaDB-log) starting as process 17062 ...
2020-01-30 12:11:26 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 4186)
2020-01-30 12:11:26 0 [Warning] Changed limits: max_open_files: 1024  max_connections: 151 (was 151)  table_cache: 421 (was 2000)
2020-01-30 12:11:26 0 [Note] Plugin 'partition' is disabled.
2020-01-30 12:11:26 0 [Note] Plugin 'SEQUENCE' is disabled.
2020-01-30 12:11:26 0 [Note] InnoDB: Using Linux native AIO
2020-01-30 12:11:26 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2020-01-30 12:11:26 0 [Note] InnoDB: Uses event mutexes
2020-01-30 12:11:26 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2020-01-30 12:11:26 0 [Note] InnoDB: Number of pools: 1
2020-01-30 12:11:26 0 [Note] InnoDB: Using generic crc32 instructions
2020-01-30 12:11:26 0 [Note] mysqld: O_TMPFILE is not supported on /dev/shm/var/tmp/2/mysqld.2 (disabling future attempts)
2020-01-30 12:11:26 0 [Note] InnoDB: Initializing buffer pool, total size = 8M, instances = 1, chunk size = 8M
2020-01-30 12:11:26 0 [Note] InnoDB: Completed initialization of buffer pool
2020-01-30 12:11:26 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2020-01-30 12:11:29 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
2020-01-30 12:11:34 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2020-01-30 12:11:34 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2020-01-30 12:11:35 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2020-01-30 12:11:35 0 [Note] InnoDB: Waiting for purge to start
2020-01-30 12:11:35 0 [Note] InnoDB: 10.4.13 started; log sequence number 68378; transaction id 34
2020-01-30 12:11:35 0 [Note] InnoDB: Loading buffer pool(s) from /dev/shm/var/2/mysqld.2/data/ib_buffer_pool
2020-01-30 12:11:35 0 [Note] InnoDB: Buffer pool(s) load completed at 200130 12:11:35
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_SYS_DATAFILES' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_SYS_TABLESTATS' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_LOCKS' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_MUTEXES' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_CMPMEM' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_CMP_PER_INDEX' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_CMP' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_FT_DELETED' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_CMP_RESET' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_LOCK_WAITS' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_TABLESPACES_ENCRYPTION' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_BUFFER_PAGE_LRU' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_SYS_FIELDS' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_CMPMEM_RESET' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'FEEDBACK' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_SYS_COLUMNS' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_FT_INDEX_TABLE' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_CMP_PER_INDEX_RESET' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'user_variables' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_FT_INDEX_CACHE' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_SYS_FOREIGN_COLS' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_FT_BEING_DELETED' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_BUFFER_POOL_STATS' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_TRX' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_SYS_FOREIGN' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_SYS_TABLES' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_FT_DEFAULT_STOPWORD' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_FT_CONFIG' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_BUFFER_PAGE' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_SYS_TABLESPACES' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_METRICS' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_SYS_INDEXES' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_SYS_VIRTUAL' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_TABLESPACES_SCRUBBING' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'INNODB_SYS_SEMAPHORE_WAITS' is disabled.
2020-01-30 12:11:35 0 [Note] Plugin 'unix_socket' is disabled.
2020-01-30 12:11:35 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-feedback-debug-startup-interval=20'
2020-01-30 12:11:35 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-feedback-debug-first-interval=60'
2020-01-30 12:11:35 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-feedback-debug-interval=60'
2020-01-30 12:11:35 0 [Warning] /usr/sbin/mysqld: unknown option '--loose-pam-debug'
2020-01-30 12:11:35 0 [Warning] /usr/sbin/mysqld: unknown option '--loose-aria'
2020-01-30 12:11:35 0 [Warning] /usr/sbin/mysqld: unknown variable 'loose-debug-sync-timeout=300'
2020-01-30 12:11:35 0 [Note] Server socket created on IP: '127.0.0.1'.
2020-01-30 12:11:35 0 [Note] Reading of all Master_info entries succeeded
2020-01-30 12:11:35 0 [Note] Added new Master_info '' to hash table
2020-01-30 12:11:36 0 [Note] /usr/sbin/mysqld: ready for connections.
Version: '10.4.13-MariaDB-log'  socket: '/dev/shm/var/tmp/2/mysqld.2.sock'  port: 16061  MariaDB Server
2020-01-30 12:11:44 9 [Note] Master connection name: ''  Master_info_file: 'master.info'  Relay_info_file: 'relay-log.info'
2020-01-30 12:11:44 9 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='3310', master_log_file='mysql-bin.000001', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16060', master_log_file='', master_log_pos='4'.
2020-01-30 12:11:44 9 [Note] Previous Using_Gtid=Current_Pos. New Using_Gtid=Current_Pos
2020-01-30 12:11:44 13 [Note] Slave I/O thread: Start asynchronous replication to master 'root@127.0.0.1:16060' in log '' at position 4
2020-01-30 12:11:44 14 [Note] Slave SQL thread initialized, starting replication in log 'FIRST' at position 4, relay log './slave-relay-bin.000001' position: 4; GTID position '0-1-5'
2020-01-30 12:11:44 13 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16060',replication starts at GTID position '0-1-5'
2020-01-30 12:11:45 14 [Note] Error reading relay log event: slave SQL thread was killed
2020-01-30 12:11:45 14 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1508; GTID position '0-1-6'
2020-01-30 12:11:45 13 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 1508; GTID position 0-1-6
2020-01-30 12:11:45 0 [Note] /usr/sbin/mysqld (initiated by: root[root] @ localhost [127.0.0.1]): Normal shutdown
2020-01-30 12:11:45 0 [Note] Event Scheduler: Purging the queue. 0 events
2020-01-30 12:11:45 0 [Note] InnoDB: FTS optimize thread exiting.
2020-01-30 12:11:45 0 [Note] InnoDB: Starting shutdown...
2020-01-30 12:11:45 0 [Note] InnoDB: Dumping buffer pool(s) to /dev/shm/var/2/mysqld.2/data/ib_buffer_pool
2020-01-30 12:11:45 0 [Note] InnoDB: Instance 0, restricted to 125 pages due to innodb_buf_pool_dump_pct=25
2020-01-30 12:11:45 0 [Note] InnoDB: Buffer pool(s) dump completed at 200130 12:11:45
2020-01-30 12:11:46 0 [Note] InnoDB: Shutdown completed; log sequence number 68387; transaction id 34
2020-01-30 12:11:46 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2020-01-30 12:11:46 0 [Note] /usr/sbin/mysqld: Shutdown complete
 
2020-01-30 12:12:03 0 [Note] /usr/sbin/mysqld (mysqld 10.4.13-MariaDB-log) starting as process 17181 ...
2020-01-30 12:12:03 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 4186)
2020-01-30 12:12:03 0 [Warning] Changed limits: max_open_files: 1024  max_connections: 151 (was 151)  table_cache: 421 (was 2000)
2020-01-30 12:12:12 0 [Note] Plugin 'partition' is disabled.
2020-01-30 12:12:12 0 [Note] Plugin 'SEQUENCE' is disabled.
2020-01-30 12:12:12 0 [Note] InnoDB: Using Linux native AIO
2020-01-30 12:12:12 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2020-01-30 12:12:12 0 [Note] InnoDB: Uses event mutexes
2020-01-30 12:12:12 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2020-01-30 12:12:12 0 [Note] InnoDB: Number of pools: 1
2020-01-30 12:12:12 0 [Note] InnoDB: Using generic crc32 instructions
2020-01-30 12:12:12 0 [Note] mysqld: O_TMPFILE is not supported on /dev/shm/var/tmp/2/mysqld.2 (disabling future attempts)
2020-01-30 12:12:12 0 [Note] InnoDB: Initializing buffer pool, total size = 8M, instances = 1, chunk size = 8M
2020-01-30 12:12:12 0 [Note] InnoDB: Completed initialization of buffer pool
2020-01-30 12:12:12 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2020-01-30 12:12:18 0x7f675ecb77c0  InnoDB: Assertion failure in file /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.4.13/storage/innobase/os/os0thread.cc line 140
InnoDB: Failing assertion: ret == 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
InnoDB: about forcing recovery.
200130 12:12:18 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.4.13-MariaDB-log
key_buffer_size=1048576
read_buffer_size=131072
max_used_connections=0
max_threads=153
thread_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63532 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x49000
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55740b0e58ae]
/usr/sbin/mysqld(handle_fatal_signal+0x54d)[0x55740ab8986d]
/lib64/libpthread.so.0(+0x12e70)[0x7f675f87ee70]
/lib64/libc.so.6(gsignal+0x145)[0x7f675ee5aeb5]
/lib64/libc.so.6(abort+0x127)[0x7f675ee45895]
/usr/sbin/mysqld(+0x596d7d)[0x55740a86bd7d]
/usr/sbin/mysqld(+0x581e98)[0x55740a856e98]
/usr/sbin/mysqld(+0x59052d)[0x55740a86552d]
/usr/sbin/mysqld(+0xa66be7)[0x55740ad3bbe7]
/usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x6c)[0x55740ab8c66c]
/usr/sbin/mysqld(+0x6aa315)[0x55740a97f315]
/usr/sbin/mysqld(_Z11plugin_initPiPPci+0x87a)[0x55740a9805aa]
/usr/sbin/mysqld(+0x5d1e7d)[0x55740a8a6e7d]
/usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x431)[0x55740a8ad131]
/lib64/libc.so.6(__libc_start_main+0xf3)[0x7f675ee46f33]
/usr/sbin/mysqld(_start+0x2e)[0x55740a8a0dde]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Writing a core file...
Working directory at /dev/shm/var/2/mysqld.2/data
Resource Limits:
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        unlimited            unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             23705                23705                processes 
Max open files            1024                 1024                 files     
Max locked memory         65536                65536                bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       23705                23705                signals   
Max msgqueue size         819200----------SERVER LOG END-------------
mysqltest failed but provided no output
The result from queries just before the failure was:
< snip >
include/rpl_stop_server.inc [server_number=2]
include/rpl_start_server.inc [server_number=2]
connection master;
INSERT INTO t1 VALUES (5);
include/save_master_gtid.inc
connection slave;
CHANGE MASTER TO master_host='127.0.0.1', master_port=SERVER_MYPORT_1;
include/start_slave.inc
include/sync_with_master_gtid.inc
FLUSH NO_WRITE_TO_BINLOG TABLES;
SELECT * FROM t1 ORDER BY a;
a
1
2
3
4
5
include/stop_slave.inc
include/rpl_stop_server.inc [server_number=2]
include/rpl_start_server.inc [server_number=2]

Generated at Thu Feb 08 08:29:53 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.