[MDEV-30261] mysqld got signal 6 ; Created: 2022-12-18  Updated: 2023-01-26  Resolved: 2023-01-26

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.3.36
Fix Version/s: N/A

Type: Bug Priority: Critical
Reporter: Jean Assignee: Marko Mäkelä
Resolution: Not a Bug Votes: 0
Labels: crash, innodb
Environment:

Linux 4.19.0-21-amd64 #1 SMP Debian 4.19.249-2 (2022-06-30) x86_64 GNU/Linux

Attachments: Text File gdb_output.log    
Issue Links:
Duplicate
is duplicated by MDEV-30100 Assertion `purge_sys.tail.trx_no <= p... Closed

 Description   

Hi,

I am newbie in the matter so first of all my apology if I got some of the tags wrong, and if my description is incomplete.

My database randomly stopped working without me doing something different than what I regularly do. I killed a python script that was sending queries to it, maybe that's what caused the issue, I can't tell for sure.

My main ask is for you to help me recover it fully, or at least some of the tables/schema, if you have the time for it of course

Below are two block of texts, the first is the result of `systemctl status mariadb`, the second is the content of `/var/log/mysql/error.log` :

 mariadb.service - MariaDB 10.3.36 database server
 
   Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
 
   Active: activating (auto-restart) (Result: signal) since Sun 2022-12-18 16:19:40 CET; 2s ago
 
     Docs: man:mysqld(8)
 
           https://mariadb.com/kb/en/library/systemd/
 
  Process: 18940 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS)
 
  Process: 18941 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
 
  Process: 18943 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`cd /usr/bin/..; /usr/bin/galera_recovery`; [ $?
 
  Process: 19101 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=killed, signal=ABRT)
 
  Process: 19140 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
 
  Process: 19143 ExecStartPost=/etc/mysql/debian-start (code=exited, status=0/SUCCESS)
 
 Main PID: 19101 (code=killed, signal=ABRT)
 
   Status: "Taking your SQL requests now..."
 
 
 
Dec 18 16:19:40 v2202207179557195704 systemd[1]: mariadb.service: Failed with result 'signal'.
 



2022-12-18 16:22:26 0 [Note] InnoDB: Using Linux native AIO
 
2022-12-18 16:22:26 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
2022-12-18 16:22:26 0 [Note] InnoDB: Uses event mutexes
 
2022-12-18 16:22:26 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
 
2022-12-18 16:22:26 0 [Note] InnoDB: Number of pools: 1
 
2022-12-18 16:22:26 0 [Note] InnoDB: Using SSE2 crc32 instructions
 
2022-12-18 16:22:27 0 [Note] InnoDB: Initializing buffer pool, total size = 19G, instances = 8, chunk size = 128M
 
2022-12-18 16:22:27 0 [Note] InnoDB: Completed initialization of buffer pool
 
2022-12-18 16:22:27 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
 
2022-12-18 16:22:27 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=3920017074621
 
2022-12-18 16:22:27 0 [Note] InnoDB: Starting final batch to recover 23 pages from redo log.
 
2022-12-18 16:22:28 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
 
2022-12-18 16:22:28 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
 
2022-12-18 16:22:28 0 [Note] InnoDB: Creating shared tablespace for temporary tables
 
2022-12-18 16:22:28 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
 
2022-12-18 16:22:28 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
 
2022-12-18 16:22:28 0 [Note] InnoDB: 10.3.36 started; log sequence number 3920017226827; transaction id 7589834735
 
2022-12-18 16:22:28 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
 
2022-12-18 16:22:28 0x7f154f36b700  InnoDB: Assertion failure in file /build/mariadb-10.3-lMTCRk/mariadb-10.3-10.3.36/storage/innobase/trx/trx0purge.cc line 118
 
InnoDB: Failing assertion: purge_sys.tail.trx_no <= purge_sys.rseg->last_trx_no()
 
InnoDB: We intentionally generate a memory trap.
 
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
 
InnoDB: If you get repeated assertion failures or crashes, even
 
InnoDB: immediately after the mysqld startup, there may be
 
InnoDB: corruption in the InnoDB tablespace. Please refer to
 
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
 
InnoDB: about forcing recovery.
 
221218 16:22:28 [ERROR] mysqld got signal 6 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed,
 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.3.36-MariaDB-0+deb10u2
 
key_buffer_size=134217728
 
read_buffer_size=131072
 
max_used_connections=0
 
max_threads=10002
 
thread_count=4
 
It is possible that mysqld could use up to
 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 22120993 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x7f1540000c08
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7f154f36ad58 thread_stack 0x49000
 
2022-12-18 16:22:28 0 [Note] Plugin 'FEEDBACK' is disabled.
 
2022-12-18 16:22:28 0 [Note] Recovering after a crash using tc.log
 
2022-12-18 16:22:28 0 [Note] Starting crash recovery...
 
2022-12-18 16:22:28 0 [Note] Crash recovery finished.
 
2022-12-18 16:22:28 0 [Note] Server socket created on IP: '::'.
 
2022-12-18 16:22:28 0 [Note] Reading of all Master_info entries succeeded
 
2022-12-18 16:22:28 0 [Note] Added new Master_info '' to hash table
 
2022-12-18 16:22:28 0 [Note] /usr/sbin/mysqld: ready for connections.
 
Version: '10.3.36-MariaDB-0+deb10u2'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Debian 10
 
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x562047420bde]
 
/usr/sbin/mysqld(handle_fatal_signal+0x54d)[0x562046f490bd]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7f1a9d1b9730]
 
linux/raise.c:51(__GI_raise)[0x7f1a9d01e8eb]
 
stdlib/abort.c:81(__GI_abort)[0x7f1a9d009535]
 
/usr/sbin/mysqld(+0x4e9ddb)[0x562046c87ddb]
 
/usr/sbin/mysqld(+0x4e690b)[0x562046c8490b]
 
/usr/sbin/mysqld(+0xa1bcd1)[0x5620471b9cd1]
 
/usr/sbin/mysqld(+0xa1c112)[0x5620471ba112]
 
/usr/sbin/mysqld(+0xa0819b)[0x5620471a619b]
 
nptl/pthread_create.c:487(start_thread)[0x7f1a9d1aefa3]
 
x86_64/clone.S:97(clone)[0x7f1a9d0e006f]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x0): (null)
 
Connection ID (thread ID): 1
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
 
 
 
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
 
information that should help you find out what is causing the crash.
 
 
 
We think the query pointer is invalid, but we will try to print it anyway.
 
Query:
 
 
 
Writing a core file...
 
Working directory at /var/lib/mysql
 
Resource Limits:
 
Limit                     Soft Limit           Hard Limit           Units
 
Max cpu time              unlimited            unlimited            seconds
 
Max file size             unlimited            unlimited            bytes
 
Max data size             unlimited            unlimited            bytes
 
Max stack size            8388608              unlimited            bytes
 
Max core file size        0                    unlimited            bytes
 
Max resident set          unlimited            unlimited            bytes
 
Max processes             96301                96301                processes
 
Max open files            32768                32768                files
 
Max locked memory         65536                65536                bytes
 
Max address space         unlimited            unlimited            bytes
 
Max file locks            unlimited            unlimited            locks
 
Max pending signals       96301                96301                signals
 
Max msgqueue size         819200               819200               bytes
 
Max nice priority         0                    0
 
Max realtime priority     0                    0
 
Max realtime timeout      unlimited            unlimited            us
 
Core pattern: core
 
 
 
Kernel version: Linux version 4.19.0-21-amd64 (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.249-2 (2022-06-30)
 
 
 
2022-12-18 16:22:31 0 [Note] InnoDB: Using Linux native AIO
 
2022-12-18 16:22:31 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
2022-12-18 16:22:31 0 [Note] InnoDB: Uses event mutexes
 
2022-12-18 16:22:31 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
 
2022-12-18 16:22:31 0 [Note] InnoDB: Number of pools: 1
 
2022-12-18 16:22:31 0 [Note] InnoDB: Using SSE2 crc32 instructions
 
2022-12-18 16:22:31 0 [Note] InnoDB: Initializing buffer pool, total size = 19G, instances = 8, chunk size = 128M
 
2022-12-18 16:22:32 0 [Note] InnoDB: Completed initialization of buffer pool
 
2022-12-18 16:22:32 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
 
2022-12-18 16:22:32 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=3920017074621
 
2022-12-18 16:22:32 0 [Note] InnoDB: Starting final batch to recover 23 pages from redo log.
 
2022-12-18 16:22:32 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
 
2022-12-18 16:22:32 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
 
2022-12-18 16:22:32 0 [Note] InnoDB: Creating shared tablespace for temporary tables
 
2022-12-18 16:22:32 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
 
2022-12-18 16:22:32 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
 
2022-12-18 16:22:32 0 [Note] InnoDB: Waiting for purge to start
 
2022-12-18 16:22:32 0x7f21fa023700  InnoDB: Assertion failure in file /build/mariadb-10.3-lMTCRk/mariadb-10.3-10.3.36/storage/innobase/trx/trx0purge.cc line 118
 
InnoDB: Failing assertion: purge_sys.tail.trx_no <= purge_sys.rseg->last_trx_no()
 
InnoDB: We intentionally generate a memory trap.
 
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
 
InnoDB: If you get repeated assertion failures or crashes, even
 
InnoDB: immediately after the mysqld startup, there may be
 
InnoDB: corruption in the InnoDB tablespace. Please refer to
 
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
 
InnoDB: about forcing recovery.
 
221218 16:22:32 [ERROR] mysqld got signal 6 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed,
 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.3.36-MariaDB-0+deb10u2
 
key_buffer_size=134217728
 
read_buffer_size=131072
 
max_used_connections=0
 
max_threads=10002
 
thread_count=4
 
It is possible that mysqld could use up to
 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 22120993 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x7f21ec000c08
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7f21fa022d58 thread_stack 0x49000
 
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55a4d6516bde]
 
/usr/sbin/mysqld(handle_fatal_signal+0x54d)[0x55a4d603f0bd]
 
2022-12-18 16:22:32 0 [Note] InnoDB: 10.3.36 started; log sequence number 3920017228253; transaction id 7589834746
 
2022-12-18 16:22:32 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
 
2022-12-18 16:22:32 0 [Note] Plugin 'FEEDBACK' is disabled.
 
2022-12-18 16:22:32 0 [Note] Recovering after a crash using tc.log
 
2022-12-18 16:22:32 0 [Note] Starting crash recovery...
 
2022-12-18 16:22:32 0 [Note] Crash recovery finished.
 
2022-12-18 16:22:32 0 [Note] Server socket created on IP: '::'.
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7f2747776730]
 
2022-12-18 16:22:32 0 [Note] Reading of all Master_info entries succeeded
 
2022-12-18 16:22:32 0 [Note] Added new Master_info '' to hash table
 
2022-12-18 16:22:32 0 [Note] /usr/sbin/mysqld: ready for connections.
 
Version: '10.3.36-MariaDB-0+deb10u2'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Debian 10
 
linux/raise.c:51(__GI_raise)[0x7f27475db8eb]
 
stdlib/abort.c:81(__GI_abort)[0x7f27475c6535]
 
/usr/sbin/mysqld(+0x4e9ddb)[0x55a4d5d7dddb]
 
/usr/sbin/mysqld(+0x4e690b)[0x55a4d5d7a90b]
 
/usr/sbin/mysqld(+0xa1bcd1)[0x55a4d62afcd1]
 
/usr/sbin/mysqld(+0xa1c112)[0x55a4d62b0112]
 
/usr/sbin/mysqld(+0xa0819b)[0x55a4d629c19b]
 
nptl/pthread_create.c:487(start_thread)[0x7f274776bfa3]
 
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f274769d06f]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x0): (null)
 
Connection ID (thread ID): 1
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
 
 
 
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
 
information that should help you find out what is causing the crash.
 
 
 
We think the query pointer is invalid, but we will try to print it anyway.
 
Query:
 
 
 
Writing a core file...
 
Working directory at /var/lib/mysql
 
Resource Limits:
 
Limit                     Soft Limit           Hard Limit           Units
 
Max cpu time              unlimited            unlimited            seconds
 
Max file size             unlimited            unlimited            bytes
 
Max data size             unlimited            unlimited            bytes
 
Max stack size            8388608              unlimited            bytes
 
Max core file size        0                    unlimited            bytes
 
Max resident set          unlimited            unlimited            bytes
 
Max processes             96301                96301                processes
 
Max open files            32768                32768                files
 
Max locked memory         65536                65536                bytes
 
Max address space         unlimited            unlimited            bytes
 
Max file locks            unlimited            unlimited            locks
 
Max pending signals       96301                96301                signals
 
Max msgqueue size         819200               819200               bytes
 
Max nice priority         0                    0
 
Max realtime priority     0                    0
 
Max realtime timeout      unlimited            unlimited            us
 
Core pattern: core
 
 
 
Kernel version: Linux version 4.19.0-21-amd64 (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.249-2 (2022-06-30)
 
 
 
2022-12-18 16:22:39 0 [Note] InnoDB: Using Linux native AIO
 
2022-12-18 16:22:39 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
2022-12-18 16:22:39 0 [Note] InnoDB: Uses event mutexes
 
2022-12-18 16:22:39 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
 
2022-12-18 16:22:39 0 [Note] InnoDB: Number of pools: 1
 
2022-12-18 16:22:39 0 [Note] InnoDB: Using SSE2 crc32 instructions
 
2022-12-18 16:22:39 0 [Note] InnoDB: Initializing buffer pool, total size = 19G, instances = 8, chunk size = 128M
 
2022-12-18 16:22:39 0 [Note] InnoDB: Completed initialization of buffer pool
 
2022-12-18 16:22:39 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
 
2022-12-18 16:22:39 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=3920017074621
 
2022-12-18 16:22:39 0 [Note] InnoDB: Starting final batch to recover 23 pages from redo log.
 
2022-12-18 16:22:40 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
 
2022-12-18 16:22:40 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
 
2022-12-18 16:22:40 0 [Note] InnoDB: Creating shared tablespace for temporary tables
 
2022-12-18 16:22:40 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
 
2022-12-18 16:22:40 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
 
2022-12-18 16:22:40 0 [Note] InnoDB: 10.3.36 started; log sequence number 3920017229454; transaction id 7589834756
 
2022-12-18 16:22:40 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
 
2022-12-18 16:22:40 0 [Note] Plugin 'FEEDBACK' is disabled.
 
2022-12-18 16:22:40 0 [Note] Recovering after a crash using tc.log
 
2022-12-18 16:22:40 0 [Note] Starting crash recovery...
 
2022-12-18 16:22:40 0 [Note] Crash recovery finished.
 
2022-12-18 16:22:40 0x7f1d4336b700  InnoDB: Assertion failure in file /build/mariadb-10.3-lMTCRk/mariadb-10.3-10.3.36/storage/innobase/trx/trx0purge.cc line 118
 
InnoDB: Failing assertion: purge_sys.tail.trx_no <= purge_sys.rseg->last_trx_no()
 
InnoDB: We intentionally generate a memory trap.
 
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
 
InnoDB: If you get repeated assertion failures or crashes, even
 
InnoDB: immediately after the mysqld startup, there may be
 
InnoDB: corruption in the InnoDB tablespace. Please refer to
 
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
 
InnoDB: about forcing recovery.
 
221218 16:22:40 [ERROR] mysqld got signal 6 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed,
 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.3.36-MariaDB-0+deb10u2
 
key_buffer_size=134217728
 
read_buffer_size=131072
 
max_used_connections=0
 
max_threads=10002
 
thread_count=5
 
It is possible that mysqld could use up to
 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 22120993 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x7f1d34000c08
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7f1d4336ad58 thread_stack 0x49000
 
2022-12-18 16:22:40 0 [Note] Server socket created on IP: '::'.
 
2022-12-18 16:22:40 0 [Note] Reading of all Master_info entries succeeded
 
2022-12-18 16:22:40 0 [Note] Added new Master_info '' to hash table
 
2022-12-18 16:22:40 0 [Note] /usr/sbin/mysqld: ready for connections.
 
Version: '10.3.36-MariaDB-0+deb10u2'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Debian 10
 
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x558f6c856bde]
 
/usr/sbin/mysqld(handle_fatal_signal+0x54d)[0x558f6c37f0bd]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7f2291114730]
 
linux/raise.c:51(__GI_raise)[0x7f2290f798eb]
 
stdlib/abort.c:81(__GI_abort)[0x7f2290f64535]
 
/usr/sbin/mysqld(+0x4e9ddb)[0x558f6c0bdddb]
 
/usr/sbin/mysqld(+0x4e690b)[0x558f6c0ba90b]
 
/usr/sbin/mysqld(+0xa1bcd1)[0x558f6c5efcd1]
 
/usr/sbin/mysqld(+0xa1c112)[0x558f6c5f0112]
 
/usr/sbin/mysqld(+0xa0819b)[0x558f6c5dc19b]
 
nptl/pthread_create.c:487(start_thread)[0x7f2291109fa3]
 
x86_64/clone.S:97(clone)[0x7f229103b06f]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x0): (null)
 
Connection ID (thread ID): 1
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
 
 
 
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
 
information that should help you find out what is causing the crash.
 
 
 
We think the query pointer is invalid, but we will try to print it anyway.
 
Query:
 
 
 
Writing a core file...
 
Working directory at /var/lib/mysql
 
Resource Limits:
 
Limit                     Soft Limit           Hard Limit           Units
 
Max cpu time              unlimited            unlimited            seconds
 
Max file size             unlimited            unlimited            bytes
 
Max data size             unlimited            unlimited            bytes
 
Max stack size            8388608              unlimited            bytes
 
Max core file size        0                    unlimited            bytes
 
Max resident set          unlimited            unlimited            bytes
 
Max processes             96301                96301                processes
 
Max open files            32768                32768                files
 
Max locked memory         65536                65536                bytes
 
Max address space         unlimited            unlimited            bytes
 
Max file locks            unlimited            unlimited            locks
 
Max pending signals       96301                96301                signals
 
Max msgqueue size         819200               819200               bytes
 
Max nice priority         0                    0
 
Max realtime priority     0                    0
 
Max realtime timeout      unlimited            unlimited            us
 
Core pattern: core
 
 
 
Kernel version: Linux version 4.19.0-21-amd64 (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.249-2 (2022-06-30)
 
 
 
2022-12-18 16:22:46 0 [Note] InnoDB: Using Linux native AIO
 
2022-12-18 16:22:46 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
2022-12-18 16:22:46 0 [Note] InnoDB: Uses event mutexes
 
2022-12-18 16:22:46 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
 
2022-12-18 16:22:46 0 [Note] InnoDB: Number of pools: 1
 
2022-12-18 16:22:46 0 [Note] InnoDB: Using SSE2 crc32 instructions
 
2022-12-18 16:22:46 0 [Note] InnoDB: Initializing buffer pool, total size = 19G, instances = 8, chunk size = 128M
 
2022-12-18 16:22:47 0 [Note] InnoDB: Completed initialization of buffer pool
 
2022-12-18 16:22:47 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
 
2022-12-18 16:22:47 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=3920017074621
 
2022-12-18 16:22:47 0 [Note] InnoDB: Starting final batch to recover 23 pages from redo log.
 
2022-12-18 16:22:47 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
 
2022-12-18 16:22:47 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
 
2022-12-18 16:22:47 0 [Note] InnoDB: Creating shared tablespace for temporary tables
 
2022-12-18 16:22:47 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
 
2022-12-18 16:22:47 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
 
2022-12-18 16:22:47 0 [Note] InnoDB: 10.3.36 started; log sequence number 3920017229454; transaction id 7589834756
 
2022-12-18 16:22:47 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
 
2022-12-18 16:22:47 0 [Note] Plugin 'FEEDBACK' is disabled.
 
2022-12-18 16:22:47 0 [Note] Recovering after a crash using tc.log
 
2022-12-18 16:22:47 0 [Note] Starting crash recovery...
 
2022-12-18 16:22:47 0 [Note] Crash recovery finished.
 
2022-12-18 16:22:47 0x7fcce9dc2700  InnoDB: Assertion failure in file /build/mariadb-10.3-lMTCRk/mariadb-10.3-10.3.36/storage/innobase/trx/trx0purge.cc line 118
 
InnoDB: Failing assertion: purge_sys.tail.trx_no <= purge_sys.rseg->last_trx_no()
 
InnoDB: We intentionally generate a memory trap.
 
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
 
InnoDB: If you get repeated assertion failures or crashes, even
 
InnoDB: immediately after the mysqld startup, there may be
 
InnoDB: corruption in the InnoDB tablespace. Please refer to
 
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
 
InnoDB: about forcing recovery.
 
221218 16:22:47 [ERROR] mysqld got signal 6 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed,
 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.3.36-MariaDB-0+deb10u2
 
key_buffer_size=134217728
 
read_buffer_size=131072
 
max_used_connections=0
 
max_threads=10002
 
thread_count=5
 
It is possible that mysqld could use up to
 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 22120993 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x7fccdc000c08
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7fcce9dc1d58 thread_stack 0x49000
 
2022-12-18 16:22:47 0 [Note] Server socket created on IP: '::'.
 
2022-12-18 16:22:47 0 [Note] Reading of all Master_info entries succeeded
 
2022-12-18 16:22:47 0 [Note] Added new Master_info '' to hash table
 
2022-12-18 16:22:47 0 [Note] /usr/sbin/mysqld: ready for connections.
 
Version: '10.3.36-MariaDB-0+deb10u2'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Debian 10
 
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x5617ae3f8bde]
 
/usr/sbin/mysqld(handle_fatal_signal+0x54d)[0x5617adf210bd]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7fd2376b2730]
 
linux/raise.c:51(__GI_raise)[0x7fd2375178eb]
 
stdlib/abort.c:81(__GI_abort)[0x7fd237502535]
 
/usr/sbin/mysqld(+0x4e9ddb)[0x5617adc5fddb]
 
/usr/sbin/mysqld(+0x4e690b)[0x5617adc5c90b]
 
/usr/sbin/mysqld(+0xa1bcd1)[0x5617ae191cd1]
 
/usr/sbin/mysqld(+0xa1c112)[0x5617ae192112]
 
/usr/sbin/mysqld(+0xa0819b)[0x5617ae17e19b]
 
nptl/pthread_create.c:487(start_thread)[0x7fd2376a7fa3]
 
x86_64/clone.S:97(clone)[0x7fd2375d906f]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x0): (null)
 
Connection ID (thread ID): 1
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
 
 
 
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
 
information that should help you find out what is causing the crash.
 
 
 
We think the query pointer is invalid, but we will try to print it anyway.
 
Query:
 
 
 
Writing a core file...
 
Working directory at /var/lib/mysql
 
Resource Limits:
 
Limit                     Soft Limit           Hard Limit           Units
 
Max cpu time              unlimited            unlimited            seconds
 
Max file size             unlimited            unlimited            bytes
 
Max data size             unlimited            unlimited            bytes
 
Max stack size            8388608              unlimited            bytes
 
Max core file size        0                    unlimited            bytes
 
Max resident set          unlimited            unlimited            bytes
 
Max processes             96301                96301                processes
 
Max open files            32768                32768                files
 
Max locked memory         65536                65536                bytes
 
Max address space         unlimited            unlimited            bytes
 
Max file locks            unlimited            unlimited            locks
 
Max pending signals       96301                96301                signals
 
Max msgqueue size         819200               819200               bytes
 
Max nice priority         0                    0
 
Max realtime priority     0                    0
 
Max realtime timeout      unlimited            unlimited            us
 
Core pattern: core
 
 
 
Kernel version: Linux version 4.19.0-21-amd64 (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.249-2 (2022-06-30)
 
 
 
2022-12-18 16:22:54 0 [Note] InnoDB: Using Linux native AIO
 
2022-12-18 16:22:54 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
2022-12-18 16:22:54 0 [Note] InnoDB: Uses event mutexes
 
2022-12-18 16:22:54 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
 
2022-12-18 16:22:54 0 [Note] InnoDB: Number of pools: 1
 
2022-12-18 16:22:54 0 [Note] InnoDB: Using SSE2 crc32 instructions
 
2022-12-18 16:22:54 0 [Note] InnoDB: Initializing buffer pool, total size = 19G, instances = 8, chunk size = 128M
 
2022-12-18 16:22:54 0 [Note] InnoDB: Completed initialization of buffer pool
 
2022-12-18 16:22:54 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
 
2022-12-18 16:22:55 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=3920017074621
 
2022-12-18 16:22:55 0 [Note] InnoDB: Starting final batch to recover 23 pages from redo log.
 
2022-12-18 16:22:56 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
 
2022-12-18 16:22:56 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
 
2022-12-18 16:22:56 0 [Note] InnoDB: Creating shared tablespace for temporary tables
 
2022-12-18 16:22:56 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
 
2022-12-18 16:22:56 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
 
2022-12-18 16:22:56 0 [Note] InnoDB: 10.3.36 started; log sequence number 3920017229454; transaction id 7589834756
 
2022-12-18 16:22:56 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
 
2022-12-18 16:22:56 0 [Note] Plugin 'FEEDBACK' is disabled.
 
2022-12-18 16:22:56 0 [Note] Recovering after a crash using tc.log
 
2022-12-18 16:22:56 0 [Note] Starting crash recovery...
 
2022-12-18 16:22:56 0 [Note] Crash recovery finished.
 
2022-12-18 16:22:56 0 [Note] Server socket created on IP: '::'.
 
2022-12-18 16:22:56 0x7fe5da901700  InnoDB: Assertion failure in file /build/mariadb-10.3-lMTCRk/mariadb-10.3-10.3.36/storage/innobase/trx/trx0purge.cc line 118
 
InnoDB: Failing assertion: purge_sys.tail.trx_no <= purge_sys.rseg->last_trx_no()
 
InnoDB: We intentionally generate a memory trap.
 
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
 
InnoDB: If you get repeated assertion failures or crashes, even
 
InnoDB: immediately after the mysqld startup, there may be
 
InnoDB: corruption in the InnoDB tablespace. Please refer to
 
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
 
InnoDB: about forcing recovery.
 
221218 16:22:56 [ERROR] mysqld got signal 6 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed,
 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.3.36-MariaDB-0+deb10u2
 
key_buffer_size=134217728
 
read_buffer_size=131072
 
max_used_connections=0
 
max_threads=10002
 
thread_count=5
 
It is possible that mysqld could use up to
 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 22120993 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x7fe5d0000c08
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7fe5da900d58 thread_stack 0x49000
 
2022-12-18 16:22:56 0 [Note] Reading of all Master_info entries succeeded
 
2022-12-18 16:22:56 0 [Note] Added new Master_info '' to hash table
 
2022-12-18 16:22:56 0 [Note] /usr/sbin/mysqld: ready for connections.
 
Version: '10.3.36-MariaDB-0+deb10u2'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Debian 10
 
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55d4e1ff5bde]
 
/usr/sbin/mysqld(handle_fatal_signal+0x54d)[0x55d4e1b1e0bd]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7feb289ac730]
 
linux/raise.c:51(__GI_raise)[0x7feb288118eb]
 
stdlib/abort.c:81(__GI_abort)[0x7feb287fc535]
 
/usr/sbin/mysqld(+0x4e9ddb)[0x55d4e185cddb]
 
/usr/sbin/mysqld(+0x4e690b)[0x55d4e185990b]
 
/usr/sbin/mysqld(+0xa1bcd1)[0x55d4e1d8ecd1]
 
/usr/sbin/mysqld(+0xa1c112)[0x55d4e1d8f112]
 
/usr/sbin/mysqld(+0xa0819b)[0x55d4e1d7b19b]
 
nptl/pthread_create.c:487(start_thread)[0x7feb289a1fa3]
 
x86_64/clone.S:97(clone)[0x7feb288d306f]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x0): (null)
 
Connection ID (thread ID): 1
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
 
 
 
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
 
information that should help you find out what is causing the crash.
 
 
 
We think the query pointer is invalid, but we will try to print it anyway.
 
Query:
 
 
 
Writing a core file...
 
Working directory at /var/lib/mysql
 
Resource Limits:
 
Limit                     Soft Limit           Hard Limit           Units
 
Max cpu time              unlimited            unlimited            seconds
 
Max file size             unlimited            unlimited            bytes
 
Max data size             unlimited            unlimited            bytes
 
Max stack size            8388608              unlimited            bytes
 
Max core file size        0                    unlimited            bytes
 
Max resident set          unlimited            unlimited            bytes
 
Max processes             96301                96301                processes
 
Max open files            32768                32768                files
 
Max locked memory         65536                65536                bytes
 
Max address space         unlimited            unlimited            bytes
 
Max file locks            unlimited            unlimited            locks
 
Max pending signals       96301                96301                signals
 
Max msgqueue size         819200               819200               bytes
 
Max nice priority         0                    0
 
Max realtime priority     0                    0
 
Max realtime timeout      unlimited            unlimited            us
 
Core pattern: core
 
 
 
Kernel version: Linux version 4.19.0-21-amd64 (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.249-2 (2022-06-30)
 
 
 
2022-12-18 16:23:02 0 [Note] InnoDB: Using Linux native AIO
 
2022-12-18 16:23:02 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
2022-12-18 16:23:02 0 [Note] InnoDB: Uses event mutexes
 
2022-12-18 16:23:02 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
 
2022-12-18 16:23:02 0 [Note] InnoDB: Number of pools: 1
 
2022-12-18 16:23:02 0 [Note] InnoDB: Using SSE2 crc32 instructions
 
2022-12-18 16:23:02 0 [Note] InnoDB: Initializing buffer pool, total size = 19G, instances = 8, chunk size = 128M
 
2022-12-18 16:23:02 0 [Note] InnoDB: Completed initialization of buffer pool
 
2022-12-18 16:23:02 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
 
2022-12-18 16:23:02 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=3920017074621
 
2022-12-18 16:23:02 0 [Note] InnoDB: Starting final batch to recover 23 pages from redo log.
 
2022-12-18 16:23:03 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
 
2022-12-18 16:23:03 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
 
2022-12-18 16:23:03 0 [Note] InnoDB: Creating shared tablespace for temporary tables
 
2022-12-18 16:23:03 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
 
2022-12-18 16:23:03 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
 
2022-12-18 16:23:03 0 [Note] InnoDB: 10.3.36 started; log sequence number 3920017229454; transaction id 7589834756
 
2022-12-18 16:23:03 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
 
2022-12-18 16:23:03 0 [Note] Plugin 'FEEDBACK' is disabled.
 
2022-12-18 16:23:03 0 [Note] Recovering after a crash using tc.log
 
2022-12-18 16:23:03 0 [Note] Starting crash recovery...
 
2022-12-18 16:23:03 0 [Note] Crash recovery finished.
 
2022-12-18 16:23:03 0 [Note] Server socket created on IP: '::'.
 
2022-12-18 16:23:03 0x7fe1851b5700  InnoDB: Assertion failure in file /build/mariadb-10.3-lMTCRk/mariadb-10.3-10.3.36/storage/innobase/trx/trx0purge.cc line 118
 
InnoDB: Failing assertion: purge_sys.tail.trx_no <= purge_sys.rseg->last_trx_no()
 
InnoDB: We intentionally generate a memory trap.
 
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
 
InnoDB: If you get repeated assertion failures or crashes, even
 
InnoDB: immediately after the mysqld startup, there may be
 
InnoDB: corruption in the InnoDB tablespace. Please refer to
 
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
 
InnoDB: about forcing recovery.
 
221218 16:23:03 [ERROR] mysqld got signal 6 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed,
 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.3.36-MariaDB-0+deb10u2
 
key_buffer_size=134217728
 
read_buffer_size=131072
 
max_used_connections=0
 
max_threads=10002
 
thread_count=5
 
It is possible that mysqld could use up to
 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 22120993 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x7fe178000c08
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
2022-12-18 16:23:03 0 [Note] Reading of all Master_info entries succeeded
 
2022-12-18 16:23:03 0 [Note] Added new Master_info '' to hash table
 
2022-12-18 16:23:03 0 [Note] /usr/sbin/mysqld: ready for connections.
 
Version: '10.3.36-MariaDB-0+deb10u2'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Debian 10
 
stack_bottom = 0x7fe1851b4d58 thread_stack 0x49000
 
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55eb0ad07bde]
 
/usr/sbin/mysqld(handle_fatal_signal+0x54d)[0x55eb0a8300bd]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7fe6d28c7730]
 
linux/raise.c:51(__GI_raise)[0x7fe6d272c8eb]
 
stdlib/abort.c:81(__GI_abort)[0x7fe6d2717535]
 
/usr/sbin/mysqld(+0x4e9ddb)[0x55eb0a56eddb]
 
/usr/sbin/mysqld(+0x4e690b)[0x55eb0a56b90b]
 
/usr/sbin/mysqld(+0xa1bcd1)[0x55eb0aaa0cd1]
 
/usr/sbin/mysqld(+0xa1c112)[0x55eb0aaa1112]
 
/usr/sbin/mysqld(+0xa0819b)[0x55eb0aa8d19b]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7fa3)[0x7fe6d28bcfa3]
 
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7fe6d27ee06f]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x0): (null)
 
Connection ID (thread ID): 2
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
 
 
 
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
 
information that should help you find out what is causing the crash.
 
 
 
We think the query pointer is invalid, but we will try to print it anyway.
 
Query:
 
 
 
Writing a core file...
 
Working directory at /var/lib/mysql
 
Resource Limits:
 
Limit                     Soft Limit           Hard Limit           Units
 
Max cpu time              unlimited            unlimited            seconds
 
Max file size             unlimited            unlimited            bytes
 
Max data size             unlimited            unlimited            bytes
 
Max stack size            8388608              unlimited            bytes
 
Max core file size        0                    unlimited            bytes
 
Max resident set          unlimited            unlimited            bytes
 
Max processes             96301                96301                processes
 
Max open files            32768                32768                files
 
Max locked memory         65536                65536                bytes
 
Max address space         unlimited            unlimited            bytes
 
Max file locks            unlimited            unlimited            locks
 
Max pending signals       96301                96301                signals
 
Max msgqueue size         819200               819200               bytes
 
Max nice priority         0                    0
 
Max realtime priority     0                    0
 
Max realtime timeout      unlimited            unlimited            us
 
Core pattern: core
 
 
 
Kernel version: Linux version 4.19.0-21-amd64 (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.249-2 (2022-06-30)
 
 
 
2022-12-18 16:23:10 0 [Note] InnoDB: Using Linux native AIO
 
2022-12-18 16:23:10 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
2022-12-18 16:23:10 0 [Note] InnoDB: Uses event mutexes
 
2022-12-18 16:23:10 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
 
2022-12-18 16:23:10 0 [Note] InnoDB: Number of pools: 1
 
2022-12-18 16:23:10 0 [Note] InnoDB: Using SSE2 crc32 instructions
 
2022-12-18 16:23:10 0 [Note] InnoDB: Initializing buffer pool, total size = 19G, instances = 8, chunk size = 128M
 
2022-12-18 16:23:10 0 [Note] InnoDB: Completed initialization of buffer pool
 
2022-12-18 16:23:10 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
 
2022-12-18 16:23:10 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=3920017074621
 
2022-12-18 16:23:10 0 [Note] InnoDB: Starting final batch to recover 23 pages from redo log.
 
2022-12-18 16:23:11 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
 
2022-12-18 16:23:11 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
 
2022-12-18 16:23:11 0 [Note] InnoDB: Creating shared tablespace for temporary tables
 
2022-12-18 16:23:11 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
 
2022-12-18 16:23:11 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
 
2022-12-18 16:23:11 0 [Note] InnoDB: 10.3.36 started; log sequence number 3920017230203; transaction id 7589834756
 
2022-12-18 16:23:11 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
 
2022-12-18 16:23:11 0 [Note] Plugin 'FEEDBACK' is disabled.
 
2022-12-18 16:23:11 0 [Note] Recovering after a crash using tc.log
 
2022-12-18 16:23:11 0 [Note] Starting crash recovery...
 
2022-12-18 16:23:11 0 [Note] Crash recovery finished.
 
2022-12-18 16:23:11 0 [Note] Server socket created on IP: '::'.
 
2022-12-18 16:23:11 0x7fd4ee698700  InnoDB: Assertion failure in file /build/mariadb-10.3-lMTCRk/mariadb-10.3-10.3.36/storage/innobase/trx/trx0purge.cc line 118
 
InnoDB: Failing assertion: purge_sys.tail.trx_no <= purge_sys.rseg->last_trx_no()
 
InnoDB: We intentionally generate a memory trap.
 
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
 
InnoDB: If you get repeated assertion failures or crashes, even
 
InnoDB: immediately after the mysqld startup, there may be
 
InnoDB: corruption in the InnoDB tablespace. Please refer to
 
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
 
InnoDB: about forcing recovery.
 
221218 16:23:11 [ERROR] mysqld got signal 6 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed,
 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.3.36-MariaDB-0+deb10u2
 
key_buffer_size=134217728
 
read_buffer_size=131072
 
max_used_connections=0
 
max_threads=10002
 
thread_count=6
 
It is possible that mysqld could use up to
 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 22120993 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x7fd4e8000c08
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7fd4ee697d58 thread_stack 0x49000
 
2022-12-18 16:23:11 0 [Note] Reading of all Master_info entries succeeded
 
2022-12-18 16:23:11 0 [Note] Added new Master_info '' to hash table
 
2022-12-18 16:23:11 0 [Note] /usr/sbin/mysqld: ready for connections.
 
Version: '10.3.36-MariaDB-0+deb10u2'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Debian 10
 
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x561544305bde]
 
/usr/sbin/mysqld(handle_fatal_signal+0x54d)[0x561543e2e0bd]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7fda3c8d0730]
 
linux/raise.c:51(__GI_raise)[0x7fda3c7358eb]
 
stdlib/abort.c:81(__GI_abort)[0x7fda3c720535]
 
/usr/sbin/mysqld(+0x4e9ddb)[0x561543b6cddb]
 
/usr/sbin/mysqld(+0x4e690b)[0x561543b6990b]
 
/usr/sbin/mysqld(+0xa1bcd1)[0x56154409ecd1]
 
/usr/sbin/mysqld(+0xa1c112)[0x56154409f112]
 
/usr/sbin/mysqld(+0xa0819b)[0x56154408b19b]
 
nptl/pthread_create.c:487(start_thread)[0x7fda3c8c5fa3]
 
x86_64/clone.S:97(clone)[0x7fda3c7f706f]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x0): (null)
 
Connection ID (thread ID): 2
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
 
 
 
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
 
information that should help you find out what is causing the crash.
 
 
 
We think the query pointer is invalid, but we will try to print it anyway.
 
Query:
 
 
 
Writing a core file...
 
Working directory at /var/lib/mysql
 
Resource Limits:
 
Limit                     Soft Limit           Hard Limit           Units
 
Max cpu time              unlimited            unlimited            seconds
 
Max file size             unlimited            unlimited            bytes
 
Max data size             unlimited            unlimited            bytes
 
Max stack size            8388608              unlimited            bytes
 
Max core file size        0                    unlimited            bytes
 
Max resident set          unlimited            unlimited            bytes
 
Max processes             96301                96301                processes
 
Max open files            32768                32768                files
 
Max locked memory         65536                65536                bytes
 
Max address space         unlimited            unlimited            bytes
 
Max file locks            unlimited            unlimited            locks
 
Max pending signals       96301                96301                signals
 
Max msgqueue size         819200               819200               bytes
 
Max nice priority         0                    0
 
Max realtime priority     0                    0
 
Max realtime timeout      unlimited            unlimited            us
 
Core pattern: core
 
 
 
Kernel version: Linux version 4.19.0-21-amd64 (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.249-2 (2022-06-30)
 
 
 
2022-12-18 16:23:18 0 [Note] InnoDB: Using Linux native AIO
 
2022-12-18 16:23:18 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
2022-12-18 16:23:18 0 [Note] InnoDB: Uses event mutexes
 
2022-12-18 16:23:18 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
 
2022-12-18 16:23:18 0 [Note] InnoDB: Number of pools: 1
 
2022-12-18 16:23:18 0 [Note] InnoDB: Using SSE2 crc32 instructions
 
2022-12-18 16:23:18 0 [Note] InnoDB: Initializing buffer pool, total size = 19G, instances = 8, chunk size = 128M
 
2022-12-18 16:23:18 0 [Note] InnoDB: Completed initialization of buffer pool
 
2022-12-18 16:23:18 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
 
2022-12-18 16:23:18 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=3920017074621
 
2022-12-18 16:23:18 0 [Note] InnoDB: Starting final batch to recover 23 pages from redo log.
 
2022-12-18 16:23:19 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
 
2022-12-18 16:23:19 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
 
2022-12-18 16:23:19 0 [Note] InnoDB: Creating shared tablespace for temporary tables
 
2022-12-18 16:23:19 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
 
2022-12-18 16:23:19 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
 
2022-12-18 16:23:19 0 [Note] InnoDB: Waiting for purge to start
 
2022-12-18 16:23:19 0x7f38958f0700  InnoDB: Assertion failure in file /build/mariadb-10.3-lMTCRk/mariadb-10.3-10.3.36/storage/innobase/trx/trx0purge.cc line 118
 
InnoDB: Failing assertion: purge_sys.tail.trx_no <= purge_sys.rseg->last_trx_no()
 
InnoDB: We intentionally generate a memory trap.
 
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
 
InnoDB: If you get repeated assertion failures or crashes, even
 
InnoDB: immediately after the mysqld startup, there may be
 
InnoDB: corruption in the InnoDB tablespace. Please refer to
 
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
 
InnoDB: about forcing recovery.
 
221218 16:23:19 [ERROR] mysqld got signal 6 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed,
 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.3.36-MariaDB-0+deb10u2
 
key_buffer_size=134217728
 
read_buffer_size=131072
 
max_used_connections=0
 
max_threads=10002
 
thread_count=4
 
It is possible that mysqld could use up to
 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 22120993 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x7f3844000c08
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7f38958efd58 thread_stack 0x49000
 
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x5638e1de8bde]
 
/usr/sbin/mysqld(handle_fatal_signal+0x54d)[0x5638e19110bd]
 
2022-12-18 16:23:19 0 [Note] InnoDB: 10.3.36 started; log sequence number 3920017230203; transaction id 7589834756
 
2022-12-18 16:23:19 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
 
2022-12-18 16:23:19 0 [Note] Plugin 'FEEDBACK' is disabled.
 
2022-12-18 16:23:19 0 [Note] Recovering after a crash using tc.log
 
2022-12-18 16:23:19 0 [Note] Starting crash recovery...
 
2022-12-18 16:23:19 0 [Note] Crash recovery finished.
 
2022-12-18 16:23:19 0 [Note] Server socket created on IP: '::'.
 
2022-12-18 16:23:19 0 [Note] Reading of all Master_info entries succeeded
 
2022-12-18 16:23:19 0 [Note] Added new Master_info '' to hash table
 
2022-12-18 16:23:19 0 [Note] /usr/sbin/mysqld: ready for connections.
 
Version: '10.3.36-MariaDB-0+deb10u2'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Debian 10
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7f3de3015730]
 
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x10b)[0x7f3de2e7a8eb]
 
linux/raise.c:51(__GI_raise)[0x7f3de2e65535]
 
/usr/sbin/mysqld(+0x4e9ddb)[0x5638e164fddb]
 
/usr/sbin/mysqld(+0x4e690b)[0x5638e164c90b]
 
/usr/sbin/mysqld(+0xa1bcd1)[0x5638e1b81cd1]
 
/usr/sbin/mysqld(+0xa1c112)[0x5638e1b82112]
 
/usr/sbin/mysqld(+0xa0819b)[0x5638e1b6e19b]
 
nptl/pthread_create.c:487(start_thread)[0x7f3de300afa3]
 
x86_64/clone.S:97(clone)[0x7f3de2f3c06f]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x0): (null)
 
Connection ID (thread ID): 4
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
 
 
 
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
 
information that should help you find out what is causing the crash.
 
 
 
We think the query pointer is invalid, but we will try to print it anyway.
 
Query:
 
 
 
Writing a core file...
 
Working directory at /var/lib/mysql
 
Resource Limits:
 
Limit                     Soft Limit           Hard Limit           Units
 
Max cpu time              unlimited            unlimited            seconds
 
Max file size             unlimited            unlimited            bytes
 
Max data size             unlimited            unlimited            bytes
 
Max stack size            8388608              unlimited            bytes
 
Max core file size        0                    unlimited            bytes
 
Max resident set          unlimited            unlimited            bytes
 
Max processes             96301                96301                processes
 
Max open files            32768                32768                files
 
Max locked memory         65536                65536                bytes
 
Max address space         unlimited            unlimited            bytes
 
Max file locks            unlimited            unlimited            locks
 
Max pending signals       96301                96301                signals
 
Max msgqueue size         819200               819200               bytes
 
Max nice priority         0                    0
 
Max realtime priority     0                    0
 
Max realtime timeout      unlimited            unlimited            us
 
Core pattern: core
 
 
 
Kernel version: Linux version 4.19.0-21-amd64 (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.249-2 (2022-06-30)
 
 
 
2022-12-18 16:23:25 0 [Note] InnoDB: Using Linux native AIO
 
2022-12-18 16:23:25 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
2022-12-18 16:23:25 0 [Note] InnoDB: Uses event mutexes
 
2022-12-18 16:23:25 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
 
2022-12-18 16:23:25 0 [Note] InnoDB: Number of pools: 1
 
2022-12-18 16:23:25 0 [Note] InnoDB: Using SSE2 crc32 instructions
 
2022-12-18 16:23:25 0 [Note] InnoDB: Initializing buffer pool, total size = 19G, instances = 8, chunk size = 128M
 
2022-12-18 16:23:25 0 [Note] InnoDB: Completed initialization of buffer pool
 
2022-12-18 16:23:25 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
 
2022-12-18 16:23:26 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=3920017074621
 
2022-12-18 16:23:26 0 [Note] InnoDB: Starting final batch to recover 23 pages from redo log.
 
2022-12-18 16:23:27 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
 
2022-12-18 16:23:27 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
 
2022-12-18 16:23:27 0 [Note] InnoDB: Creating shared tablespace for temporary tables
 
2022-12-18 16:23:27 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
 
2022-12-18 16:23:27 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
 
2022-12-18 16:23:27 0 [Note] InnoDB: Waiting for purge to start
 
2022-12-18 16:23:27 0x7f3fae42f700  InnoDB: Assertion failure in file /build/mariadb-10.3-lMTCRk/mariadb-10.3-10.3.36/storage/innobase/trx/trx0purge.cc line 118
 
InnoDB: Failing assertion: purge_sys.tail.trx_no <= purge_sys.rseg->last_trx_no()
 
InnoDB: We intentionally generate a memory trap.
 
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
 
InnoDB: If you get repeated assertion failures or crashes, even
 
InnoDB: immediately after the mysqld startup, there may be
 
InnoDB: corruption in the InnoDB tablespace. Please refer to
 
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
 
InnoDB: about forcing recovery.
 
221218 16:23:27 [ERROR] mysqld got signal 6 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed,
 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.3.36-MariaDB-0+deb10u2
 
key_buffer_size=134217728
 
read_buffer_size=131072
 
max_used_connections=0
 
max_threads=10002
 
thread_count=4
 
It is possible that mysqld could use up to
 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 22120993 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x7f3f68000c08
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7f3fae42ed58 thread_stack 0x49000
 
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55b979341bde]
 
/usr/sbin/mysqld(handle_fatal_signal+0x54d)[0x55b978e6a0bd]
 
2022-12-18 16:23:27 0 [Note] InnoDB: 10.3.36 started; log sequence number 3920017230203; transaction id 7589834756
 
2022-12-18 16:23:27 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
 
2022-12-18 16:23:27 0 [Note] Plugin 'FEEDBACK' is disabled.
 
2022-12-18 16:23:27 0 [Note] Recovering after a crash using tc.log
 
2022-12-18 16:23:27 0 [Note] Starting crash recovery...
 
2022-12-18 16:23:27 0 [Note] Crash recovery finished.
 
2022-12-18 16:23:27 0 [Note] Server socket created on IP: '::'.
 
2022-12-18 16:23:27 0 [Note] Reading of all Master_info entries succeeded
 
2022-12-18 16:23:27 0 [Note] Added new Master_info '' to hash table
 
2022-12-18 16:23:27 0 [Note] /usr/sbin/mysqld: ready for connections.
 
Version: '10.3.36-MariaDB-0+deb10u2'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Debian 10
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7f44fc60d730]
 
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x10b)[0x7f44fc4728eb]
 
linux/raise.c:51(__GI_raise)[0x7f44fc45d535]
 
/usr/sbin/mysqld(+0x4e9ddb)[0x55b978ba8ddb]
 
/usr/sbin/mysqld(+0x4e690b)[0x55b978ba590b]
 
/usr/sbin/mysqld(+0xa1bcd1)[0x55b9790dacd1]
 
/usr/sbin/mysqld(+0xa1c112)[0x55b9790db112]
 
/usr/sbin/mysqld(+0xa0819b)[0x55b9790c719b]
 
nptl/pthread_create.c:487(start_thread)[0x7f44fc602fa3]
 
x86_64/clone.S:97(clone)[0x7f44fc53406f]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x0): (null)
 
Connection ID (thread ID): 2
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on
 
 
 
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
 
information that should help you find out what is causing the crash.
 
 
 
We think the query pointer is invalid, but we will try to print it anyway.
 
Query:
 
 
 
Writing a core file...
 
Working directory at /var/lib/mysql
 
Resource Limits:
 
Limit                     Soft Limit           Hard Limit           Units
 
Max cpu time              unlimited            unlimited            seconds
 
Max file size             unlimited            unlimited            bytes
 
Max data size             unlimited            unlimited            bytes
 
Max stack size            8388608              unlimited            bytes
 
Max core file size        0                    unlimited            bytes
 
Max resident set          unlimited            unlimited            bytes
 
Max processes             96301                96301                processes
 
Max open files            32768                32768                files
 
Max locked memory         65536                65536                bytes
 
Max address space         unlimited            unlimited            bytes
 
Max file locks            unlimited            unlimited            locks
 
Max pending signals       96301                96301                signals
 
Max msgqueue size         819200               819200               bytes
 
Max nice priority         0                    0
 
Max realtime priority     0                    0
 
Max realtime timeout      unlimited            unlimited            us
 
Core pattern: core
 
 
 
Kernel version: Linux version 4.19.0-21-amd64 (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.249-2 (2022-06-30)
 
 

Thank you

 Comments   
Comment by Daniel Black [ 2022-12-18 ]

Hi Dubois, thanks for the bug report. Do you know what SQL the python was executing at the time of failure?

Can you install the mariadb-server-core-10.3-dbgsym package from the repo listed on https://wiki.debian.org/AutomaticDebugPackages (I assume your on buster).

With this can you attempt a stack trace from the corefile in /var/lib/mysql based on the logs.

Comment by Jean [ 2022-12-19 ]

Hi @Daniel Black, first of all thank you very much for your reply. A friend showed me the steps I needed to make to backup my data, using the following commands :

`mysqld --innodb_force_recovery=3`
`mysqldump --user= --password= --databases db1 db2 db3 > dump.sql`

Then, and here is why I can't give you a stack trace, is because we deleted everything in /var/lib/mysql/
*
`mysql_install_db`
`mysql db1 < dump.sql` (which didn't actually just restore db1 but also db2 and db3, even though they were not specified in the command)

I was a little panicked and wanted to retrieve everything as soon as possible so I didn't take time to wait for answers in this thread, I'm sorry.

I have multiple scripts running doing all kinds of queries so it would be complicated for me to tell which one was the culprit. However all of them are very simple `INSERT table(col1, col2) VALUES(val1, val2)` or `SELECT * FROM table WHERE id=4`, or some can be a little more complicated and use `JSON_SET / JSON_EXTRACT / JSON_ARRAY_APPEND / JSON_MERGE` but that's about it

Hope you have a great day however and thank you again for showing interest in my issue

Comment by Daniel Black [ 2022-12-19 ]

I'm glad you're operational again. Please take the time to get full backups.

Our main task here is to work out what happened so this quite inconvenient panic won't happen to other people.

The SQL would be something modifying, maybe an {{ALTER TABLE}}s occurring? Are you using virtual columns, partitions or other slightly more complex features?

Comment by Jean [ 2022-12-19 ]

I don't think I use virtual columns or partitions, I will check

But I use it very simply : I created the table with MySQL Workbench by right clicking a schema and clicking "Create table". The rows are very simple too I only change their names and put a type like INT / VARCHAR(X) and LONGTEXT.

Then my python scripts INSERT data into it, UPDATE data, or SELECT data from it
I never use anything other instruction than those three ones. So I don't use ALTER or something like that. I never create tables from my script neither.

I will take time to read the link you provided

Comment by Marko Mäkelä [ 2022-12-19 ]

What is the history of this database? With which version of MySQL or MariaDB was it originally created, and which versions of MariaDB were used for upgrading it? This bug might have been fixed in MDEV-27800, assuming that the upgrade from a pre-10.3 version would be done straight to MariaDB 10.3.35 or later.

Comment by Jean [ 2022-12-19 ]

I just had the same issue again after making a new database.. It worked a few hours, then mysql crashed, most likely got corrupted and won't start again

So the history of this database is just on "mariadb Ver 15.1 Distrib 10.3.36-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2"

Unfortunately I'm having issues to install the mariadb-server-core-10.3-dbgsym package, I read the link @Daniel Black provided but I can't understand it. update:I managed to get mariadb-server-core-10.3, but not mariadb-server-core-10.3-dbgsym, I can't tell if that's a big deal. I will try to make the stack trace now

I would paste the new logs from `/var/log/mysql/error.log` but the file is really big I'm not sure if you want that or not

Comment by Jean [ 2022-12-19 ]

Here is the stack trace I got (file attached to message) gdb_output.log

I got it doing by first adding `core_file` in my mariadb.cnf file, then systemctl restart mariadb mysql
Then I did the following commands :
sudo gdb /usr/sbin/mysqld /var/lib/mysql/core
inside gdb :
set logging on
set pagination off
set print frame-arguments all
thread apply all bt full
set logging off

Then I just copied the file here.

A notable thing : The issue started since I started about a hundred processes that send queries in localhost to the database(queries contain message from users, which can contain symbols). Before that I had scripts that sent queries of the same type (basic insert/select/update) in localhost to the database, but with much lower rate. But the cpu seemed to only reach about 20% usage

The first error logged by my python script is :
(2013, 'Lost connection to MySQL server during query')

And I use aiomysql's pools to create connection

Comment by Marko Mäkelä [ 2022-12-20 ]

Dubois, can you please install the -dbgsym package for the server and upload new stack traces? The ?? need to be resolved to actual function names.

How did you initialize a new database? From SQL statements or logical dump, instead of some backup copy of data files?

Comment by Jean [ 2022-12-26 ] 

Jean, can you please install the -dbgsym package for the server and upload new stack traces? The ?? need to be resolved to actual function names.

I've been trying so when I had time the past days but I can't manage to install the -dbgsym package :/ And my Mysql database keeps crashing every few hours(I'm looking at the /var/log/mysql/error.log), sometime it restarts on its own, sometime it stays down and I have to re-do the full mysqld --innodb_force_recovery=3 into create a dump.sql, re-installing mysql/mariadb and then `mysql < dump.sql`, but there's over 100 GB of data so it takes hours every time this is really frustrating (I must have done it like 4 times only this week)

@Marko I initialized a new database by doing a logical dump and created a .sql, then I re-installed mysql (after removing all the files there was), then I did I started mysql and did : mysql < dump.sql

Comment by Jean [ 2022-12-27 ]

I have some news !
I found out about this file: /var/log/syslog and it's interesting because it says why mysql got killed : https://ctxt.io/2/AAAQl4uRFg (The interesting part to me being those 2 lines: Out of memory: Kill process 10042 (mysqld) score 333 or sacrifice child. Killed process 10042 (mysqld)

And I checked with htop, mysql is indeed eating my memory more and more over the hours, and when it finally eat it all, debian or I don't know how to call it, decides to kill mysql because it's the one eating too much memory.

Here is my .cnf :

#
 
# These groups are read by MariaDB server.
 
# Use it for options that only the server (but not clients) should see
 
#
 
# See the examples of server my.cnf files in /usr/share/mysql
 
 
 
# this is read by the standalone daemon and embedded servers
 
[server]
 
 
 
# this is only for the mysqld standalone daemon
 
[mysqld]
 
 
 
#
 
# * Basic Settings
 
#
 
user                    = mysql
 
pid-file                = /run/mysqld/mysqld.pid
 
socket                  = /run/mysqld/mysqld.sock
 
#port                   = 3306
 
basedir                 = /usr
 
datadir                 = /var/lib/mysql
 
tmpdir                  = /tmp
 
lc-messages-dir         = /usr/share/mysql
 
#skip-external-locking
 
 
 
# Instead of skip-networking the default is now to listen only on
 
# localhost which is more compatible and is not less secure.
 
bind-address            = 0.0.0.0
 
innodb_buffer_pool_size = 16G
 
innodb_log_buffer_size = 32M
 
innodb_log_file_size = 2047M
 
 
 
    #
 
    # * Fine Tuning
 
    #
 
    #key_buffer_size        = 16M
 
    max_allowed_packet     = 2G
 
    #thread_stack           = 192K
 
    thread_cache_size      = 38
 
    # This replaces the startup script and checks MyISAM tables if needed
 
    # the first time they are touched
 
    #myisam_recover_options = BACKUP
 
    max_connections        = 10000
 
    table_cache            = 3000
 
    #thread_concurrency     = 10
 
    innodb_read_io_threads  = 8
 
    innodb_write_io_threads = 8
 
    innodb_file_per_table   = 1
 
    innodb_flush_log_at_trx_commit = 2
 
    enforce_storage_engine  = InnoDB
 
    #
 
    # * Query Cache Configuration
 
    #
 
    #query_cache_limit      = 1M
 
    query_cache_type                = 0
 
    #
 
    # * Logging and Replication
 
    #
 
    # Both location gets rotated by the cronjob.
 
    # Be aware that this log type is a performance killer.
 
    # As of 5.1 you can enable the log at runtime!
 
    #general_log_file       = /var/log/mysql/mysql.log
 
    #general_log            = 1
 
    #
 
    # Error log - should be very few entries.
 
    #
 
    log_error = /var/log/mysql/error.log
 
    #
 
    # Enable the slow query log to see queries with especially long duration
 
    #slow_query_log_file    = /var/log/mysql/mariadb-slow.log
 
    #long_query_time        = 10
 
    #log_slow_rate_limit    = 1000
 
    #log_slow_verbosity     = query_plan
 
    #log-queries-not-using-indexes
 
    #
 
    # The following can be used as easy to replay backup logs or for replication.
 
    # note: if you are setting up a replication slave, see README.Debian about
 
    #       other settings you may need to change.
 
    #server-id              = 1
 
    #log_bin                = /var/log/mysql/mysql-bin.log
 
    expire_logs_days        = 10
 
    #max_binlog_size        = 100M
 
    #binlog_do_db           = include_database_name
 
    #binlog_ignore_db       = exclude_database_name
 
    
    #
 
    # * Security Features
 
    #
 
    # Read the manual, too, if you want chroot!
 
    #chroot = /var/lib/mysql/
 
    #
 
    # For generating SSL certificates you can use for example the GUI tool "tinyca".
 
    #
 
    #ssl-ca = /etc/mysql/cacert.pem
 
    #ssl-cert = /etc/mysql/server-cert.pem
 
    #ssl-key = /etc/mysql/server-key.pem
 
    #
 
    # Accept only connections using the latest and most secure TLS protocol version.
 
    # ..when MariaDB is compiled with OpenSSL:
 
    #ssl-cipher = TLSv1.2
 
    # ..when MariaDB is compiled with YaSSL (default in Debian):
 
    #ssl = on
 
    
    #
 
    # * Character sets
 
    #
 
    # MySQL/MariaDB default is Latin1, but in Debian we rather default to the full
 
    # utf8 4-byte character set. See also client.cnf
 
    #
 
    character-set-server  = utf8mb4
 
    collation-server      = utf8mb4_general_ci
 
    
    #
 
    # * InnoDB
 
    #
 
    # InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
 
    # Read the manual for more InnoDB related options. There are many!
 
    
    #
 
    # * Unix socket authentication plugin is built-in since 10.0.22-6
 
    #
 
    # Needed so the root database user can authenticate without a password but
 
    # only when running as the unix root user.
 
    #
 
    # Also available for other users if required.
 
    # See https://mariadb.com/kb/en/unix_socket-authentication-plugin/
 
    
    # this is only for embedded server
 
    [embedded]
 
    
    # This group is only read by MariaDB servers, not by MySQL.
 
    # If you use the same .cnf file for MySQL and MariaDB,
 
    # you can put MariaDB-only options here
 
    [mariadb]
 
    # This group is only read by MariaDB-10.3 servers.
 
    # If you use the same .cnf file for MariaDB of different versions,
 
    # use this group for options that older servers don't understand
 
    [mariadb-10.3]

My VPS has 24GB of memory available, no swap

Comment by Jean [ 2022-12-27 ]

Alright, issue resolved <3 YAY !

So I feel ashamed, and I considered just leaving discretely this thread, but it wouldn't be correct as you guys tried to help me and I really appreciate it.

So the issue was that I had a python script running on my server, that was leaking, and I didn't think a python script could leak so I didn't consider it at all. But then when I looked memory on htop, I would see it increase its memory usage, but it would make no sense because nothing in the script is intended to produce such behavior.

Now I fixed the leak with the script(I couldn't find how so I chose the easy solution: When it ends its loop, kill it, then restart it, it's ugly but it works ! D:

And it's been 2 and a half hour mysql hasn't crashed, which is almost a record. Memory usage is stable, nothing gets OOM-killed anymore.

The reason why some crashes resulted in the MariaDB not starting up again I don't know, I guess a crash always has its share of unexpected behavior and it's just that.

Anyway! One lesson I will remember is to check /var/log/syslog because it actually had the reason printed out(Out of memory)

Comment by Daniel Black [ 2022-12-28 ]

Thanks Dubois. Much appreciate the confession. We'll try to add checking these kind of things more on new bug reports.

Happy MariaDBing and have a happy new year.

Regardless of the cause, it looks like we've still got a crash recovery issue to resolve.

Generated at Thu Feb 08 10:14:54 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.