[MDEV-17498] MariaDB 10.2.18 mysqld got signal 11 - buffer overflow detected Created: 2018-10-19  Updated: 2019-02-25  Resolved: 2019-02-25

Status: Closed
Project: MariaDB Server
Component/s: Platform Debian
Affects Version/s: 10.2.18
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Miquel Orobitg Assignee: Unassigned
Resolution: Incomplete Votes: 0
Labels: Buffer, need_feedback, overflow
Environment:

Servidor Debian 8 (Jessie)
4 CPU
8GB of RAM
100GB SSD

Issue Links:
Blocks
is blocked by MDEV-17199 Assertion `pos < table->n_v_def' fail... Closed

 Description   

Hi,

I migrated MariaDb 10.1.36 to MariaDB 10.2.18 and now a couple of times a day, the server stops with a buffer overflow detected. I've seen that this error is produced when databases backups or when long time queries are are done.

Message error:

2018-10-18 22:32:09 139819698378496 [Warning] Aborted connection 930 to db: 'DB19034' user: 'cloud' host: '82.223.68.30' (Got timeout reading communication packets)
 
2018-10-18 22:32:09 139819703531264 [Warning] Aborted connection 931 to db: 'DB19034' user: 'cloud' host: '82.223.68.30' (Got timeout reading communication packets)
 
2018-10-18 22:32:09 139819698075392 [Warning] Aborted connection 932 to db: 'DB19034' user: 'cloud' host: '82.223.68.30' (Got timeout reading communication packets)
 
2018-10-18 22:32:09 139819697772288 [Warning] Aborted connection 933 to db: 'DB19034' user: 'cloud' host: '82.223.68.30' (Got timeout reading communication packets)
 
2018-10-18 22:44:01 139819921102592 [Warning] Aborted connection 314 to db: 'DB17378' user: 'cloud' host: '127.0.0.1' (Got timeout reading communication packets)
 
2018-10-18 22:45:43 139821727639296 [Warning] Aborted connection 51 to db: 'DB17378' user: 'cloud' host: '127.0.0.1' (Got timeout reading communication packets)
 
181018 23:09:25 [ERROR] mysqld got signal 11 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.2.18-MariaDB-10.2.18+maria~jessie-log
 
key_buffer_size=8388608
 
read_buffer_size=131072
 
max_used_connections=799
 
max_threads=1001
 
thread_count=207
 
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2207743 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x7f2ac086c2b8
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7f2a4f054e38 thread_stack 0x49000
 
*** buffer overflow detected ***: /usr/sbin/mysqld terminated
 
======= Backtrace: =========
 
/lib/x86_64-linux-gnu/libc.so.6(+0x731af)[0x7f2b14aff1af]
 
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7f2b14b84aa7]
 
/lib/x86_64-linux-gnu/libc.so.6(+0xf6cc0)[0x7f2b14b82cc0]
 
/lib/x86_64-linux-gnu/libc.so.6(+0xf8a17)[0x7f2b14b84a17]
 
/usr/sbin/mysqld(my_addr_resolve+0xda)[0x55a81e515c2a]
 
/usr/sbin/mysqld(my_print_stacktrace+0x1d2)[0x55a81e4ff322]
 
/usr/sbin/mysqld(handle_fatal_signal+0x345)[0x55a81df78955]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0xf890)[0x7f2b16737890]
 
/usr/sbin/mysqld(+0x587b52)[0x55a81de78b52]
 
/usr/sbin/mysqld(_Z15parse_vcol_defsP3THDP11st_mem_rootP5TABLEPb+0x68f)[0x55a81de7ee0f]
 
/usr/sbin/mysqld(_Z21open_table_from_shareP3THDP11TABLE_SHAREPKcjjjP5TABLEb+0x85f)[0x55a81de8447f]
 
/usr/sbin/mysqld(_Z14get_all_tablesP3THDP10TABLE_LISTP4Item+0xc34)[0x55a81de41bd4]
 
/usr/sbin/mysqld(_Z24get_schema_tables_resultP4JOIN23enum_schema_table_state+0x266)[0x55a81de43296]
 
/usr/sbin/mysqld(_ZN4JOIN10exec_innerEv+0x8a5)[0x55a81de267a5]
 
/usr/sbin/mysqld(_ZN4JOIN4execEv+0x33)[0x55a81de26d23]
 
/usr/sbin/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xec)[0x55a81de26e6c]
 
/usr/sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x137)[0x55a81de27857]
 
/usr/sbin/mysqld(+0x40dbcf)[0x55a81dcfebcf]
 
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x5b3a)[0x55a81ddcc65a]
 
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x28a)[0x55a81ddcee4a]
 
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x1f7c)[0x55a81ddd1c2c]
 
/usr/sbin/mysqld(_Z10do_commandP3THD+0x17b)[0x55a81ddd24ab]
 
/usr/sbin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x232)[0x55a81deab612]
 
/usr/sbin/mysqld(handle_one_connection+0x3d)[0x55a81deab7ad]
 
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8064)[0x7f2b16730064]
 
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f2b14b7462d]
 
======= Memory map: ========
 
55a81d8f1000-55a81ea2d000 r-xp 00000000 fe:00 397523                     /usr/sbin/mysqld
 
55a81ec2c000-55a81ed0c000 r--p 0113b000 fe:00 397523                     /usr/sbin/mysqld
 
55a81ed0c000-55a81edc5000 rw-p 0121b000 fe:00 397523                     /usr/sbin/mysqld
 
55a81edc5000-55a81f65c000 rw-p 00000000 00:00 0 
55a820859000-55a8307fe000 rw-p 00000000 00:00 0                          [heap]
 
7f29f0000000-7f29f04bc000 rw-p 00000000 00:00 0 
7f29f04bc000-7f29f4000000 ---p 00000000 00:00 0 
7f29f8000000-7f29fb96b000 rw-p 00000000 00:00 0 
7f29fb96b000-7f29fc000000 ---p 00000000 00:00 0 
7f2a00000000-7f2a03ed6000 rw-p 00000000 00:00 0 
7f2a03ed6000-7f2a04000000 ---p 00000000 00:00 0 
7f2a08000000-7f2a0bfff000 rw-p 00000000 00:00 0 
7f2a0bfff000-7f2a0c000000 ---p 00000000 00:00 0 
7f2a10000000-7f2a13fff000 rw-p 00000000 00:00 0 
7f2a13fff000-7f2a14000000 ---p 00000000 00:00 0 
7f2a18000000-7f2a194b5000 rw-p 00000000 00:00 0 
7f2a194b5000-7f2a1c000000 ---p 00000000 00:00 0 
7f2a20000000-7f2a23fff000 rw-p 00000000 00:00 0 
f2a28000000-7f2a2ffff000 rw-p 00000000 00:00 0 
7f2a2ffff000-7f2a30000000 ---p 00000000 00:00 0 
7f2a30000000-7f2a37fff000 rw-p 00000000 00:00 0 
7f2a37fff000-7f2a38000000 ---p 00000000 00:00 0 
7f2a38000000-7f2a40000000 rw-p 00000000 00:00 0 
7f2a40000000-7f2a43ffd000 rw-p 00000000 00:00 0 
7f2a43ffd000-7f2a44000000 ---p 00000000 00:00 0 
7f2a44000000-7f2a48000000 rw-p 00000000 00:00 0 
7f2a48000000-7f2a4bffd000 rw-p 00000000 00:00 0 
7f2a4bffd000-7f2a4c000000 ---p 00000000 00:00 0 
7f2a4f00c000-7f2a4f00d000 ---p 00000000 00:00 0 
7f2a4f00d000-7f2a4f056000 rw-p 00000000 00:00 0                          [stack:55242]
 
7f2a4f056000-7f2a4f057000 ---p 00000000 00:00 0 
7f2a4f057000-7f2a4f0a0000 rw-p 00000000 00:00 0                          [stack:55237]
 
7f2a4f0a0000-7f2a4f0a1000 ---p 00000000 00:00 0 
7f2a4f0a1000-7f2a4f0ea000 rw-p 00000000 00:00 0                          [stack:55094]
 
7f2a4f0ea000-7f2a4f0eb000 ---p 00000000 00:00 0 
7f2a4f0eb000-7f2a4f134000 rw-p 00000000 00:00 0                          [stack:55086]
 
7f2a4f134000-7f2a4f135000 ---p 00000000 00:00 0 
7f2a4f135000-7f2a4f17e000 rw-p 00000000 00:00 0                          [stack:54618]
 
7f2a4f1c8000-7f2a4f1c9000 ---p 00000000 00:00 0 
7f2a4f1c9000-7f2a4f212000 rw-p 00000000 00:00 0 
7f2a4f25c000-7f2a4f25d000 ---p 00000000 00:00 0 
7f2a4f25d000-7f2a4f2a6000 rw-p 00000000 00:00 0                          [stack:53514]
 
7f2a4f2a6000-7f2a4f2a7000 ---p 00000000 00:00 0 
7f2a4f2a7000-7f2a4f2f0000 rw-p 00000000 00:00 0                          [stack:53022]
 
7f2a4f2f0000-7f2a4f2f1000 ---p 00000000 00:00 0 
....
 
7f2b13509000-7f2b13514000 r-xp 00000000 fe:00 792369                     /lib/x86_64-linux-gnu/libnss_files-2.19.so
 
7f2b13514000-7f2b13713000 ---p 0000b000 fe:00 792369                     /lib/x86_64-linux-gnu/libnss_files-2.19.so
 
7f2b13713000-7f2b13714000 r--p 0000a000 fe:00 792369                     /lib/x86_64-linux-gnu/libnss_files-2.19.so
 
7f2b13714000-7f2b13715000 rw-p 0000b000 fe:00 792369                     /lib/x86_64-linux-gnu/libnss_files-2.19.so
 
7f2b13715000-7f2b1371f000 r-xp 00000000 fe:00 792371                     /lib/x86_64-linux-gnu/libnss_nis-2.19.so
 
7f2b1371f000-7f2b1391e000 ---p 0000a000 fe:00 792371                     /lib/x86_64-linux-gnu/libnss_nis-2.19.so
 
7f2b1391e000-7f2b1391f000 r--p 00009000 fe:00 792371                     /lib/x86_64-linux-gnu/libnss_nis-2.19.so
 
7f2b1391f000-7f2b13920000 rw-p 0000a000 fe:00 792371                     /lib/x86_64-linux-gnu/libnss_nis-2.19.so
 
7f2b13920000-7f2b13935000 r-xp 00000000 fe:00 792366                     /lib/x86_64-linux-gnu/libnsl-2.19.so
 
7f2b13935000-7f2b13b34000 ---p 00015000 fe:00 792366                     /lib/x86_64-linux-gnu/libnsl-2.19.so
 
7f2b13b34000-7f2b13b35000 r--p 00014000 fe:00 792366                     /lib/x86_64-linux-gnu/libnsl-2.19.so
 
7f2b13b35000-7f2b13b36000 rw-p 00015000 fe:00 792366                     /lib/x86_64-linux-gnu/libnsl-2.19.so
 
7f2b13b36000-7f2b13b38000 rw-p 00000000 00:00 0 
7f2b13b38000-7f2b13b3f000 r-xp 00000000 fe:00 792367                     /lib/x86_64-linux-gnu/libnss_compat-2.19.so
 
7f2b13b3f000-7f2b13d3e000 ---p 00007000 fe:00 792367                     /lib/x86_64-linux-gnu/libnss_compat-2.19.so
 
7f2b13d3e000-7f2b13d3f000 r--p 00006000 fe:00 792367                     /lib/x86_64-linux-gnu/libnss_compat-2.19.so
 
7f2b13d3f000-7f2b13d40000 rw-p 00007000 fe:00 792367                     /lib/x86_64-linux-gnu/libnss_compat-2.19.so
 
7f2b13d40000-7f2b13d51000 r-xp 00000000 fe:00 786544                     /lib/x86_64-linux-gnu/libgpg-error.so.0.13.0
 
7f2b13d51000-7f2b13f50000 ---p 00011000 fe:00 786544                     /lib/x86_64-linux-gnu/libgpg-error.so.0.13.0
 
7f2b13f50000-7f2b13f51000 r--p 00010000 fe:00 786544                     /lib/x86_64-linux-gnu/libgpg-error.so.0.13.0
 
7f2b13f51000-7f2b13f52000 rw-p 00011000 fe:00 786544                     /lib/x86_64-linux-gnu/libgpg-error.so.0.13.0
 
7f2b13f52000-7f2b13f68000 r-xp 00000000 fe:00 795744                     /lib/x86_64-linux-gnu/libgcc_s.so.1
 
7f2b13f68000-7f2b14167000 ---p 00016000 fe:00 795744                     /lib/x86_64-linux-gnu/libgcc_s.so.1
 
7f2b14167000-7f2b14168000 rw-p 00015000 fe:00 795744                     /lib/x86_64-linux-gnu/libgcc_s.so.1
 
7f2b14168000-7f2b1417c000 r-xp 00000000 fe:00 792374                     /lib/x86_64-linux-gnu/libresolv-2.19.so
 
7f2b1417c000-7f2b1437b000 ---p 00014000 fe:00 792374                     /lib/x86_64-linux-gnu/libresolv-2.19.so
 
7f2b1437b000-7f2b1437c000 r--p 00013000 fe:00 792374                     /lib/x86_64-linux-gnu/libresolv-2.19.so
 
7f2b1437c000-7f2b1437d000 rw-p 00014000 fe:00 792374                     /lib/x86_64-linux-gnu/libresolv-2.19.so
 
7f2b1437d000-7f2b1437f000 rw-p 00000000 00:00 0 
7f2b1437f000-7f2b14457000 r-xp 00000000 fe:00 787647                     /lib/x86_64-linux-gnu/libgcrypt.so.20.0.3
 
7f2b14457000-7f2b14657000 ---p 000d8000 fe:00 787647                     /lib/x86_64-linux-gnu/libgcrypt.so.20.0.3
 
7f2b14657000-7f2b14658000 r--p 000d8000 fe:00 787647                     /lib/x86_64-linux-gnu/libgcrypt.so.20.0.3
 
7f2b14658000-7f2b14661000 rw-p 000d9000 fe:00 787647                     /lib/x86_64-linux-gnu/libgcrypt.so.20.0.3
 
7f2b14661000-7f2b14683000 r-xp 00000000 fe:00 786450                     /lib/x86_64-linux-gnu/liblzma.so.5.0.0
 
....
 
7f2b1696c000-7f2b16971000 rw-s 00000000 00:0a 4936333                    /[aio] (deleted)
 
7f2b16971000-7f2b16976000 rw-s 00000000 00:0a 4936332                    /[aio] (deleted)
 
7f2b16976000-7f2b1697b000 rw-s 00000000 00:0a 4936331                    /[aio] (deleted)
 
7f2b1697b000-7f2b16980000 rw-s 00000000 00:0a 4936330                    /[aio] (deleted)
 
7f2b16980000-7f2b16985000 rw-s 00000000 00:0a 4936329                    /[aio] (deleted)
 
7f2b16985000-7f2b1698a000 rw-s 00000000 00:0a 4936328                    /[aio] (deleted)
 
7f2b1698a000-7f2b1698f000 rw-s 00000000 00:0a 4936327                    /[aio] (deleted)
 
7f2b1698f000-7f2b16994000 rw-s 00000000 00:0a 4936326                    /[aio] (deleted)
 
7f2b16994000-7f2b16999000 rw-s 00000000 00:0a 4936325                    /[aio] (deleted)
 
7f2b16999000-7f2b16b5c000 rw-p 00000000 00:00 0 
7f2b16b5d000-7f2b16b65000 rw-p 00000000 00:00 0 
7f2b16b65000-7f2b16b66000 r--p 00020000 fe:00 792355                     /lib/x86_64-linux-gnu/ld-2.19.so
 
7f2b16b66000-7f2b16b67000 rw-p 00021000 fe:00 792355                     /lib/x86_64-linux-gnu/ld-2.19.so
 
7f2b16b67000-7f2b16b68000 rw-p 00000000 00:00 0 
7ffc86693000-7ffc866b4000 rw-p 00000000 00:00 0                          [stack]
 
7ffc86751000-7ffc86754000 r--p 00000000 00:00 0                          [vvar]
 
7ffc86754000-7ffc86756000 r-xp 00000000 00:00 0                          [vdso]
 
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
 
Fatal signal 6 while backtracing
 
2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Uses event mutexes
 
2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Compressed tables use zlib 1.2.8
 
2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Using Linux native AIO
 
2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Adjusting innodb_buffer_pool_instances from 8 to 1 since innodb_buffer_pool_size is less than 1024 MiB
 
2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Number of pools: 1
 
2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Using SSE2 crc32 instructions
 
2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Initializing buffer pool, total size = 512M, instances = 1, chunk size = 128M
 
2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Completed initialization of buffer pool
 
2018-10-18 23:09:31 140627733841664 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
 
2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Highest supported file format is Barracuda.
 
2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Starting crash recovery from checkpoint LSN=93166438230
 
2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Last binlog file '/var/log/mysql/mariadb-bin.000003', position 6415487
 
2018-10-18 23:14:21 140628574136192 [Note] InnoDB: 128 out of 128 rollback segments are active.
 
2018-10-18 23:14:21 140628574136192 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
 
2018-10-18 23:14:21 140628574136192 [Note] InnoDB: Creating shared tablespace for temporary tables
 
2018-10-18 23:14:21 140628574136192 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
 
2018-10-18 23:14:21 140628574136192 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
 
2018-10-18 23:14:21 140628574136192 [Note] InnoDB: 5.7.23 started; log sequence number 93166438249
 
2018-10-18 23:14:21 140627446568704 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool



 Comments   
Comment by Elena Stepanova [ 2018-10-19 ]

Could you try to find out which query on which table causes it? I suppose it might be related to MDEV-17199.

Comment by Miquel Orobitg [ 2018-10-19 ]

For example, one query I find out that causes the buffer overflow failure is the following one:

SELECT ENGINE,SUM(DATA_LENGTH+INDEX_LENGTH),COUNT(ENGINE),SUM(DATA_LENGTH),SUM(INDEX_LENGTH) FROM information_schema.TABLES WHERE TABLE_SCHEMA NOT IN ('information_schema', 'performance_schema', 'mysql') AND ENGINE IS NOT NULL GROUP BY ENGINE ORDER BY ENGINE ASC;

The server has 317 databases.

Comment by Elena Stepanova [ 2018-10-21 ]

I see. Unfortunately, it doesn't point at any specific table, as I hoped in order to confirm or rule out the relation to MDEV-17199.
I've now raised the priority of MDEV-17199, as soon as we have a fix, we'll ask you to try it and see if the problem goes away.

Comment by Sergei Golubchik [ 2018-10-29 ]

Here it crashes when trying to parse the virtual column declaration (parse_vcol_defs).
So it'd be useful to know the table definition.

orobitg, you can narrow it down by doing, for example, show create table for every table. Like, run 

select table_schema,table_name from information_schema.tables

, and then for every table run show create. Or even simpler,

select concat('show create table ', table_schema, '.', table_name, ';') from information_schema.tables

save the output into a file and pipe it into mysql command-line client.

Comment by Miquel Orobitg [ 2018-10-31 ]

I ran the following sql: 

select concat('show create table ', table_schema, '.', table_name, ';') from information_schema.tables

I saved the results (show create table for all tables of all databases) into a sql file and then I have run this file. The problem is that, after some seconds, the execution crashes and breaks the mysql service with a buffer overflow detected.

I've tried the same procedure in other server with the same version but less databases, and the execution has run correctly.

Comment by Elena Stepanova [ 2018-10-31 ]

If it crashes reliably, you have almost found the "guilty" table already.
Normally the crashing query would appear in the error log, but with buffer overflow it's not always the case. If there is no query in the error log, please enable the general log (SET GLOBAL general_log=1) and run the queries from the file again. After it crashes, check the general log, the last query will be the crashing one.

Comment by Miquel Orobitg [ 2018-11-06 ]

I've logged the last queries of two different crashes:

The first one -> Doing backups of all databases (I have to deactivate this process again):

17229 Query     SELECT TRIGGER_NAME FROM INFORMATION_SCHEMA.TRIGGERS WHERE EVENT_OBJECT_SCHEMA = DATABASE() AND EVENT_OBJECT_TABLE = 'Wkp__Workplace'
 
               17229 Query     SET SESSION character_set_results = 'utf8'
 
               17229 Query     ROLLBACK TO SAVEPOINT sp
 
               17229 Query     SELECT engine FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'Wkp__WorkplaceTranslation'
 
               17229 Query     SET SQL_QUOTE_SHOW_CREATE=1
 
               17229 Query     SET SESSION character_set_results = 'binary'
 
               17229 Query     show create table `Wkp__WorkplaceTranslation`
 
               17229 Query     SET SESSION character_set_results = 'utf8'
 
               17229 Query     show fields from `Wkp__WorkplaceTranslation`
 
               17229 Query     SELECT /*!40001 SQL_NO_CACHE* / * FROM `Wkp__WorkplaceTranslation`
 
               17229 Query     SET SESSION character_set_results = 'binary'
 
               17229 Init DB   DB16883
 
               17229 Query     select @@collation_database
 
               17229 Query     SELECT TRIGGER_NAME FROM INFORMATION_SCHEMA.TRIGGERS WHERE EVENT_OBJECT_SCHEMA = DATABASE() AND EVENT_OBJECT_TABLE = 'Wkp__WorkplaceTranslation'
 
               17229 Query     SET SESSION character_set_results = 'utf8'
 
               17229 Query     ROLLBACK TO SAVEPOINT sp
 
               17229 Query     SELECT engine FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'XXX'

The second one it broke when I've opened the phpmyadmin and I've tried to navigate to the following database DB16883 (I need to test if with other databases it also happens):

181106  9:26:24   485 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
                  506 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
                  226 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
                  514 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
                  484 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
                  472 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
                  507 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
                  490 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
                  531 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
                  512 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
                  553 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
                  530 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
                  546 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
                  554 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
                 1234 Connect   phpmyadmin@localhost as anonymous on 
                 1235 Connect   root@localhost as anonymous on 
                 1235 Query     SET CHARACTER SET 'utf8mb4'
 
                 1235 Query     SET collation_connection = 'utf8mb4_general_ci'
 
                 1235 Query     SELECT `SCHEMA_NAME` FROM `INFORMATION_SCHEMA`.`SCHEMATA`, (select DB_first_level from ( SELECT distinct SUBSTRING_INDEX(SCHEMA_NAME, '_', 1) DB_first_level FROM INFORMATION_SCHEMA.SCHEMATA WHERE TRUE AND `SCHEMA_NAME` LIKE '%DB16883%' ) t ORDER BY DB_first_level ASC LIMIT 0, 250) t2 where 1 = locate(concat(DB_first_level, '_'), concat(SCHEMA_NAME, '_')) order by SCHEMA_NAME ASC
 
                 1235 Query     SELECT COUNT(*) FROM `INFORMATION_SCHEMA`.`TABLES` WHERE `TABLE_SCHEMA`='DB16883' AND `TABLE_TYPE`='BASE TABLE'
 
                  556 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
                  551 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
 
181106  9:26:25   568 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:25'      ORDER BY TE.ServerStartDate ASC
 
                  567 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:25'      ORDER BY TE.ServerStartDate ASC
 
                  611 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:25'      ORDER BY TE.ServerStartDate ASC

I haven't already tested the sql I've mentioned before.

Moreover, when it crashes the mysqld daemon process stills running. This does not allow me to start mysql again since the start process hangs. When I kill the daemon process, after few seconds it is automatically activated again, not allowing me to restart the mariadb service.

Comment by Miquel Orobitg [ 2018-11-06 ]

I've done another test to confirm that DB16883 database is the problem. I've run mysqlcheck for all databses and mariadb service has crashed after analysing the tables of this database.

DB16883.Wkp__Workplace                             OK
 
DB16883.Wkp__WorkplaceTranslation                  OK
 
mysqlcheck: Got error: 2013: Lost connection to MySQL server during query when executing 'CHECK TABLE ... '

Wkp__WorkplaceTranslation is the last table of the database:

CREATE TABLE `Wkp__WorkplaceTranslation` (
 
  `WorkplaceId` int(11) NOT NULL,
 
  `LanguageId` int(11) NOT NULL,
 
  `Name` varchar(100) DEFAULT NULL,
 
  PRIMARY KEY (`WorkplaceId`,`LanguageId`),
 
  KEY `WorkplaceTranslation_Language_FK` (`LanguageId`),
 
  CONSTRAINT `WorkplaceTranslation_Language_FK` FOREIGN KEY (`LanguageId`) REFERENCES `Language` (`LanguageId`) ON UPDATE CASCADE,
 
  CONSTRAINT `WorkplaceTranslation_Workplace_FK` FOREIGN KEY (`WorkplaceId`) REFERENCES `Wkp__Workplace` (`WorkplaceId`) ON DELETE CASCADE ON UPDATE CASCADE
 
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

After, I've drop this database (before I've stored a backup) and I have run mysqlcheck again and it has finished correctly.

I cannot understand what happens because this server has other 300 databases with the same schema.

Finally, I've restored the backup to another server (with less databases) with the same mariaDB version. I've run similar tests that had been crashed in the original server, and these tests work correctly.

Comment by Elena Stepanova [ 2018-12-29 ]

Wkp__WorkplaceTranslation is not the table, for this one check went okay.
The table in question is the next one, the name of which hides under "..." obfuscation:

mysqlcheck: Got error: 2013: Lost connection to MySQL server during query when executing 'CHECK TABLE ... '

Also, if the part of the general log you previously pasted reflects the very last records, then the problematic table (either the same or a different one) would be the one obfuscated as "XXX":

SELECT engine FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'XXX'

Generated at Thu Feb 08 08:36:55 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.