Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-17498

MariaDB 10.2.18 mysqld got signal 11 - buffer overflow detected

Details

    Description

      Hi,

      I migrated MariaDb 10.1.36 to MariaDB 10.2.18 and now a couple of times a day, the server stops with a buffer overflow detected. I've seen that this error is produced when databases backups or when long time queries are are done.

      Message error:

      2018-10-18 22:32:09 139819698378496 [Warning] Aborted connection 930 to db: 'DB19034' user: 'cloud' host: '82.223.68.30' (Got timeout reading communication packets)
      2018-10-18 22:32:09 139819703531264 [Warning] Aborted connection 931 to db: 'DB19034' user: 'cloud' host: '82.223.68.30' (Got timeout reading communication packets)
      2018-10-18 22:32:09 139819698075392 [Warning] Aborted connection 932 to db: 'DB19034' user: 'cloud' host: '82.223.68.30' (Got timeout reading communication packets)
      2018-10-18 22:32:09 139819697772288 [Warning] Aborted connection 933 to db: 'DB19034' user: 'cloud' host: '82.223.68.30' (Got timeout reading communication packets)
      2018-10-18 22:44:01 139819921102592 [Warning] Aborted connection 314 to db: 'DB17378' user: 'cloud' host: '127.0.0.1' (Got timeout reading communication packets)
      2018-10-18 22:45:43 139821727639296 [Warning] Aborted connection 51 to db: 'DB17378' user: 'cloud' host: '127.0.0.1' (Got timeout reading communication packets)
      181018 23:09:25 [ERROR] mysqld got signal 11 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed, 
      something is definitely wrong and this may fail.
       
      Server version: 10.2.18-MariaDB-10.2.18+maria~jessie-log
      key_buffer_size=8388608
      read_buffer_size=131072
      max_used_connections=799
      max_threads=1001
      thread_count=207
      It is possible that mysqld could use up to 
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2207743 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x7f2ac086c2b8
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7f2a4f054e38 thread_stack 0x49000
      *** buffer overflow detected ***: /usr/sbin/mysqld terminated
      ======= Backtrace: =========
      /lib/x86_64-linux-gnu/libc.so.6(+0x731af)[0x7f2b14aff1af]
      /lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7f2b14b84aa7]
      /lib/x86_64-linux-gnu/libc.so.6(+0xf6cc0)[0x7f2b14b82cc0]
      /lib/x86_64-linux-gnu/libc.so.6(+0xf8a17)[0x7f2b14b84a17]
      /usr/sbin/mysqld(my_addr_resolve+0xda)[0x55a81e515c2a]
      /usr/sbin/mysqld(my_print_stacktrace+0x1d2)[0x55a81e4ff322]
      /usr/sbin/mysqld(handle_fatal_signal+0x345)[0x55a81df78955]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0xf890)[0x7f2b16737890]
      /usr/sbin/mysqld(+0x587b52)[0x55a81de78b52]
      /usr/sbin/mysqld(_Z15parse_vcol_defsP3THDP11st_mem_rootP5TABLEPb+0x68f)[0x55a81de7ee0f]
      /usr/sbin/mysqld(_Z21open_table_from_shareP3THDP11TABLE_SHAREPKcjjjP5TABLEb+0x85f)[0x55a81de8447f]
      /usr/sbin/mysqld(_Z14get_all_tablesP3THDP10TABLE_LISTP4Item+0xc34)[0x55a81de41bd4]
      /usr/sbin/mysqld(_Z24get_schema_tables_resultP4JOIN23enum_schema_table_state+0x266)[0x55a81de43296]
      /usr/sbin/mysqld(_ZN4JOIN10exec_innerEv+0x8a5)[0x55a81de267a5]
      /usr/sbin/mysqld(_ZN4JOIN4execEv+0x33)[0x55a81de26d23]
      /usr/sbin/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_jP8st_orderS9_S7_S9_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xec)[0x55a81de26e6c]
      /usr/sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x137)[0x55a81de27857]
      /usr/sbin/mysqld(+0x40dbcf)[0x55a81dcfebcf]
      /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x5b3a)[0x55a81ddcc65a]
      /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x28a)[0x55a81ddcee4a]
      /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x1f7c)[0x55a81ddd1c2c]
      /usr/sbin/mysqld(_Z10do_commandP3THD+0x17b)[0x55a81ddd24ab]
      /usr/sbin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x232)[0x55a81deab612]
      /usr/sbin/mysqld(handle_one_connection+0x3d)[0x55a81deab7ad]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x8064)[0x7f2b16730064]
      /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f2b14b7462d]
      ======= Memory map: ========
      55a81d8f1000-55a81ea2d000 r-xp 00000000 fe:00 397523                     /usr/sbin/mysqld
      55a81ec2c000-55a81ed0c000 r--p 0113b000 fe:00 397523                     /usr/sbin/mysqld
      55a81ed0c000-55a81edc5000 rw-p 0121b000 fe:00 397523                     /usr/sbin/mysqld
      55a81edc5000-55a81f65c000 rw-p 00000000 00:00 0 
      55a820859000-55a8307fe000 rw-p 00000000 00:00 0                          [heap]
      7f29f0000000-7f29f04bc000 rw-p 00000000 00:00 0 
      7f29f04bc000-7f29f4000000 ---p 00000000 00:00 0 
      7f29f8000000-7f29fb96b000 rw-p 00000000 00:00 0 
      7f29fb96b000-7f29fc000000 ---p 00000000 00:00 0 
      7f2a00000000-7f2a03ed6000 rw-p 00000000 00:00 0 
      7f2a03ed6000-7f2a04000000 ---p 00000000 00:00 0 
      7f2a08000000-7f2a0bfff000 rw-p 00000000 00:00 0 
      7f2a0bfff000-7f2a0c000000 ---p 00000000 00:00 0 
      7f2a10000000-7f2a13fff000 rw-p 00000000 00:00 0 
      7f2a13fff000-7f2a14000000 ---p 00000000 00:00 0 
      7f2a18000000-7f2a194b5000 rw-p 00000000 00:00 0 
      7f2a194b5000-7f2a1c000000 ---p 00000000 00:00 0 
      7f2a20000000-7f2a23fff000 rw-p 00000000 00:00 0 
      f2a28000000-7f2a2ffff000 rw-p 00000000 00:00 0 
      7f2a2ffff000-7f2a30000000 ---p 00000000 00:00 0 
      7f2a30000000-7f2a37fff000 rw-p 00000000 00:00 0 
      7f2a37fff000-7f2a38000000 ---p 00000000 00:00 0 
      7f2a38000000-7f2a40000000 rw-p 00000000 00:00 0 
      7f2a40000000-7f2a43ffd000 rw-p 00000000 00:00 0 
      7f2a43ffd000-7f2a44000000 ---p 00000000 00:00 0 
      7f2a44000000-7f2a48000000 rw-p 00000000 00:00 0 
      7f2a48000000-7f2a4bffd000 rw-p 00000000 00:00 0 
      7f2a4bffd000-7f2a4c000000 ---p 00000000 00:00 0 
      7f2a4f00c000-7f2a4f00d000 ---p 00000000 00:00 0 
      7f2a4f00d000-7f2a4f056000 rw-p 00000000 00:00 0                          [stack:55242]
      7f2a4f056000-7f2a4f057000 ---p 00000000 00:00 0 
      7f2a4f057000-7f2a4f0a0000 rw-p 00000000 00:00 0                          [stack:55237]
      7f2a4f0a0000-7f2a4f0a1000 ---p 00000000 00:00 0 
      7f2a4f0a1000-7f2a4f0ea000 rw-p 00000000 00:00 0                          [stack:55094]
      7f2a4f0ea000-7f2a4f0eb000 ---p 00000000 00:00 0 
      7f2a4f0eb000-7f2a4f134000 rw-p 00000000 00:00 0                          [stack:55086]
      7f2a4f134000-7f2a4f135000 ---p 00000000 00:00 0 
      7f2a4f135000-7f2a4f17e000 rw-p 00000000 00:00 0                          [stack:54618]
      7f2a4f1c8000-7f2a4f1c9000 ---p 00000000 00:00 0 
      7f2a4f1c9000-7f2a4f212000 rw-p 00000000 00:00 0 
      7f2a4f25c000-7f2a4f25d000 ---p 00000000 00:00 0 
      7f2a4f25d000-7f2a4f2a6000 rw-p 00000000 00:00 0                          [stack:53514]
      7f2a4f2a6000-7f2a4f2a7000 ---p 00000000 00:00 0 
      7f2a4f2a7000-7f2a4f2f0000 rw-p 00000000 00:00 0                          [stack:53022]
      7f2a4f2f0000-7f2a4f2f1000 ---p 00000000 00:00 0 
      ....
      7f2b13509000-7f2b13514000 r-xp 00000000 fe:00 792369                     /lib/x86_64-linux-gnu/libnss_files-2.19.so
      7f2b13514000-7f2b13713000 ---p 0000b000 fe:00 792369                     /lib/x86_64-linux-gnu/libnss_files-2.19.so
      7f2b13713000-7f2b13714000 r--p 0000a000 fe:00 792369                     /lib/x86_64-linux-gnu/libnss_files-2.19.so
      7f2b13714000-7f2b13715000 rw-p 0000b000 fe:00 792369                     /lib/x86_64-linux-gnu/libnss_files-2.19.so
      7f2b13715000-7f2b1371f000 r-xp 00000000 fe:00 792371                     /lib/x86_64-linux-gnu/libnss_nis-2.19.so
      7f2b1371f000-7f2b1391e000 ---p 0000a000 fe:00 792371                     /lib/x86_64-linux-gnu/libnss_nis-2.19.so
      7f2b1391e000-7f2b1391f000 r--p 00009000 fe:00 792371                     /lib/x86_64-linux-gnu/libnss_nis-2.19.so
      7f2b1391f000-7f2b13920000 rw-p 0000a000 fe:00 792371                     /lib/x86_64-linux-gnu/libnss_nis-2.19.so
      7f2b13920000-7f2b13935000 r-xp 00000000 fe:00 792366                     /lib/x86_64-linux-gnu/libnsl-2.19.so
      7f2b13935000-7f2b13b34000 ---p 00015000 fe:00 792366                     /lib/x86_64-linux-gnu/libnsl-2.19.so
      7f2b13b34000-7f2b13b35000 r--p 00014000 fe:00 792366                     /lib/x86_64-linux-gnu/libnsl-2.19.so
      7f2b13b35000-7f2b13b36000 rw-p 00015000 fe:00 792366                     /lib/x86_64-linux-gnu/libnsl-2.19.so
      7f2b13b36000-7f2b13b38000 rw-p 00000000 00:00 0 
      7f2b13b38000-7f2b13b3f000 r-xp 00000000 fe:00 792367                     /lib/x86_64-linux-gnu/libnss_compat-2.19.so
      7f2b13b3f000-7f2b13d3e000 ---p 00007000 fe:00 792367                     /lib/x86_64-linux-gnu/libnss_compat-2.19.so
      7f2b13d3e000-7f2b13d3f000 r--p 00006000 fe:00 792367                     /lib/x86_64-linux-gnu/libnss_compat-2.19.so
      7f2b13d3f000-7f2b13d40000 rw-p 00007000 fe:00 792367                     /lib/x86_64-linux-gnu/libnss_compat-2.19.so
      7f2b13d40000-7f2b13d51000 r-xp 00000000 fe:00 786544                     /lib/x86_64-linux-gnu/libgpg-error.so.0.13.0
      7f2b13d51000-7f2b13f50000 ---p 00011000 fe:00 786544                     /lib/x86_64-linux-gnu/libgpg-error.so.0.13.0
      7f2b13f50000-7f2b13f51000 r--p 00010000 fe:00 786544                     /lib/x86_64-linux-gnu/libgpg-error.so.0.13.0
      7f2b13f51000-7f2b13f52000 rw-p 00011000 fe:00 786544                     /lib/x86_64-linux-gnu/libgpg-error.so.0.13.0
      7f2b13f52000-7f2b13f68000 r-xp 00000000 fe:00 795744                     /lib/x86_64-linux-gnu/libgcc_s.so.1
      7f2b13f68000-7f2b14167000 ---p 00016000 fe:00 795744                     /lib/x86_64-linux-gnu/libgcc_s.so.1
      7f2b14167000-7f2b14168000 rw-p 00015000 fe:00 795744                     /lib/x86_64-linux-gnu/libgcc_s.so.1
      7f2b14168000-7f2b1417c000 r-xp 00000000 fe:00 792374                     /lib/x86_64-linux-gnu/libresolv-2.19.so
      7f2b1417c000-7f2b1437b000 ---p 00014000 fe:00 792374                     /lib/x86_64-linux-gnu/libresolv-2.19.so
      7f2b1437b000-7f2b1437c000 r--p 00013000 fe:00 792374                     /lib/x86_64-linux-gnu/libresolv-2.19.so
      7f2b1437c000-7f2b1437d000 rw-p 00014000 fe:00 792374                     /lib/x86_64-linux-gnu/libresolv-2.19.so
      7f2b1437d000-7f2b1437f000 rw-p 00000000 00:00 0 
      7f2b1437f000-7f2b14457000 r-xp 00000000 fe:00 787647                     /lib/x86_64-linux-gnu/libgcrypt.so.20.0.3
      7f2b14457000-7f2b14657000 ---p 000d8000 fe:00 787647                     /lib/x86_64-linux-gnu/libgcrypt.so.20.0.3
      7f2b14657000-7f2b14658000 r--p 000d8000 fe:00 787647                     /lib/x86_64-linux-gnu/libgcrypt.so.20.0.3
      7f2b14658000-7f2b14661000 rw-p 000d9000 fe:00 787647                     /lib/x86_64-linux-gnu/libgcrypt.so.20.0.3
      7f2b14661000-7f2b14683000 r-xp 00000000 fe:00 786450                     /lib/x86_64-linux-gnu/liblzma.so.5.0.0
      ....
      7f2b1696c000-7f2b16971000 rw-s 00000000 00:0a 4936333                    /[aio] (deleted)
      7f2b16971000-7f2b16976000 rw-s 00000000 00:0a 4936332                    /[aio] (deleted)
      7f2b16976000-7f2b1697b000 rw-s 00000000 00:0a 4936331                    /[aio] (deleted)
      7f2b1697b000-7f2b16980000 rw-s 00000000 00:0a 4936330                    /[aio] (deleted)
      7f2b16980000-7f2b16985000 rw-s 00000000 00:0a 4936329                    /[aio] (deleted)
      7f2b16985000-7f2b1698a000 rw-s 00000000 00:0a 4936328                    /[aio] (deleted)
      7f2b1698a000-7f2b1698f000 rw-s 00000000 00:0a 4936327                    /[aio] (deleted)
      7f2b1698f000-7f2b16994000 rw-s 00000000 00:0a 4936326                    /[aio] (deleted)
      7f2b16994000-7f2b16999000 rw-s 00000000 00:0a 4936325                    /[aio] (deleted)
      7f2b16999000-7f2b16b5c000 rw-p 00000000 00:00 0 
      7f2b16b5d000-7f2b16b65000 rw-p 00000000 00:00 0 
      7f2b16b65000-7f2b16b66000 r--p 00020000 fe:00 792355                     /lib/x86_64-linux-gnu/ld-2.19.so
      7f2b16b66000-7f2b16b67000 rw-p 00021000 fe:00 792355                     /lib/x86_64-linux-gnu/ld-2.19.so
      7f2b16b67000-7f2b16b68000 rw-p 00000000 00:00 0 
      7ffc86693000-7ffc866b4000 rw-p 00000000 00:00 0                          [stack]
      7ffc86751000-7ffc86754000 r--p 00000000 00:00 0                          [vvar]
      7ffc86754000-7ffc86756000 r-xp 00000000 00:00 0                          [vdso]
      ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
      Fatal signal 6 while backtracing
      2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
      2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Uses event mutexes
      2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Compressed tables use zlib 1.2.8
      2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Using Linux native AIO
      2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Adjusting innodb_buffer_pool_instances from 8 to 1 since innodb_buffer_pool_size is less than 1024 MiB
      2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Number of pools: 1
      2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Using SSE2 crc32 instructions
      2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Initializing buffer pool, total size = 512M, instances = 1, chunk size = 128M
      2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Completed initialization of buffer pool
      2018-10-18 23:09:31 140627733841664 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
      2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Highest supported file format is Barracuda.
      2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Starting crash recovery from checkpoint LSN=93166438230
      2018-10-18 23:09:31 140628574136192 [Note] InnoDB: Last binlog file '/var/log/mysql/mariadb-bin.000003', position 6415487
      2018-10-18 23:14:21 140628574136192 [Note] InnoDB: 128 out of 128 rollback segments are active.
      2018-10-18 23:14:21 140628574136192 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
      2018-10-18 23:14:21 140628574136192 [Note] InnoDB: Creating shared tablespace for temporary tables
      2018-10-18 23:14:21 140628574136192 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
      2018-10-18 23:14:21 140628574136192 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
      2018-10-18 23:14:21 140628574136192 [Note] InnoDB: 5.7.23 started; log sequence number 93166438249
      2018-10-18 23:14:21 140627446568704 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
      

      Attachments

        Issue Links

          Activity

            orobitg Miquel Orobitg added a comment - - edited

            I ran the following sql:

            select concat('show create table ', table_schema, '.', table_name, ';') from information_schema.tables
            

            I saved the results (show create table for all tables of all databases) into a sql file and then I have run this file. The problem is that, after some seconds, the execution crashes and breaks the mysql service with a buffer overflow detected.

            I've tried the same procedure in other server with the same version but less databases, and the execution has run correctly.

            orobitg Miquel Orobitg added a comment - - edited I ran the following sql: select concat('show create table ', table_schema, '.', table_name, ';') from information_schema.tables I saved the results (show create table for all tables of all databases) into a sql file and then I have run this file. The problem is that, after some seconds, the execution crashes and breaks the mysql service with a buffer overflow detected. I've tried the same procedure in other server with the same version but less databases, and the execution has run correctly.

            If it crashes reliably, you have almost found the "guilty" table already.
            Normally the crashing query would appear in the error log, but with buffer overflow it's not always the case. If there is no query in the error log, please enable the general log (SET GLOBAL general_log=1) and run the queries from the file again. After it crashes, check the general log, the last query will be the crashing one.

            elenst Elena Stepanova added a comment - If it crashes reliably, you have almost found the "guilty" table already. Normally the crashing query would appear in the error log, but with buffer overflow it's not always the case. If there is no query in the error log, please enable the general log ( SET GLOBAL general_log=1 ) and run the queries from the file again. After it crashes, check the general log, the last query will be the crashing one.
            orobitg Miquel Orobitg added a comment - - edited

            I've logged the last queries of two different crashes:

            The first one -> Doing backups of all databases (I have to deactivate this process again):

            17229 Query     SELECT TRIGGER_NAME FROM INFORMATION_SCHEMA.TRIGGERS WHERE EVENT_OBJECT_SCHEMA = DATABASE() AND EVENT_OBJECT_TABLE = 'Wkp__Workplace'
                           17229 Query     SET SESSION character_set_results = 'utf8'
                           17229 Query     ROLLBACK TO SAVEPOINT sp
                           17229 Query     SELECT engine FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'Wkp__WorkplaceTranslation'
                           17229 Query     SET SQL_QUOTE_SHOW_CREATE=1
                           17229 Query     SET SESSION character_set_results = 'binary'
                           17229 Query     show create table `Wkp__WorkplaceTranslation`
                           17229 Query     SET SESSION character_set_results = 'utf8'
                           17229 Query     show fields from `Wkp__WorkplaceTranslation`
                           17229 Query     SELECT /*!40001 SQL_NO_CACHE* / * FROM `Wkp__WorkplaceTranslation`
                           17229 Query     SET SESSION character_set_results = 'binary'
                           17229 Init DB   DB16883
                           17229 Query     select @@collation_database
                           17229 Query     SELECT TRIGGER_NAME FROM INFORMATION_SCHEMA.TRIGGERS WHERE EVENT_OBJECT_SCHEMA = DATABASE() AND EVENT_OBJECT_TABLE = 'Wkp__WorkplaceTranslation'
                           17229 Query     SET SESSION character_set_results = 'utf8'
                           17229 Query     ROLLBACK TO SAVEPOINT sp
                           17229 Query     SELECT engine FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'XXX'
            

            The second one it broke when I've opened the phpmyadmin and I've tried to navigate to the following database DB16883 (I need to test if with other databases it also happens):

            181106  9:26:24   485 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
                              506 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
                              226 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
                              514 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
                              484 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
                              472 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
                              507 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
                              490 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
                              531 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
                              512 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
                              553 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
                              530 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
                              546 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
                              554 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
                             1234 Connect   phpmyadmin@localhost as anonymous on 
                             1235 Connect   root@localhost as anonymous on 
                             1235 Query     SET CHARACTER SET 'utf8mb4'
                             1235 Query     SET collation_connection = 'utf8mb4_general_ci'
                             1235 Query     SELECT `SCHEMA_NAME` FROM `INFORMATION_SCHEMA`.`SCHEMATA`, (select DB_first_level from ( SELECT distinct SUBSTRING_INDEX(SCHEMA_NAME, '_', 1) DB_first_level FROM INFORMATION_SCHEMA.SCHEMATA WHERE TRUE AND `SCHEMA_NAME` LIKE '%DB16883%' ) t ORDER BY DB_first_level ASC LIMIT 0, 250) t2 where 1 = locate(concat(DB_first_level, '_'), concat(SCHEMA_NAME, '_')) order by SCHEMA_NAME ASC
                             1235 Query     SELECT COUNT(*) FROM `INFORMATION_SCHEMA`.`TABLES` WHERE `TABLE_SCHEMA`='DB16883' AND `TABLE_TYPE`='BASE TABLE'
                              556 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
                              551 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:24'      ORDER BY TE.ServerStartDate ASC
            181106  9:26:25   568 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:25'      ORDER BY TE.ServerStartDate ASC
                              567 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:25'      ORDER BY TE.ServerStartDate ASC
                              611 Query     SELECT TE.* FROM Tsk__TaskExecution TE      INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled=0)      WHERE TE.ServerStartDate<='2018-11-06 09:26:25'      ORDER BY TE.ServerStartDate ASC
            

            I haven't already tested the sql I've mentioned before.

            Moreover, when it crashes the mysqld daemon process stills running. This does not allow me to start mysql again since the start process hangs. When I kill the daemon process, after few seconds it is automatically activated again, not allowing me to restart the mariadb service.

            orobitg Miquel Orobitg added a comment - - edited I've logged the last queries of two different crashes: The first one -> Doing backups of all databases (I have to deactivate this process again): 17229 Query SELECT TRIGGER_NAME FROM INFORMATION_SCHEMA.TRIGGERS WHERE EVENT_OBJECT_SCHEMA = DATABASE() AND EVENT_OBJECT_TABLE = 'Wkp__Workplace' 17229 Query SET SESSION character_set_results = 'utf8' 17229 Query ROLLBACK TO SAVEPOINT sp 17229 Query SELECT engine FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'Wkp__WorkplaceTranslation' 17229 Query SET SQL_QUOTE_SHOW_CREATE= 1 17229 Query SET SESSION character_set_results = 'binary' 17229 Query show create table `Wkp__WorkplaceTranslation` 17229 Query SET SESSION character_set_results = 'utf8' 17229 Query show fields from `Wkp__WorkplaceTranslation` 17229 Query SELECT /*! 40001 SQL_NO_CACHE* / * FROM `Wkp__WorkplaceTranslation` 17229 Query SET SESSION character_set_results = 'binary' 17229 Init DB DB16883 17229 Query select @ @collation_database 17229 Query SELECT TRIGGER_NAME FROM INFORMATION_SCHEMA.TRIGGERS WHERE EVENT_OBJECT_SCHEMA = DATABASE() AND EVENT_OBJECT_TABLE = 'Wkp__WorkplaceTranslation' 17229 Query SET SESSION character_set_results = 'utf8' 17229 Query ROLLBACK TO SAVEPOINT sp 17229 Query SELECT engine FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'XXX' The second one it broke when I've opened the phpmyadmin and I've tried to navigate to the following database DB16883 (I need to test if with other databases it also happens): 181106 9 : 26 : 24 485 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 506 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 226 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 514 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 484 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 472 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 507 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 490 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 531 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 512 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 553 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 530 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 546 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 554 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 1234 Connect phpmyadmin @localhost as anonymous on 1235 Connect root @localhost as anonymous on 1235 Query SET CHARACTER SET 'utf8mb4' 1235 Query SET collation_connection = 'utf8mb4_general_ci' 1235 Query SELECT `SCHEMA_NAME` FROM `INFORMATION_SCHEMA`.`SCHEMATA`, (select DB_first_level from ( SELECT distinct SUBSTRING_INDEX(SCHEMA_NAME, '_' , 1 ) DB_first_level FROM INFORMATION_SCHEMA.SCHEMATA WHERE TRUE AND `SCHEMA_NAME` LIKE '%DB16883%' ) t ORDER BY DB_first_level ASC LIMIT 0 , 250 ) t2 where 1 = locate(concat(DB_first_level, '_' ), concat(SCHEMA_NAME, '_' )) order by SCHEMA_NAME ASC 1235 Query SELECT COUNT(*) FROM `INFORMATION_SCHEMA`.`TABLES` WHERE `TABLE_SCHEMA`= 'DB16883' AND `TABLE_TYPE`= 'BASE TABLE' 556 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 551 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:24' ORDER BY TE.ServerStartDate ASC 181106 9 : 26 : 25 568 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:25' ORDER BY TE.ServerStartDate ASC 567 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:25' ORDER BY TE.ServerStartDate ASC 611 Query SELECT TE.* FROM Tsk__TaskExecution TE INNER JOIN Tsk__Task T ON (T.TaskId=TE.TaskId AND T.Disabled= 0 ) WHERE TE.ServerStartDate<= '2018-11-06 09:26:25' ORDER BY TE.ServerStartDate ASC I haven't already tested the sql I've mentioned before. Moreover, when it crashes the mysqld daemon process stills running. This does not allow me to start mysql again since the start process hangs. When I kill the daemon process, after few seconds it is automatically activated again, not allowing me to restart the mariadb service.

            I've done another test to confirm that DB16883 database is the problem. I've run mysqlcheck for all databses and mariadb service has crashed after analysing the tables of this database.

            DB16883.Wkp__Workplace                             OK
            DB16883.Wkp__WorkplaceTranslation                  OK
            mysqlcheck: Got error: 2013: Lost connection to MySQL server during query when executing 'CHECK TABLE ... '
            

            Wkp__WorkplaceTranslation is the last table of the database:

            CREATE TABLE `Wkp__WorkplaceTranslation` (
              `WorkplaceId` int(11) NOT NULL,
              `LanguageId` int(11) NOT NULL,
              `Name` varchar(100) DEFAULT NULL,
              PRIMARY KEY (`WorkplaceId`,`LanguageId`),
              KEY `WorkplaceTranslation_Language_FK` (`LanguageId`),
              CONSTRAINT `WorkplaceTranslation_Language_FK` FOREIGN KEY (`LanguageId`) REFERENCES `Language` (`LanguageId`) ON UPDATE CASCADE,
              CONSTRAINT `WorkplaceTranslation_Workplace_FK` FOREIGN KEY (`WorkplaceId`) REFERENCES `Wkp__Workplace` (`WorkplaceId`) ON DELETE CASCADE ON UPDATE CASCADE
            ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
            

            After, I've drop this database (before I've stored a backup) and I have run mysqlcheck again and it has finished correctly.

            I cannot understand what happens because this server has other 300 databases with the same schema.

            Finally, I've restored the backup to another server (with less databases) with the same mariaDB version. I've run similar tests that had been crashed in the original server, and these tests work correctly.

            orobitg Miquel Orobitg added a comment - I've done another test to confirm that DB16883 database is the problem. I've run mysqlcheck for all databses and mariadb service has crashed after analysing the tables of this database. DB16883.Wkp__Workplace OK DB16883.Wkp__WorkplaceTranslation OK mysqlcheck: Got error: 2013 : Lost connection to MySQL server during query when executing 'CHECK TABLE ... ' Wkp__WorkplaceTranslation is the last table of the database: CREATE TABLE `Wkp__WorkplaceTranslation` ( `WorkplaceId` int (11) NOT NULL , `LanguageId` int (11) NOT NULL , ` Name ` varchar (100) DEFAULT NULL , PRIMARY KEY (`WorkplaceId`,`LanguageId`), KEY `WorkplaceTranslation_Language_FK` (`LanguageId`), CONSTRAINT `WorkplaceTranslation_Language_FK` FOREIGN KEY (`LanguageId`) REFERENCES `Language` (`LanguageId`) ON UPDATE CASCADE , CONSTRAINT `WorkplaceTranslation_Workplace_FK` FOREIGN KEY (`WorkplaceId`) REFERENCES `Wkp__Workplace` (`WorkplaceId`) ON DELETE CASCADE ON UPDATE CASCADE ) ENGINE=InnoDB DEFAULT CHARSET=utf8; After, I've drop this database (before I've stored a backup) and I have run mysqlcheck again and it has finished correctly. I cannot understand what happens because this server has other 300 databases with the same schema. Finally, I've restored the backup to another server (with less databases) with the same mariaDB version. I've run similar tests that had been crashed in the original server, and these tests work correctly.

            Wkp__WorkplaceTranslation is not the table, for this one check went okay.
            The table in question is the next one, the name of which hides under "..." obfuscation:

            mysqlcheck: Got error: 2013: Lost connection to MySQL server during query when executing 'CHECK TABLE ... '
            

            Also, if the part of the general log you previously pasted reflects the very last records, then the problematic table (either the same or a different one) would be the one obfuscated as "XXX":

            SELECT engine FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'XXX'
            

            elenst Elena Stepanova added a comment - Wkp__WorkplaceTranslation is not the table, for this one check went okay. The table in question is the next one, the name of which hides under "..." obfuscation: mysqlcheck: Got error: 2013: Lost connection to MySQL server during query when executing 'CHECK TABLE ... ' Also, if the part of the general log you previously pasted reflects the very last records, then the problematic table (either the same or a different one) would be the one obfuscated as "XXX": SELECT engine FROM INFORMATION_SCHEMA.TABLES WHERE table_schema = DATABASE() AND table_name = 'XXX'

            People

              Unassigned Unassigned
              orobitg Miquel Orobitg
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.