Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-20991

MariaDB server is not starting after updating from previous version

Details

    Description

      Hi,

      I have many server running MariaDB 10.1.x and 10.2.x

      They have all stopped working after the latest update from 10.1.41 to 10.1.42 and 10.2.27 to 10.2.28

      Error message as below, systemctl status mariadb.service

      ● mariadb.service - MariaDB 10.2.28 database server
      Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
      Drop-In: /etc/systemd/system/mariadb.service.d
      └─migrated-from-my.cnf-settings.conf
      /usr/lib/systemd/system/mariadb.service.d
      └─respawn.conf
      Active: activating (auto-restart) (Result: signal) since Wed 2019-11-06 07:02:01 UTC; 4s ago
      Docs: man:mysqld(8)
      https://mariadb.com/kb/en/library/systemd/
      Process: 5831 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS --basedir=/usr $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=killed, signal=ABRT)
      Process: 5800 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= || VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ] && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/SUCCESS)
      Process: 5798 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
      Main PID: 5831 (code=killed, signal=ABRT)

      Nov 06 07:02:01 xxxx.xxxx.xyz systemd[1]: Failed to start MariaDB 10.2.28 database server.
      Nov 06 07:02:01 xxxx.xxxx.xyz systemd[1]: Unit mariadb.service entered failed state.
      Nov 06 07:02:01 xxxx.xxxx.xyz systemd[1]: mariadb.service failed.

      Attachments

        Issue Links

          Activity

            Can you please confirm if this could be a duplicate of MDEV-20987? If you have InnoDB tables that contain both FULLTEXT INDEX and FOREIGN KEY, that should be the case.

            What is in the server error log? I suppose that you could dump it with

            journalctl -xe
            

            Case-insensitively search for 'assert' or 'signal'.

            marko Marko Mäkelä added a comment - Can you please confirm if this could be a duplicate of MDEV-20987 ? If you have InnoDB tables that contain both FULLTEXT INDEX and FOREIGN KEY , that should be the case. What is in the server error log? I suppose that you could dump it with journalctl -xe Case-insensitively search for 'assert' or 'signal'.
            AusWeb Priyan A added a comment -

            ~Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Hope that's ok; if not, decrease some variables in the equation.
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Thread pointer: 0x0
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Attempting backtrace. You can use the following information to find out
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: where mysqld died. If you see no messages after this, something went
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: terribly wrong...
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: stack_bottom = 0x0 thread_stack 0x49000
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x56515b88b72e]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(handle_fatal_signal+0x30d)[0x56515b31005d]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: sigaction.c:0(__restore_rt)[0x7fdddcde65f0]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: :0(__GI_raise)[0x7fdddb0b7337]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: :0(__GI_abort)[0x7fdddb0b8a28]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(+0x431a59)[0x56515b0bda59]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(+0x9fe9e7)[0x56515b68a9e7]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(+0xa14bf2)[0x56515b6a0bf2]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(+0xa1509d)[0x56515b6a109d]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(+0xa16019)[0x56515b6a2019]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(+0x9fe480)[0x56515b68a480]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(+0xa632b8)[0x56515b6ef2b8]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(+0xa6bbad)[0x56515b6f7bad]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(+0x8820ca)[0x56515b50e0ca]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(+0x93faaa)[0x56515b5cbaaa]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(+0x824501)[0x56515b4b0501]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x64)[0x56515b312714]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(+0x4fc1a0)[0x56515b1881a0]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(_Z11plugin_initPiPPci+0x9a2)[0x56515b189a72]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(+0x455241)[0x56515b0e1241]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x1e39)[0x56515b0e67c9]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fdddb0a3505]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: /usr/sbin/mysqld(+0x44e68d)[0x56515b0da68d]
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: information that should help you find out what is causing the crash.
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Writing a core file...
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Working directory at /var/lib/mysql
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Resource Limits:
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Limit Soft Limit Hard Limit Units
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max cpu time unlimited unlimited seconds
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max file size unlimited unlimited bytes
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max data size unlimited unlimited bytes
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max stack size 8388608 unlimited bytes
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max core file size 0 unlimited bytes
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max resident set unlimited unlimited bytes
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max processes 31086 31086 processes
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max open files 16364 16364 files
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max locked memory 65536 65536 bytes
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max address space unlimited unlimited bytes
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max file locks unlimited unlimited locks
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max pending signals 31086 31086 signals
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max msgqueue size 819200 819200 bytes
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max nice priority 0 0
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max realtime priority 0 0
            Nov 06 07:15:28 xxxx.xxxx.xyz mysqld[13443]: Max realtime timeout unlimited unlimited us
            Nov 06 07:15:28 xxxx.xxxx.xyz systemd[1]: mariadb.service: main process exited, code=killed, status=6/ABRT
            Nov 06 07:15:28 xxxx.xxxx.xyz systemd[1]: Failed to start MariaDB 10.2.28 database server.
            – Subject: Unit mariadb.service has failed
            – Defined-By: systemd
            – Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel

            -- Unit mariadb.service has failed.

            -- The result is failed.
            Nov 06 07:15:28 xxxx.xxxx.xyz systemd[1]: Unit mariadb.service entered failed state.
            Nov 06 07:15:28 xxxx.xxxx.xyz systemd[1]: mariadb.service failed.~

            AusWeb Priyan A added a comment - ~Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Hope that's ok; if not, decrease some variables in the equation. Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Thread pointer: 0x0 Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Attempting backtrace. You can use the following information to find out Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : where mysqld died. If you see no messages after this, something went Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : terribly wrong... Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : stack_bottom = 0x0 thread_stack 0x49000 Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(my_print_stacktrace+0x2e) [0x56515b88b72e] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(handle_fatal_signal+0x30d) [0x56515b31005d] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : sigaction.c:0(__restore_rt) [0x7fdddcde65f0] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : :0(__GI_raise) [0x7fdddb0b7337] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : :0(__GI_abort) [0x7fdddb0b8a28] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(+0x431a59) [0x56515b0bda59] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(+0x9fe9e7) [0x56515b68a9e7] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(+0xa14bf2) [0x56515b6a0bf2] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(+0xa1509d) [0x56515b6a109d] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(+0xa16019) [0x56515b6a2019] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(+0x9fe480) [0x56515b68a480] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(+0xa632b8) [0x56515b6ef2b8] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(+0xa6bbad) [0x56515b6f7bad] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(+0x8820ca) [0x56515b50e0ca] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(+0x93faaa) [0x56515b5cbaaa] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(+0x824501) [0x56515b4b0501] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x64) [0x56515b312714] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(+0x4fc1a0) [0x56515b1881a0] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(_Z11plugin_initPiPPci+0x9a2) [0x56515b189a72] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(+0x455241) [0x56515b0e1241] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x1e39) [0x56515b0e67c9] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fdddb0a3505] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : /usr/sbin/mysqld(+0x44e68d) [0x56515b0da68d] Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : information that should help you find out what is causing the crash. Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Writing a core file... Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Working directory at /var/lib/mysql Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Resource Limits: Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Limit Soft Limit Hard Limit Units Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max cpu time unlimited unlimited seconds Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max file size unlimited unlimited bytes Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max data size unlimited unlimited bytes Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max stack size 8388608 unlimited bytes Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max core file size 0 unlimited bytes Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max resident set unlimited unlimited bytes Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max processes 31086 31086 processes Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max open files 16364 16364 files Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max locked memory 65536 65536 bytes Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max address space unlimited unlimited bytes Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max file locks unlimited unlimited locks Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max pending signals 31086 31086 signals Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max msgqueue size 819200 819200 bytes Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max nice priority 0 0 Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max realtime priority 0 0 Nov 06 07:15:28 xxxx.xxxx.xyz mysqld [13443] : Max realtime timeout unlimited unlimited us Nov 06 07:15:28 xxxx.xxxx.xyz systemd [1] : mariadb.service: main process exited, code=killed, status=6/ABRT Nov 06 07:15:28 xxxx.xxxx.xyz systemd [1] : Failed to start MariaDB 10.2.28 database server. – Subject: Unit mariadb.service has failed – Defined-By: systemd – Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel – -- Unit mariadb.service has failed. – -- The result is failed. Nov 06 07:15:28 xxxx.xxxx.xyz systemd [1] : Unit mariadb.service entered failed state. Nov 06 07:15:28 xxxx.xxxx.xyz systemd [1] : mariadb.service failed.~
            FrankNL Frank added a comment -

            The same thing is happening on one of our servers that got updated from 10.3.18 to 10.3.19 this morning.

            nov 06 08:55:02 hostname.example.com systemd[1]: mariadb.service: Service hold-off time over, scheduling restart.
            nov 06 08:55:02 hostname.example.com systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 5.
            nov 06 08:55:02 hostname.example.com systemd[1]: Stopped MariaDB 10.3.19 database server.
            nov 06 08:55:02 hostname.example.com systemd[1]: Starting MariaDB 10.3.19 database server...
            root@services01:~# journalctl -xe
            nov 06 08:55:03 hostname.example.com mysqld[3623]: or misconfigured. This error can also be caused by malfunctioning hardware.
            nov 06 08:55:03 hostname.example.com mysqld[3623]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
            nov 06 08:55:03 hostname.example.com mysqld[3623]: We will try our best to scrape up some info that will hopefully help
            nov 06 08:55:03 hostname.example.com mysqld[3623]: diagnose the problem, but since we have already crashed,
            nov 06 08:55:03 hostname.example.com mysqld[3623]: something is definitely wrong and this may fail.
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Server version: 10.3.19-MariaDB-1:10.3.19+maria~bionic-log
            nov 06 08:55:03 hostname.example.com mysqld[3623]: key_buffer_size=134217728
            nov 06 08:55:03 hostname.example.com mysqld[3623]: read_buffer_size=2097152
            nov 06 08:55:03 hostname.example.com mysqld[3623]: max_used_connections=0
            nov 06 08:55:03 hostname.example.com mysqld[3623]: max_threads=102
            nov 06 08:55:03 hostname.example.com mysqld[3623]: thread_count=0
            nov 06 08:55:03 hostname.example.com mysqld[3623]: It is possible that mysqld could use up to
            nov 06 08:55:03 hostname.example.com mysqld[3623]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 760034 K  bytes of memory
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Hope that's ok; if not, decrease some variables in the equation.
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Thread pointer: 0x0
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Attempting backtrace. You can use the following information to find out
            nov 06 08:55:03 hostname.example.com mysqld[3623]: where mysqld died. If you see no messages after this, something went
            nov 06 08:55:03 hostname.example.com mysqld[3623]: terribly wrong...
            nov 06 08:55:03 hostname.example.com mysqld[3623]: stack_bottom = 0x0 thread_stack 0x49000
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x5636c38721fe]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(handle_fatal_signal+0x515)[0x5636c33088a5]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7fde46b2b890]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7fde4603fe97]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /lib/x86_64-linux-gnu/libc.so.6(abort+0x141)[0x7fde46041801]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0x4a55d8)[0x5636c30535d8]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0xa9f868)[0x5636c364d868]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0xab40be)[0x5636c36620be]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0xab4841)[0x5636c3662841]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0xab5a14)[0x5636c3663a14]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0xaa4936)[0x5636c3652936]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0xafd898)[0x5636c36ab898]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0xb06581)[0x5636c36b4581]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0x935b93)[0x5636c34e3b93]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0x9ecc0f)[0x5636c359ac0f]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0x8dc369)[0x5636c348a369]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x62)[0x5636c330ae72]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0x58d8e9)[0x5636c313b8e9]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(_Z11plugin_initPiPPci+0x85a)[0x5636c313ca5a]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0x4c45f1)[0x5636c30725f1]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x53a)[0x5636c307887a]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7fde46022b97]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(_start+0x2a)[0x5636c306b71a]
            nov 06 08:55:03 hostname.example.com mysqld[3623]: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
            nov 06 08:55:03 hostname.example.com mysqld[3623]: information that should help you find out what is causing the crash.
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Writing a core file...
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Working directory at /var/lib/mysql
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Resource Limits:
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Limit                     Soft Limit           Hard Limit           Units
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max cpu time              unlimited            unlimited            seconds
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max file size             unlimited            unlimited            bytes
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max data size             unlimited            unlimited            bytes
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max stack size            8388608              unlimited            bytes
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max core file size        0                    unlimited            bytes
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max resident set          unlimited            unlimited            bytes
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max processes             3736                 3736                 processes
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max open files            16364                16364                files
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max locked memory         16777216             16777216             bytes
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max address space         unlimited            unlimited            bytes
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max file locks            unlimited            unlimited            locks
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max pending signals       3736                 3736                 signals
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max msgqueue size         819200               819200               bytes
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max nice priority         0                    0
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max realtime priority     0                    0
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Max realtime timeout      unlimited            unlimited            us
            nov 06 08:55:03 hostname.example.com mysqld[3623]: Core pattern: |/usr/share/apport/apport %p %s %c %d %P
            nov 06 08:55:04 hostname.example.com systemd[1]: mariadb.service: Main process exited, code=dumped, status=6/ABRT
            nov 06 08:55:04 hostname.example.com systemd[1]: mariadb.service: Failed with result 'core-dump'.
            nov 06 08:55:04 hostname.example.com systemd[1]: Failed to start MariaDB 10.3.19 database server.
            -- Subject: Unit mariadb.service has failed
            -- Defined-By: systemd
            -- Support: http://www.ubuntu.com/support
            -- 
            -- Unit mariadb.service has failed.
            -- 
            -- The result is RESULT.
            

            This server is running Ubuntu 18.04.3 and is using the provided apt repo.

            FrankNL Frank added a comment - The same thing is happening on one of our servers that got updated from 10.3.18 to 10.3.19 this morning. nov 06 08:55:02 hostname.example.com systemd[1]: mariadb.service: Service hold-off time over, scheduling restart. nov 06 08:55:02 hostname.example.com systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 5. nov 06 08:55:02 hostname.example.com systemd[1]: Stopped MariaDB 10.3.19 database server. nov 06 08:55:02 hostname.example.com systemd[1]: Starting MariaDB 10.3.19 database server... root@services01:~# journalctl -xe nov 06 08:55:03 hostname.example.com mysqld[3623]: or misconfigured. This error can also be caused by malfunctioning hardware. nov 06 08:55:03 hostname.example.com mysqld[3623]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs nov 06 08:55:03 hostname.example.com mysqld[3623]: We will try our best to scrape up some info that will hopefully help nov 06 08:55:03 hostname.example.com mysqld[3623]: diagnose the problem, but since we have already crashed, nov 06 08:55:03 hostname.example.com mysqld[3623]: something is definitely wrong and this may fail. nov 06 08:55:03 hostname.example.com mysqld[3623]: Server version: 10.3.19-MariaDB-1:10.3.19+maria~bionic-log nov 06 08:55:03 hostname.example.com mysqld[3623]: key_buffer_size=134217728 nov 06 08:55:03 hostname.example.com mysqld[3623]: read_buffer_size=2097152 nov 06 08:55:03 hostname.example.com mysqld[3623]: max_used_connections=0 nov 06 08:55:03 hostname.example.com mysqld[3623]: max_threads=102 nov 06 08:55:03 hostname.example.com mysqld[3623]: thread_count=0 nov 06 08:55:03 hostname.example.com mysqld[3623]: It is possible that mysqld could use up to nov 06 08:55:03 hostname.example.com mysqld[3623]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 760034 K bytes of memory nov 06 08:55:03 hostname.example.com mysqld[3623]: Hope that's ok; if not, decrease some variables in the equation. nov 06 08:55:03 hostname.example.com mysqld[3623]: Thread pointer: 0x0 nov 06 08:55:03 hostname.example.com mysqld[3623]: Attempting backtrace. You can use the following information to find out nov 06 08:55:03 hostname.example.com mysqld[3623]: where mysqld died. If you see no messages after this, something went nov 06 08:55:03 hostname.example.com mysqld[3623]: terribly wrong... nov 06 08:55:03 hostname.example.com mysqld[3623]: stack_bottom = 0x0 thread_stack 0x49000 nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x5636c38721fe] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(handle_fatal_signal+0x515)[0x5636c33088a5] nov 06 08:55:03 hostname.example.com mysqld[3623]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7fde46b2b890] nov 06 08:55:03 hostname.example.com mysqld[3623]: /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7fde4603fe97] nov 06 08:55:03 hostname.example.com mysqld[3623]: /lib/x86_64-linux-gnu/libc.so.6(abort+0x141)[0x7fde46041801] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0x4a55d8)[0x5636c30535d8] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0xa9f868)[0x5636c364d868] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0xab40be)[0x5636c36620be] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0xab4841)[0x5636c3662841] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0xab5a14)[0x5636c3663a14] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0xaa4936)[0x5636c3652936] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0xafd898)[0x5636c36ab898] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0xb06581)[0x5636c36b4581] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0x935b93)[0x5636c34e3b93] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0x9ecc0f)[0x5636c359ac0f] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0x8dc369)[0x5636c348a369] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x62)[0x5636c330ae72] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0x58d8e9)[0x5636c313b8e9] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(_Z11plugin_initPiPPci+0x85a)[0x5636c313ca5a] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(+0x4c45f1)[0x5636c30725f1] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x53a)[0x5636c307887a] nov 06 08:55:03 hostname.example.com mysqld[3623]: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7fde46022b97] nov 06 08:55:03 hostname.example.com mysqld[3623]: /usr/sbin/mysqld(_start+0x2a)[0x5636c306b71a] nov 06 08:55:03 hostname.example.com mysqld[3623]: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains nov 06 08:55:03 hostname.example.com mysqld[3623]: information that should help you find out what is causing the crash. nov 06 08:55:03 hostname.example.com mysqld[3623]: Writing a core file... nov 06 08:55:03 hostname.example.com mysqld[3623]: Working directory at /var/lib/mysql nov 06 08:55:03 hostname.example.com mysqld[3623]: Resource Limits: nov 06 08:55:03 hostname.example.com mysqld[3623]: Limit Soft Limit Hard Limit Units nov 06 08:55:03 hostname.example.com mysqld[3623]: Max cpu time unlimited unlimited seconds nov 06 08:55:03 hostname.example.com mysqld[3623]: Max file size unlimited unlimited bytes nov 06 08:55:03 hostname.example.com mysqld[3623]: Max data size unlimited unlimited bytes nov 06 08:55:03 hostname.example.com mysqld[3623]: Max stack size 8388608 unlimited bytes nov 06 08:55:03 hostname.example.com mysqld[3623]: Max core file size 0 unlimited bytes nov 06 08:55:03 hostname.example.com mysqld[3623]: Max resident set unlimited unlimited bytes nov 06 08:55:03 hostname.example.com mysqld[3623]: Max processes 3736 3736 processes nov 06 08:55:03 hostname.example.com mysqld[3623]: Max open files 16364 16364 files nov 06 08:55:03 hostname.example.com mysqld[3623]: Max locked memory 16777216 16777216 bytes nov 06 08:55:03 hostname.example.com mysqld[3623]: Max address space unlimited unlimited bytes nov 06 08:55:03 hostname.example.com mysqld[3623]: Max file locks unlimited unlimited locks nov 06 08:55:03 hostname.example.com mysqld[3623]: Max pending signals 3736 3736 signals nov 06 08:55:03 hostname.example.com mysqld[3623]: Max msgqueue size 819200 819200 bytes nov 06 08:55:03 hostname.example.com mysqld[3623]: Max nice priority 0 0 nov 06 08:55:03 hostname.example.com mysqld[3623]: Max realtime priority 0 0 nov 06 08:55:03 hostname.example.com mysqld[3623]: Max realtime timeout unlimited unlimited us nov 06 08:55:03 hostname.example.com mysqld[3623]: Core pattern: |/usr/share/apport/apport %p %s %c %d %P nov 06 08:55:04 hostname.example.com systemd[1]: mariadb.service: Main process exited, code=dumped, status=6/ABRT nov 06 08:55:04 hostname.example.com systemd[1]: mariadb.service: Failed with result 'core-dump'. nov 06 08:55:04 hostname.example.com systemd[1]: Failed to start MariaDB 10.3.19 database server. -- Subject: Unit mariadb.service has failed -- Defined-By: systemd -- Support: http://www.ubuntu.com/support -- -- Unit mariadb.service has failed. -- -- The result is RESULT. This server is running Ubuntu 18.04.3 and is using the provided apt repo.
            AusWeb Priyan A added a comment -

            Same issue everywhere! Downgrading to previous version seems to be helping,

            AusWeb Priyan A added a comment - Same issue everywhere! Downgrading to previous version seems to be helping,

            FrankNL, unfortunately your log snippet starts after the actual interesting part (the assertion failure message).
            The stack traces (with unresolved function names) posted by you and AusWeb look similar enough to the one in MDEV-20988, so I think that this report is a duplicate of MDEV-20987.

            marko Marko Mäkelä added a comment - FrankNL , unfortunately your log snippet starts after the actual interesting part (the assertion failure message). The stack traces (with unresolved function names) posted by you and AusWeb look similar enough to the one in MDEV-20988 , so I think that this report is a duplicate of MDEV-20987 .
            tcurilla Tomas Curilla added a comment -

            After upgrade from 10.3.18 to 10.3.19 on centos 7

            ```
            Nov 6 03:41:05 hostname yum[22849]: Updated: MariaDB-compat-10.3.19-1.el7.centos.x86_64
            Nov 6 03:41:06 hostname yum[22849]: Updated: MariaDB-common-10.3.19-1.el7.centos.x86_64
            Nov 6 03:41:08 hostname yum[22849]: Updated: MariaDB-client-10.3.19-1.el7.centos.x86_64
            Nov 6 03:41:11 hostname yum[22849]: Updated: galera-25.3.28-1.rhel7.el7.centos.x86_64
            Nov 6 03:41:33 hostname kernel: SELinux: Converting 2357 SID table entries...
            Nov 6 03:41:35 hostname yum[22849]: Updated: MariaDB-server-10.3.19-1.el7.centos.x86_64
            Nov 6 03:41:36 hostname dbus[988]: [system] Reloaded configuration
            Nov 6 03:41:36 hostname systemd: Reloading.
            Nov 6 03:41:38 hostname systemd: Stopping MariaDB 10.3.19 database server...
            Nov 6 03:41:39 hostname mysqld: 2019-11-06 3:41:39 0 [Note] /usr/sbin/mysqld (initiated by: unknown): Normal shutdown
            Nov 6 03:41:39 hostname mysqld: 2019-11-06 3:41:39 0 [Note] Event Scheduler: Purging the queue. 0 events
            Nov 6 03:41:49 hostname mysqld: 2019-11-06 3:41:49 0 [Note] InnoDB: FTS optimize thread exiting.
            Nov 6 03:42:57 hostname mysqld: 2019-11-06 3:42:57 0 [Note] InnoDB: Starting shutdown...
            Nov 6 03:42:57 hostname mysqld: 2019-11-06 3:42:57 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool
            Nov 6 03:42:57 hostname mysqld: 2019-11-06 3:42:57 0 [Note] InnoDB: Instance 0, restricted to 8192 pages due to innodb_buf_pool_dump_pct=25
            Nov 6 03:42:57 hostname mysqld: 2019-11-06 3:42:57 0 [Note] InnoDB: Buffer pool(s) dump completed at 191106 3:42:57
            Nov 6 03:43:08 hostname systemd: mariadb.service stop-sigterm timed out. Skipping SIGKILL.
            Nov 6 03:43:36 hostname mysqld: 2019-11-06 3:43:36 0 [Note] InnoDB: Shutdown completed; log sequence number 396040262362; transaction id 2891788452
            Nov 6 03:43:36 hostname mysqld: 2019-11-06 3:43:36 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
            Nov 6 03:43:37 hostname mysqld: 2019-11-06 3:43:37 0 [Note] /usr/sbin/mysqld: Shutdown complete
            Nov 6 03:43:39 hostname systemd: Stopped MariaDB 10.3.19 database server.
            Nov 6 03:43:39 hostname systemd: Unit mariadb.service entered failed state.
            Nov 6 03:43:39 hostname systemd: mariadb.service failed.
            Nov 6 03:43:39 hostname systemd: Starting MariaDB 10.3.19 database server...
            Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] /usr/sbin/mysqld (mysqld 10.3.19-MariaDB) starting as process 23252 ...
            Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Using Linux native AIO
            Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
            Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Uses event mutexes
            Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Compressed tables use zlib 1.2.7
            Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Number of pools: 1
            Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Using SSE2 crc32 instructions
            Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Initializing buffer pool, total size = 512M, instances = 1, chunk size = 128M
            Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Completed initialization of buffer pool
            Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
            Nov 6 03:43:40 hostname mysqld: 2019-11-06 03:43:40 0x7f2a006af8c0 InnoDB: Assertion failure in file /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.3.19/storage/innobase/dict/dict0dict.cc line 1443
            Nov 6 03:43:40 hostname mysqld: InnoDB: Failing assertion: table->can_be_evicted
            Nov 6 03:43:40 hostname mysqld: InnoDB: We intentionally generate a memory trap.
            Nov 6 03:43:40 hostname mysqld: InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
            Nov 6 03:43:40 hostname mysqld: InnoDB: If you get repeated assertion failures or crashes, even
            Nov 6 03:43:40 hostname mysqld: InnoDB: immediately after the mysqld startup, there may be
            Nov 6 03:43:40 hostname mysqld: InnoDB: corruption in the InnoDB tablespace. Please refer to
            Nov 6 03:43:40 hostname mysqld: InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
            Nov 6 03:43:40 hostname mysqld: InnoDB: about forcing recovery.
            Nov 6 03:43:40 hostname mysqld: 191106 3:43:40 [ERROR] mysqld got signal 6 ;
            Nov 6 03:43:40 hostname mysqld: This could be because you hit a bug. It is also possible that this binary
            Nov 6 03:43:40 hostname mysqld: or one of the libraries it was linked against is corrupt, improperly built,
            Nov 6 03:43:40 hostname mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.
            Nov 6 03:43:40 hostname mysqld: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
            Nov 6 03:43:40 hostname mysqld: We will try our best to scrape up some info that will hopefully help
            Nov 6 03:43:40 hostname mysqld: diagnose the problem, but since we have already crashed,
            Nov 6 03:43:40 hostname mysqld: something is definitely wrong and this may fail.
            Nov 6 03:43:40 hostname mysqld: Server version: 10.3.19-MariaDB
            Nov 6 03:43:40 hostname mysqld: key_buffer_size=134217728
            Nov 6 03:43:40 hostname mysqld: read_buffer_size=131072
            Nov 6 03:43:40 hostname mysqld: max_used_connections=0
            Nov 6 03:43:40 hostname mysqld: max_threads=153
            Nov 6 03:43:40 hostname mysqld: thread_count=0
            Nov 6 03:43:40 hostname mysqld: It is possible that mysqld could use up to
            Nov 6 03:43:40 hostname mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467422 K bytes of memory
            Nov 6 03:43:40 hostname mysqld: Hope that's ok; if not, decrease some variables in the equation.
            Nov 6 03:43:40 hostname mysqld: Thread pointer: 0x0
            Nov 6 03:43:40 hostname mysqld: Attempting backtrace. You can use the following information to find out
            Nov 6 03:43:40 hostname mysqld: where mysqld died. If you see no messages after this, something went
            Nov 6 03:43:40 hostname mysqld: terribly wrong...
            Nov 6 03:43:40 hostname mysqld: stack_bottom = 0x0 thread_stack 0x49000
            Nov 6 03:43:40 hostname mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55961fefa58e]
            Nov 6 03:43:40 hostname mysqld: /usr/sbin/mysqld(handle_fatal_signal+0x30f)[0x55961f99754f]
            Nov 6 03:43:40 hostname mysqld: sigaction.c:0(__restore_rt)[0x7f2a0029a5f0]
            Nov 6 03:43:41 hostname mysqld: :0(__GI_raise)[0x7f29fe56b337]
            Nov 6 03:43:41 hostname mysqld: :0(__GI_abort)[0x7f29fe56ca28]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0x4d3560)[0x55961f6d7560]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xaebf97)[0x55961fceff97]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xb02603)[0x55961fd06603]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xb02a1d)[0x55961fd06a1d]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xb03976)[0x55961fd07976]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xaf0d50)[0x55961fcf4d50]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xb4cd08)[0x55961fd50d08]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xb578f2)[0x55961fd5b8f2]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0x984cf3)[0x55961fb88cf3]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xa3b0e8)[0x55961fc3f0e8]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0x92a001)[0x55961fb2e001]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x64)[0x55961f999dd4]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0x5c59f0)[0x55961f7c99f0]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(_Z11plugin_initPiPPci+0x9ba)[0x55961f7cab7a]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0x4f8bf1)[0x55961f6fcbf1]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x4ae)[0x55961f702dde]
            Nov 6 03:43:41 hostname mysqld: /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f29fe557505]
            Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0x4f280d)[0x55961f6f680d]
            Nov 6 03:43:41 hostname mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
            Nov 6 03:43:41 hostname mysqld: information that should help you find out what is causing the crash.
            Nov 6 03:43:41 hostname mysqld: Writing a core file...
            Nov 6 03:43:41 hostname mysqld: Working directory at /var/lib/mysql
            Nov 6 03:43:41 hostname mysqld: Resource Limits:
            Nov 6 03:43:41 hostname mysqld: Limit Soft Limit Hard Limit Units
            Nov 6 03:43:41 hostname mysqld: Max cpu time unlimited unlimited seconds
            Nov 6 03:43:41 hostname mysqld: Max file size unlimited unlimited bytes
            Nov 6 03:43:41 hostname mysqld: Max data size unlimited unlimited bytes
            Nov 6 03:43:41 hostname mysqld: Max stack size 8388608 unlimited bytes
            Nov 6 03:43:41 hostname mysqld: Max core file size 0 unlimited bytes
            Nov 6 03:43:41 hostname mysqld: Max resident set unlimited unlimited bytes
            Nov 6 03:43:41 hostname mysqld: Max processes 15064 15064 processes
            Nov 6 03:43:41 hostname mysqld: Max open files 16364 16364 files
            Nov 6 03:43:41 hostname mysqld: Max locked memory 65536 65536 bytes
            Nov 6 03:43:41 hostname mysqld: Max address space unlimited unlimited bytes
            Nov 6 03:43:41 hostname mysqld: Max file locks unlimited unlimited locks
            Nov 6 03:43:41 hostname mysqld: Max pending signals 15064 15064 signals
            Nov 6 03:43:41 hostname mysqld: Max msgqueue size 819200 819200 bytes
            Nov 6 03:43:41 hostname mysqld: Max nice priority 0 0
            Nov 6 03:43:41 hostname mysqld: Max realtime priority 0 0
            Nov 6 03:43:41 hostname mysqld: Max realtime timeout unlimited unlimited us
            Nov 6 03:43:41 hostname mysqld: Core pattern: core
            Nov 6 03:43:41 hostname systemd: mariadb.service: main process exited, code=killed, status=6/ABRT
            Nov 6 03:43:41 hostname systemd: Failed to start MariaDB 10.3.19 database server.
            Nov 6 03:43:41 hostname systemd: Unit mariadb.service entered failed state.
            Nov 6 03:43:41 hostname systemd: mariadb.service failed.
            Nov 6 03:43:46 hostname systemd: mariadb.service holdoff time over, scheduling restart.
            Nov 6 03:43:46 hostname systemd: Stopped MariaDB 10.3.19 database server.
            ```

            tcurilla Tomas Curilla added a comment - After upgrade from 10.3.18 to 10.3.19 on centos 7 ``` Nov 6 03:41:05 hostname yum [22849] : Updated: MariaDB-compat-10.3.19-1.el7.centos.x86_64 Nov 6 03:41:06 hostname yum [22849] : Updated: MariaDB-common-10.3.19-1.el7.centos.x86_64 Nov 6 03:41:08 hostname yum [22849] : Updated: MariaDB-client-10.3.19-1.el7.centos.x86_64 Nov 6 03:41:11 hostname yum [22849] : Updated: galera-25.3.28-1.rhel7.el7.centos.x86_64 Nov 6 03:41:33 hostname kernel: SELinux: Converting 2357 SID table entries... Nov 6 03:41:35 hostname yum [22849] : Updated: MariaDB-server-10.3.19-1.el7.centos.x86_64 Nov 6 03:41:36 hostname dbus [988] : [system] Reloaded configuration Nov 6 03:41:36 hostname systemd: Reloading. Nov 6 03:41:38 hostname systemd: Stopping MariaDB 10.3.19 database server... Nov 6 03:41:39 hostname mysqld: 2019-11-06 3:41:39 0 [Note] /usr/sbin/mysqld (initiated by: unknown): Normal shutdown Nov 6 03:41:39 hostname mysqld: 2019-11-06 3:41:39 0 [Note] Event Scheduler: Purging the queue. 0 events Nov 6 03:41:49 hostname mysqld: 2019-11-06 3:41:49 0 [Note] InnoDB: FTS optimize thread exiting. Nov 6 03:42:57 hostname mysqld: 2019-11-06 3:42:57 0 [Note] InnoDB: Starting shutdown... Nov 6 03:42:57 hostname mysqld: 2019-11-06 3:42:57 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool Nov 6 03:42:57 hostname mysqld: 2019-11-06 3:42:57 0 [Note] InnoDB: Instance 0, restricted to 8192 pages due to innodb_buf_pool_dump_pct=25 Nov 6 03:42:57 hostname mysqld: 2019-11-06 3:42:57 0 [Note] InnoDB: Buffer pool(s) dump completed at 191106 3:42:57 Nov 6 03:43:08 hostname systemd: mariadb.service stop-sigterm timed out. Skipping SIGKILL. Nov 6 03:43:36 hostname mysqld: 2019-11-06 3:43:36 0 [Note] InnoDB: Shutdown completed; log sequence number 396040262362; transaction id 2891788452 Nov 6 03:43:36 hostname mysqld: 2019-11-06 3:43:36 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" Nov 6 03:43:37 hostname mysqld: 2019-11-06 3:43:37 0 [Note] /usr/sbin/mysqld: Shutdown complete Nov 6 03:43:39 hostname systemd: Stopped MariaDB 10.3.19 database server. Nov 6 03:43:39 hostname systemd: Unit mariadb.service entered failed state. Nov 6 03:43:39 hostname systemd: mariadb.service failed. Nov 6 03:43:39 hostname systemd: Starting MariaDB 10.3.19 database server... Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] /usr/sbin/mysqld (mysqld 10.3.19-MariaDB) starting as process 23252 ... Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Using Linux native AIO Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Uses event mutexes Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Compressed tables use zlib 1.2.7 Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Number of pools: 1 Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Using SSE2 crc32 instructions Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Initializing buffer pool, total size = 512M, instances = 1, chunk size = 128M Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: Completed initialization of buffer pool Nov 6 03:43:40 hostname mysqld: 2019-11-06 3:43:40 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). Nov 6 03:43:40 hostname mysqld: 2019-11-06 03:43:40 0x7f2a006af8c0 InnoDB: Assertion failure in file /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.3.19/storage/innobase/dict/dict0dict.cc line 1443 Nov 6 03:43:40 hostname mysqld: InnoDB: Failing assertion: table->can_be_evicted Nov 6 03:43:40 hostname mysqld: InnoDB: We intentionally generate a memory trap. Nov 6 03:43:40 hostname mysqld: InnoDB: Submit a detailed bug report to https://jira.mariadb.org/ Nov 6 03:43:40 hostname mysqld: InnoDB: If you get repeated assertion failures or crashes, even Nov 6 03:43:40 hostname mysqld: InnoDB: immediately after the mysqld startup, there may be Nov 6 03:43:40 hostname mysqld: InnoDB: corruption in the InnoDB tablespace. Please refer to Nov 6 03:43:40 hostname mysqld: InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/ Nov 6 03:43:40 hostname mysqld: InnoDB: about forcing recovery. Nov 6 03:43:40 hostname mysqld: 191106 3:43:40 [ERROR] mysqld got signal 6 ; Nov 6 03:43:40 hostname mysqld: This could be because you hit a bug. It is also possible that this binary Nov 6 03:43:40 hostname mysqld: or one of the libraries it was linked against is corrupt, improperly built, Nov 6 03:43:40 hostname mysqld: or misconfigured. This error can also be caused by malfunctioning hardware. Nov 6 03:43:40 hostname mysqld: To report this bug, see https://mariadb.com/kb/en/reporting-bugs Nov 6 03:43:40 hostname mysqld: We will try our best to scrape up some info that will hopefully help Nov 6 03:43:40 hostname mysqld: diagnose the problem, but since we have already crashed, Nov 6 03:43:40 hostname mysqld: something is definitely wrong and this may fail. Nov 6 03:43:40 hostname mysqld: Server version: 10.3.19-MariaDB Nov 6 03:43:40 hostname mysqld: key_buffer_size=134217728 Nov 6 03:43:40 hostname mysqld: read_buffer_size=131072 Nov 6 03:43:40 hostname mysqld: max_used_connections=0 Nov 6 03:43:40 hostname mysqld: max_threads=153 Nov 6 03:43:40 hostname mysqld: thread_count=0 Nov 6 03:43:40 hostname mysqld: It is possible that mysqld could use up to Nov 6 03:43:40 hostname mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467422 K bytes of memory Nov 6 03:43:40 hostname mysqld: Hope that's ok; if not, decrease some variables in the equation. Nov 6 03:43:40 hostname mysqld: Thread pointer: 0x0 Nov 6 03:43:40 hostname mysqld: Attempting backtrace. You can use the following information to find out Nov 6 03:43:40 hostname mysqld: where mysqld died. If you see no messages after this, something went Nov 6 03:43:40 hostname mysqld: terribly wrong... Nov 6 03:43:40 hostname mysqld: stack_bottom = 0x0 thread_stack 0x49000 Nov 6 03:43:40 hostname mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x2e) [0x55961fefa58e] Nov 6 03:43:40 hostname mysqld: /usr/sbin/mysqld(handle_fatal_signal+0x30f) [0x55961f99754f] Nov 6 03:43:40 hostname mysqld: sigaction.c:0(__restore_rt) [0x7f2a0029a5f0] Nov 6 03:43:41 hostname mysqld: :0(__GI_raise) [0x7f29fe56b337] Nov 6 03:43:41 hostname mysqld: :0(__GI_abort) [0x7f29fe56ca28] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0x4d3560) [0x55961f6d7560] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xaebf97) [0x55961fceff97] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xb02603) [0x55961fd06603] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xb02a1d) [0x55961fd06a1d] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xb03976) [0x55961fd07976] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xaf0d50) [0x55961fcf4d50] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xb4cd08) [0x55961fd50d08] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xb578f2) [0x55961fd5b8f2] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0x984cf3) [0x55961fb88cf3] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0xa3b0e8) [0x55961fc3f0e8] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0x92a001) [0x55961fb2e001] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x64) [0x55961f999dd4] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0x5c59f0) [0x55961f7c99f0] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(_Z11plugin_initPiPPci+0x9ba) [0x55961f7cab7a] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0x4f8bf1) [0x55961f6fcbf1] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x4ae) [0x55961f702dde] Nov 6 03:43:41 hostname mysqld: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f29fe557505] Nov 6 03:43:41 hostname mysqld: /usr/sbin/mysqld(+0x4f280d) [0x55961f6f680d] Nov 6 03:43:41 hostname mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains Nov 6 03:43:41 hostname mysqld: information that should help you find out what is causing the crash. Nov 6 03:43:41 hostname mysqld: Writing a core file... Nov 6 03:43:41 hostname mysqld: Working directory at /var/lib/mysql Nov 6 03:43:41 hostname mysqld: Resource Limits: Nov 6 03:43:41 hostname mysqld: Limit Soft Limit Hard Limit Units Nov 6 03:43:41 hostname mysqld: Max cpu time unlimited unlimited seconds Nov 6 03:43:41 hostname mysqld: Max file size unlimited unlimited bytes Nov 6 03:43:41 hostname mysqld: Max data size unlimited unlimited bytes Nov 6 03:43:41 hostname mysqld: Max stack size 8388608 unlimited bytes Nov 6 03:43:41 hostname mysqld: Max core file size 0 unlimited bytes Nov 6 03:43:41 hostname mysqld: Max resident set unlimited unlimited bytes Nov 6 03:43:41 hostname mysqld: Max processes 15064 15064 processes Nov 6 03:43:41 hostname mysqld: Max open files 16364 16364 files Nov 6 03:43:41 hostname mysqld: Max locked memory 65536 65536 bytes Nov 6 03:43:41 hostname mysqld: Max address space unlimited unlimited bytes Nov 6 03:43:41 hostname mysqld: Max file locks unlimited unlimited locks Nov 6 03:43:41 hostname mysqld: Max pending signals 15064 15064 signals Nov 6 03:43:41 hostname mysqld: Max msgqueue size 819200 819200 bytes Nov 6 03:43:41 hostname mysqld: Max nice priority 0 0 Nov 6 03:43:41 hostname mysqld: Max realtime priority 0 0 Nov 6 03:43:41 hostname mysqld: Max realtime timeout unlimited unlimited us Nov 6 03:43:41 hostname mysqld: Core pattern: core Nov 6 03:43:41 hostname systemd: mariadb.service: main process exited, code=killed, status=6/ABRT Nov 6 03:43:41 hostname systemd: Failed to start MariaDB 10.3.19 database server. Nov 6 03:43:41 hostname systemd: Unit mariadb.service entered failed state. Nov 6 03:43:41 hostname systemd: mariadb.service failed. Nov 6 03:43:46 hostname systemd: mariadb.service holdoff time over, scheduling restart. Nov 6 03:43:46 hostname systemd: Stopped MariaDB 10.3.19 database server. ```

            People

              Unassigned Unassigned
              AusWeb Priyan A
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.