Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Cannot Reproduce
    • 10.3(EOL)
    • N/A
    • None
    • Hots with Debian 10

    Description

      Sorry about my English

      Today in the morning I noticed that my zabbix server was not working. Initially having a problem with the server, I tried to restart the server or even restart the service, but it did not return to operation.

      Analyzing the log, something new came to me and I can't find the problem:

      2020-02-24 13:18:12 0 [Note] InnoDB: Using Linux native AIO
      2020-02-24 13:18:12 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
      2020-02-24 13:18:12 0 [Note] InnoDB: Uses event mutexes
      2020-02-24 13:18:12 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2020-02-24 13:18:12 0 [Note] InnoDB: Number of pools: 1
      2020-02-24 13:18:12 0 [Note] InnoDB: Using generic crc32 instructions
      2020-02-24 13:18:12 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
      2020-02-24 13:18:12 0 [Note] InnoDB: Completed initialization of buffer pool
      2020-02-24 13:18:12 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
      2020-02-24 13:18:12 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=644349686286
      2020-02-24 13:18:12 0 [Note] InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 734 row operations to undo
      2020-02-24 13:18:12 0 [Note] InnoDB: Trx id counter is 139251516
      2020-02-24 13:18:12 0 [Note] InnoDB: Starting final batch to recover 2535 pages from redo log.
      2020-02-24 13:18:12 0 [ERROR] [FATAL] InnoDB: is_short 1, info_and_status_bits 2, offset 99, o_offset 5, mismatch index 18446744073709551610, end_seg_len 19 parsed len 1
      200224 13:18:12 [ERROR] mysqld got signal 6 ;
      

      Alguem poderia me ajudar?
      I have no backup of the bank.

      Attachments

        Issue Links

          Activity

            Which version is it? Please provide the rest of the error log, the part after 'signal 6' and the error log from the previous server run, which must have ended in a crash (since your server is performing crash recovery).

            elenst Elena Stepanova added a comment - Which version is it? Please provide the rest of the error log, the part after 'signal 6' and the error log from the previous server run, which must have ended in a crash (since your server is performing crash recovery).
            dionialves Dioni Alves de Oliveira added a comment - - edited

            I haven't been able to solve the problem with the bank yet, zabbix is still on par. Below full log.

            Complet log:

            2020-03-10  8:57:27 0 [Note] InnoDB: Using Linux native AIO
            2020-03-10  8:57:27 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
            2020-03-10  8:57:27 0 [Note] InnoDB: Uses event mutexes
            2020-03-10  8:57:27 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
            2020-03-10  8:57:27 0 [Note] InnoDB: Number of pools: 1
            2020-03-10  8:57:27 0 [Note] InnoDB: Using generic crc32 instructions
            2020-03-10  8:57:27 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
            2020-03-10  8:57:27 0 [Note] InnoDB: Completed initialization of buffer pool
            2020-03-10  8:57:27 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
            2020-03-10  8:57:27 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=644349686286
            2020-03-10  8:57:27 0 [Note] InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 734 row operations to undo
            2020-03-10  8:57:27 0 [Note] InnoDB: Trx id counter is 139251516
            2020-03-10  8:57:27 0 [Note] InnoDB: Starting final batch to recover 2535 pages from redo log.
            2020-03-10  8:57:27 0 [ERROR] [FATAL] InnoDB: is_short 1, info_and_status_bits 2, offset 99, o_offset 5, mismatch index 18446744073709551610, end_seg_len 19 parsed len 1
            200310  8:57:27 [ERROR] mysqld got signal 6 ;
            This could be because you hit a bug. It is also possible that this binary
            or one of the libraries it was linked against is corrupt, improperly built,
            or misconfigured. This error can also be caused by malfunctioning hardware.
             
            To report this bug, see https://mariadb.com/kb/en/reporting-bugs
             
            We will try our best to scrape up some info that will hopefully help
            diagnose the problem, but since we have already crashed,
            something is definitely wrong and this may fail.
             
            Server version: 10.3.22-MariaDB-0+deb10u1
            key_buffer_size=134217728
            read_buffer_size=131072
            max_used_connections=0
            max_threads=153
            thread_count=0
            It is possible that mysqld could use up to
            key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467423 K  bytes of memory
            Hope that's ok; if not, decrease some variables in the equation.
             
            Thread pointer: 0x0
            Attempting backtrace. You can use the following information to find out
            where mysqld died. If you see no messages after this, something went
            terribly wrong...
            stack_bottom = 0x0 thread_stack 0x49000
            /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55d25670391e]
            /usr/sbin/mysqld(handle_fatal_signal+0x54d)[0x55d2562339ed]
            /lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7f5a885a6730]
            /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x10b)[0x7f5a87b637bb]
            /lib/x86_64-linux-gnu/libc.so.6(abort+0x121)[0x7f5a87b4e535]
            /usr/sbin/mysqld(+0x4d9a59)[0x55d255f7aa59]
            /usr/sbin/mysqld(+0x4bfda2)[0x55d255f60da2]
            /usr/sbin/mysqld(+0x973d29)[0x55d256414d29]
            /usr/sbin/mysqld(+0x95c4e9)[0x55d2563fd4e9]
            /usr/sbin/mysqld(+0x95d2ef)[0x55d2563fe2ef]
            /usr/sbin/mysqld(+0x95ebfe)[0x55d2563ffbfe]
            /usr/sbin/mysqld(+0x4d2d03)[0x55d255f73d03]
            /usr/sbin/mysqld(+0x9027d4)[0x55d2563a37d4]
            /usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x68)[0x55d2562360b8]
            /usr/sbin/mysqld(+0x5d0f59)[0x55d256071f59]
            /usr/sbin/mysqld(_Z11plugin_initPiPPci+0x98a)[0x55d2560730da]
            /usr/sbin/mysqld(+0x512951)[0x55d255fb3951]
            /usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x3f9)[0x55d255fba509]
            /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb)[0x7f5a87b5009b]
            /usr/sbin/mysqld(_start+0x2a)[0x55d255fada2a]
            The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
            information that should help you find out what is causing the crash.
            Writing a core file...
            Working directory at /var/lib/mysql
            Resource Limits:
            Limit                     Soft Limit           Hard Limit           Units
            Max cpu time              unlimited            unlimited            seconds
            Max file size             unlimited            unlimited            bytes
            Max data size             unlimited            unlimited            bytes
            Max stack size            8388608              unlimited            bytes
            Max core file size        0                    unlimited            bytes
            Max resident set          unlimited            unlimited            bytes
            Max processes             64054                64054                processes
            Max open files            16364                16364                files
            Max locked memory         65536                65536                bytes
            Max address space         unlimited            unlimited            bytes
            Max file locks            unlimited            unlimited            locks
            Max pending signals       64054                64054                signals
            Max msgqueue size         819200               819200               bytes
            Max nice priority         0                    0
            Max realtime priority     0                    0
            Max realtime timeout      unlimited            unlimited            us
            Core pattern: core
            

            dionialves Dioni Alves de Oliveira added a comment - - edited I haven't been able to solve the problem with the bank yet, zabbix is still on par. Below full log. Complet log: 2020-03-10 8:57:27 0 [Note] InnoDB: Using Linux native AIO 2020-03-10 8:57:27 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2020-03-10 8:57:27 0 [Note] InnoDB: Uses event mutexes 2020-03-10 8:57:27 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 2020-03-10 8:57:27 0 [Note] InnoDB: Number of pools: 1 2020-03-10 8:57:27 0 [Note] InnoDB: Using generic crc32 instructions 2020-03-10 8:57:27 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M 2020-03-10 8:57:27 0 [Note] InnoDB: Completed initialization of buffer pool 2020-03-10 8:57:27 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2020-03-10 8:57:27 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=644349686286 2020-03-10 8:57:27 0 [Note] InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 734 row operations to undo 2020-03-10 8:57:27 0 [Note] InnoDB: Trx id counter is 139251516 2020-03-10 8:57:27 0 [Note] InnoDB: Starting final batch to recover 2535 pages from redo log. 2020-03-10 8:57:27 0 [ERROR] [FATAL] InnoDB: is_short 1, info_and_status_bits 2, offset 99, o_offset 5, mismatch index 18446744073709551610, end_seg_len 19 parsed len 1 200310 8:57:27 [ERROR] mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware.   To report this bug, see https://mariadb.com/kb/en/reporting-bugs   We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail.   Server version: 10.3.22-MariaDB-0+deb10u1 key_buffer_size=134217728 read_buffer_size=131072 max_used_connections=0 max_threads=153 thread_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467423 K bytes of memory Hope that's ok; if not, decrease some variables in the equation.   Thread pointer: 0x0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x0 thread_stack 0x49000 /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55d25670391e] /usr/sbin/mysqld(handle_fatal_signal+0x54d)[0x55d2562339ed] /lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7f5a885a6730] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x10b)[0x7f5a87b637bb] /lib/x86_64-linux-gnu/libc.so.6(abort+0x121)[0x7f5a87b4e535] /usr/sbin/mysqld(+0x4d9a59)[0x55d255f7aa59] /usr/sbin/mysqld(+0x4bfda2)[0x55d255f60da2] /usr/sbin/mysqld(+0x973d29)[0x55d256414d29] /usr/sbin/mysqld(+0x95c4e9)[0x55d2563fd4e9] /usr/sbin/mysqld(+0x95d2ef)[0x55d2563fe2ef] /usr/sbin/mysqld(+0x95ebfe)[0x55d2563ffbfe] /usr/sbin/mysqld(+0x4d2d03)[0x55d255f73d03] /usr/sbin/mysqld(+0x9027d4)[0x55d2563a37d4] /usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x68)[0x55d2562360b8] /usr/sbin/mysqld(+0x5d0f59)[0x55d256071f59] /usr/sbin/mysqld(_Z11plugin_initPiPPci+0x98a)[0x55d2560730da] /usr/sbin/mysqld(+0x512951)[0x55d255fb3951] /usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x3f9)[0x55d255fba509] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb)[0x7f5a87b5009b] /usr/sbin/mysqld(_start+0x2a)[0x55d255fada2a] The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. Writing a core file... Working directory at /var/lib/mysql Resource Limits: Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size 0 unlimited bytes Max resident set unlimited unlimited bytes Max processes 64054 64054 processes Max open files 16364 16364 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 64054 64054 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us Core pattern: core

            The server is intentionally killed in page_cur_parse_insert_rec() because the log record does not match the page contents.

            A possible cause of this could be a race condition in InnoDB crash recovery, affecting all versions from MySQL 3.23 till the fix of MDEV-24449.

            MariaDB Server 10.5 should never have been affected by this thanks to MDEV-19514, and even if it were, the symptoms should be different because the redo log format and related code was changed in MDEV-12353 and MDEV-21724.

            marko Marko Mäkelä added a comment - The server is intentionally killed in page_cur_parse_insert_rec() because the log record does not match the page contents. A possible cause of this could be a race condition in InnoDB crash recovery, affecting all versions from MySQL 3.23 till the fix of MDEV-24449 . MariaDB Server 10.5 should never have been affected by this thanks to MDEV-19514 , and even if it were, the symptoms should be different because the redo log format and related code was changed in MDEV-12353 and MDEV-21724 .

            People

              marko Marko Mäkelä
              dionialves Dioni Alves de Oliveira
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.