Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Incomplete
    • 10.0.28
    • N/A
    • None
    • CentOS 6.9

    Description

      10.0.28

      InnoDB: Page may be a freshly allocated page
      InnoDB: Database page corruption on disk or a failed
      InnoDB: file read of page 12862421.
      InnoDB: You may have to recover from a backup.
      InnoDB: It is also possible that your operating
      InnoDB: system has corrupted its own file cache
      InnoDB: and rebooting your computer removes the
      InnoDB: error.
      InnoDB: If the corrupt page is an index page
      InnoDB: you can also try to fix the corruption
      InnoDB: by dumping, dropping, and reimporting
      InnoDB: the corrupt table. You can use CHECK
      InnoDB: TABLE to scan your table for corruption.
      InnoDB: See also http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
      InnoDB: about forcing recovery.
      InnoDB: Error: Unable to read tablespace 61037 page no 12862421 into the buffer pool after 100 attempts
      InnoDB: The most probable cause of this error may be that the table has been corrupted.
      InnoDB: You can try to fix this problem by using innodb_force_recovery.
      InnoDB: Please see reference manual for more details.
      InnoDB: Aborting...
      2018-07-14 19:25:31 7edabadfa700  InnoDB: Assertion failure in thread 139477903189760 in file buf0buf.cc line 2895
      InnoDB: We intentionally generate a memory trap.
      InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
      InnoDB: If you get repeated assertion failures or crashes, even
      InnoDB: immediately after the mysqld startup, there may be
      InnoDB: corruption in the InnoDB tablespace. Please refer to
      InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
      InnoDB: about forcing recovery.
      180714 19:25:31 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed,
      something is definitely wrong and this may fail.
       
      Server version: 10.0.28-MariaDB
      key_buffer_size=25165824
      read_buffer_size=2097152
      max_used_connections=1911
      max_threads=2002
      thread_count=101
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8264428 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x0x0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x0 thread_stack 0x48000
      (my_addr_resolve failure: fork)
      /engn001/masvc01/GMRPP/mysql/bin/mysqld(my_print_stacktrace+0x2e) [0xbda1ae]
      /engn001/masvc01/GMRPP/mysql/bin/mysqld(handle_fatal_signal+0x49a) [0x722a9a]
      /lib64/libpthread.so.0() [0x3063c0f790]
      /lib64/libc.so.6(gsignal+0x35) [0x3063832625]
      /lib64/libc.so.6(abort+0x175) [0x3063833e05]
      /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9d135f]
      /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9b498e]
      /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9b69b7]
      /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9590ea]
      /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9572f3]
      /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x958c93]
      /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x925ef8]
      /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x982228]
      /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x96d40e]
      /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9733e0]
      /lib64/libpthread.so.0() [0x3063c07a51]
      /lib64/libc.so.6(clone+0x6d) [0x30638e896d]
      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.
       
      ...
       
      180714 19:32:48 server_audit: MariaDB Audit Plugin version 1.4.0 STARTED.
      180714 19:32:48 server_audit: logging started to the file server_audit.log.
      180714 19:32:48 [Note] Server socket created on IP: '::'.
      180714 19:32:48 [ERROR] mysqld: Table './mysql/user' is marked as crashed and should be repaired
      180714 19:32:48 [ERROR] mysqld: Table 'user' is marked as crashed and should be repaired
      180714 19:32:48 [Warning] Checking table:   './mysql/user'
      180714 19:32:48 [ERROR] mysql.user: 1 client is using or hasn't closed the table properly
      180714 19:32:48 [ERROR] mysqld: Table './mysql/db' is marked as crashed and should be repaired
      180714 19:32:48 [ERROR] mysqld: Table 'db' is marked as crashed and should be repaired
      180714 19:32:48 [Warning] Checking table:   './mysql/db'
      180714 19:32:48 [ERROR] mysql.db: 1 client is using or hasn't closed the table properly
      180714 19:32:48 [ERROR] mysqld: Table './mysql/tables_priv' is marked as crashed and should be repaired
      180714 19:32:48 [ERROR] mysqld: Table 'tables_priv' is marked as crashed and should be repaired
      180714 19:32:48 [Warning] Checking table:   './mysql/tables_priv'
      180714 19:32:48 [ERROR] mysql.tables_priv: 1 client is using or hasn't closed the table properly
      180714 19:32:48 [ERROR] mysqld: Table './mysql/procs_priv' is marked as crashed and should be repaired
      

      We have confirmed that the db is restarting indefinitely after signal 6 in the error log.
      Can you tell us the cause of signal 6?

      Attachments

        1. cpu.txt
          57 kB
        2. disk.txt
          0.7 kB
        3. engine.txt
          35 kB
        4. gmrpe_table_size.txt
          5 kB
        5. gmrpp_table_size.txt
          21 kB
        6. memory.txt
          0.2 kB
        7. plugin.txt
          5 kB
        8. sts.txt
          11 kB
        9. var.txt
          15 kB

        Issue Links

          Activity

            kyoung-yeon ssauravy created issue -
            elenst Elena Stepanova made changes -
            Field Original Value New Value
            Component/s Storage Engine - InnoDB [ 10129 ]
            Component/s Optimizer [ 10200 ]
            Labels innodb need_feedback
            elenst Elena Stepanova made changes -
            Labels need_feedback
            elenst Elena Stepanova made changes -
            Description InnoDB: Page may be a freshly allocated page
            InnoDB: Database page corruption on disk or a failed
            InnoDB: file read of page 12862421.
            InnoDB: You may have to recover from a backup.
            InnoDB: It is also possible that your operating
            InnoDB: system has corrupted its own file cache
            InnoDB: and rebooting your computer removes the
            InnoDB: error.
            InnoDB: If the corrupt page is an index page
            InnoDB: you can also try to fix the corruption
            InnoDB: by dumping, dropping, and reimporting
            InnoDB: the corrupt table. You can use CHECK
            InnoDB: TABLE to scan your table for corruption.
            InnoDB: See also http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
            InnoDB: about forcing recovery.
            InnoDB: Error: Unable to read tablespace 61037 page no 12862421 into the buffer pool after 100 attempts
            InnoDB: The most probable cause of this error may be that the table has been corrupted.
            InnoDB: You can try to fix this problem by using innodb_force_recovery.
            InnoDB: Please see reference manual for more details.
            InnoDB: Aborting...
            2018-07-14 19:25:31 7edabadfa700 InnoDB: Assertion failure in thread 139477903189760 in file buf0buf.cc line 2895
            InnoDB: We intentionally generate a memory trap.
            InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
            InnoDB: If you get repeated assertion failures or crashes, even
            InnoDB: immediately after the mysqld startup, there may be
            InnoDB: corruption in the InnoDB tablespace. Please refer to
            InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
            InnoDB: about forcing recovery.
            180714 19:25:31 [ERROR] mysqld got signal 6 ;
            This could be because you hit a bug. It is also possible that this binary
            or one of the libraries it was linked against is corrupt, improperly built,
            or misconfigured. This error can also be caused by malfunctioning hardware.

            To report this bug, see https://mariadb.com/kb/en/reporting-bugs

            We will try our best to scrape up some info that will hopefully help
            diagnose the problem, but since we have already crashed,
            something is definitely wrong and this may fail.

            Server version: 10.0.28-MariaDB
            key_buffer_size=25165824
            read_buffer_size=2097152
            max_used_connections=1911
            max_threads=2002
            thread_count=101
            It is possible that mysqld could use up to
            key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8264428 K bytes of memory
            Hope that's ok; if not, decrease some variables in the equation.

            Thread pointer: 0x0x0
            Attempting backtrace. You can use the following information to find out
            where mysqld died. If you see no messages after this, something went
            terribly wrong...
            stack_bottom = 0x0 thread_stack 0x48000
            (my_addr_resolve failure: fork)
            /engn001/masvc01/GMRPP/mysql/bin/mysqld(my_print_stacktrace+0x2e) [0xbda1ae]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld(handle_fatal_signal+0x49a) [0x722a9a]
            /lib64/libpthread.so.0() [0x3063c0f790]
            /lib64/libc.so.6(gsignal+0x35) [0x3063832625]
            /lib64/libc.so.6(abort+0x175) [0x3063833e05]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9d135f]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9b498e]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9b69b7]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9590ea]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9572f3]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x958c93]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x925ef8]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x982228]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x96d40e]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9733e0]
            /lib64/libpthread.so.0() [0x3063c07a51]
            /lib64/libc.so.6(clone+0x6d) [0x30638e896d]
            The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
            information that should help you find out what is causing the crash.

            ...

            180714 19:32:48 server_audit: MariaDB Audit Plugin version 1.4.0 STARTED.
            180714 19:32:48 server_audit: logging started to the file server_audit.log.
            180714 19:32:48 [Note] Server socket created on IP: '::'.
            180714 19:32:48 [ERROR] mysqld: Table './mysql/user' is marked as crashed and should be repaired
            180714 19:32:48 [ERROR] mysqld: Table 'user' is marked as crashed and should be repaired
            180714 19:32:48 [Warning] Checking table: './mysql/user'
            180714 19:32:48 [ERROR] mysql.user: 1 client is using or hasn't closed the table properly
            180714 19:32:48 [ERROR] mysqld: Table './mysql/db' is marked as crashed and should be repaired
            180714 19:32:48 [ERROR] mysqld: Table 'db' is marked as crashed and should be repaired
            180714 19:32:48 [Warning] Checking table: './mysql/db'
            180714 19:32:48 [ERROR] mysql.db: 1 client is using or hasn't closed the table properly
            180714 19:32:48 [ERROR] mysqld: Table './mysql/tables_priv' is marked as crashed and should be repaired
            180714 19:32:48 [ERROR] mysqld: Table 'tables_priv' is marked as crashed and should be repaired
            180714 19:32:48 [Warning] Checking table: './mysql/tables_priv'
            180714 19:32:48 [ERROR] mysql.tables_priv: 1 client is using or hasn't closed the table properly
            180714 19:32:48 [ERROR] mysqld: Table './mysql/procs_priv' is marked as crashed and should be repaired


            We have confirmed that the db is restarting indefinitely after signal 6 in the error log.
            Can you tell us the cause of signal 6?
            {noformat:title=10.0.28}
            InnoDB: Page may be a freshly allocated page
            InnoDB: Database page corruption on disk or a failed
            InnoDB: file read of page 12862421.
            InnoDB: You may have to recover from a backup.
            InnoDB: It is also possible that your operating
            InnoDB: system has corrupted its own file cache
            InnoDB: and rebooting your computer removes the
            InnoDB: error.
            InnoDB: If the corrupt page is an index page
            InnoDB: you can also try to fix the corruption
            InnoDB: by dumping, dropping, and reimporting
            InnoDB: the corrupt table. You can use CHECK
            InnoDB: TABLE to scan your table for corruption.
            InnoDB: See also http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
            InnoDB: about forcing recovery.
            InnoDB: Error: Unable to read tablespace 61037 page no 12862421 into the buffer pool after 100 attempts
            InnoDB: The most probable cause of this error may be that the table has been corrupted.
            InnoDB: You can try to fix this problem by using innodb_force_recovery.
            InnoDB: Please see reference manual for more details.
            InnoDB: Aborting...
            2018-07-14 19:25:31 7edabadfa700 InnoDB: Assertion failure in thread 139477903189760 in file buf0buf.cc line 2895
            InnoDB: We intentionally generate a memory trap.
            InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
            InnoDB: If you get repeated assertion failures or crashes, even
            InnoDB: immediately after the mysqld startup, there may be
            InnoDB: corruption in the InnoDB tablespace. Please refer to
            InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
            InnoDB: about forcing recovery.
            180714 19:25:31 [ERROR] mysqld got signal 6 ;
            This could be because you hit a bug. It is also possible that this binary
            or one of the libraries it was linked against is corrupt, improperly built,
            or misconfigured. This error can also be caused by malfunctioning hardware.

            To report this bug, see https://mariadb.com/kb/en/reporting-bugs

            We will try our best to scrape up some info that will hopefully help
            diagnose the problem, but since we have already crashed,
            something is definitely wrong and this may fail.

            Server version: 10.0.28-MariaDB
            key_buffer_size=25165824
            read_buffer_size=2097152
            max_used_connections=1911
            max_threads=2002
            thread_count=101
            It is possible that mysqld could use up to
            key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8264428 K bytes of memory
            Hope that's ok; if not, decrease some variables in the equation.

            Thread pointer: 0x0x0
            Attempting backtrace. You can use the following information to find out
            where mysqld died. If you see no messages after this, something went
            terribly wrong...
            stack_bottom = 0x0 thread_stack 0x48000
            (my_addr_resolve failure: fork)
            /engn001/masvc01/GMRPP/mysql/bin/mysqld(my_print_stacktrace+0x2e) [0xbda1ae]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld(handle_fatal_signal+0x49a) [0x722a9a]
            /lib64/libpthread.so.0() [0x3063c0f790]
            /lib64/libc.so.6(gsignal+0x35) [0x3063832625]
            /lib64/libc.so.6(abort+0x175) [0x3063833e05]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9d135f]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9b498e]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9b69b7]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9590ea]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9572f3]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x958c93]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x925ef8]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x982228]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x96d40e]
            /engn001/masvc01/GMRPP/mysql/bin/mysqld() [0x9733e0]
            /lib64/libpthread.so.0() [0x3063c07a51]
            /lib64/libc.so.6(clone+0x6d) [0x30638e896d]
            The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
            information that should help you find out what is causing the crash.

            ...

            180714 19:32:48 server_audit: MariaDB Audit Plugin version 1.4.0 STARTED.
            180714 19:32:48 server_audit: logging started to the file server_audit.log.
            180714 19:32:48 [Note] Server socket created on IP: '::'.
            180714 19:32:48 [ERROR] mysqld: Table './mysql/user' is marked as crashed and should be repaired
            180714 19:32:48 [ERROR] mysqld: Table 'user' is marked as crashed and should be repaired
            180714 19:32:48 [Warning] Checking table: './mysql/user'
            180714 19:32:48 [ERROR] mysql.user: 1 client is using or hasn't closed the table properly
            180714 19:32:48 [ERROR] mysqld: Table './mysql/db' is marked as crashed and should be repaired
            180714 19:32:48 [ERROR] mysqld: Table 'db' is marked as crashed and should be repaired
            180714 19:32:48 [Warning] Checking table: './mysql/db'
            180714 19:32:48 [ERROR] mysql.db: 1 client is using or hasn't closed the table properly
            180714 19:32:48 [ERROR] mysqld: Table './mysql/tables_priv' is marked as crashed and should be repaired
            180714 19:32:48 [ERROR] mysqld: Table 'tables_priv' is marked as crashed and should be repaired
            180714 19:32:48 [Warning] Checking table: './mysql/tables_priv'
            180714 19:32:48 [ERROR] mysql.tables_priv: 1 client is using or hasn't closed the table properly
            180714 19:32:48 [ERROR] mysqld: Table './mysql/procs_priv' is marked as crashed and should be repaired
            {noformat}

            We have confirmed that the db is restarting indefinitely after signal 6 in the error log.
            Can you tell us the cause of signal 6?
            elenst Elena Stepanova made changes -
            Labels need_feedback
            elenst Elena Stepanova made changes -
            Labels need_feedback
            ossk_db ossk_db made changes -
            Attachment cpu.txt [ 45936 ]
            Attachment disk.txt [ 45937 ]
            Attachment engine.txt [ 45938 ]
            Attachment gmrpe_table_size.txt [ 45939 ]
            Attachment gmrpp_table_size.txt [ 45940 ]
            Attachment memory.txt [ 45941 ]
            Attachment plugin.txt [ 45942 ]
            Attachment sts.txt [ 45943 ]
            Attachment var.txt [ 45944 ]
            elenst Elena Stepanova made changes -
            Assignee Marko Mäkelä [ marko ]
            elenst Elena Stepanova made changes -
            Fix Version/s 10.0 [ 16000 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            danblack Daniel Black made changes -
            Fix Version/s N/A [ 14700 ]
            Fix Version/s 10.0 [ 16000 ]
            Resolution Incomplete [ 4 ]
            Status Open [ 1 ] Closed [ 6 ]
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 88418 ] MariaDB v4 [ 154669 ]

            People

              marko Marko Mäkelä
              kyoung-yeon ssauravy
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.