Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-23344

InnoDB: Fatal: Trying to access page number 57 in space 182 space name zabbix/items.

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Incomplete
    • 10.1.38, 5.5(EOL), 10.0(EOL), 10.1(EOL), 10.2(EOL), 10.3(EOL), 10.4(EOL), 10.5, 10.6
    • N/A
    • Debian release 9.9
      4 core: Version: Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz
      Storage:
      Size Used Avail Use% Mounted on
      484G 272G 190G 59% /

    Description

      Hi guys, I was trying to solve “outside tablespace bounds” MySQL error but I could not up the even the MySQL server.
      I got this message:
      Status: "InnoDB: Fatal: Trying to access page number 57 in space 182 space name zabbix/items, which is outside the tablespace bounds. Byte offset 0, len 16384 i/o ty...

      I tried to set innodb_force_recovery=1 and then up to 6 and but still got the same error.
      When I try to start mysql service I got the message:

      ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 "Connection refused")

      But there is mysqld.sock in this folder.

      Could someone show me a light?

      Attachments

        Issue Links

          Activity

            zbdba jingbo zhao added a comment -

            hi Everton Barbosa Kopec, what do on your MySQL server to cause this problem? and please paste the complete mysql error log.

            zbdba jingbo zhao added a comment - hi Everton Barbosa Kopec, what do on your MySQL server to cause this problem? and please paste the complete mysql error log.

            Hi Jingbo Zhao,
            thank you for answering me. Follow the files.
            The situation was this way, at my work there is a Zabbix and no one could not explain correctly what's happened. Simply the server crashed and they do not have a backup for database only a server snapshot. So they restored the snapshot but the MySql could not startup. After all the problems they called for my help so far so good, but I am stuck in this situation.

            evertonkopec Everton Barbosa Kopec added a comment - Hi Jingbo Zhao, thank you for answering me. Follow the files. The situation was this way, at my work there is a Zabbix and no one could not explain correctly what's happened. Simply the server crashed and they do not have a backup for database only a server snapshot. So they restored the snapshot but the MySql could not startup. After all the problems they called for my help so far so good, but I am stuck in this situation.

            According to the message, something seems to attempt accessing a page right after the end of the data file.

            Be aware that setting innodb_force_recovery to 4 or higher may corrupt the database further. The setting 6 is the worst. Do not expect anything to be consistent.

            Setting innodb_force_recovery to 4 should prevent any background access to the table. But it may also corrupt secondary indexes in tables. But you may attempt to rescue data by mysqldump or similar.

            A simple trick could be to remove or rename that data file so that it cannot be accessed.

            I cannot help with the socket problem. I would suggest to check the server error log and the process table. Maybe an old server process did not die and is preventing a new one from acquiring advisory locks on the files?

            I wonder if this could be something that we fixed in MDEV-23190?

            marko Marko Mäkelä added a comment - According to the message, something seems to attempt accessing a page right after the end of the data file. Be aware that setting innodb_force_recovery to 4 or higher may corrupt the database further. The setting 6 is the worst. Do not expect anything to be consistent. Setting innodb_force_recovery to 4 should prevent any background access to the table. But it may also corrupt secondary indexes in tables. But you may attempt to rescue data by mysqldump or similar. A simple trick could be to remove or rename that data file so that it cannot be accessed. I cannot help with the socket problem. I would suggest to check the server error log and the process table. Maybe an old server process did not die and is preventing a new one from acquiring advisory locks on the files? I wonder if this could be something that we fixed in MDEV-23190 ?

            For the socket problem – the socket is removed upon normal shutdown. Since your server keeps crashing, it doesn't get to remove the socket, that's why it is still there.

            elenst Elena Stepanova added a comment - For the socket problem – the socket is removed upon normal shutdown. Since your server keeps crashing , it doesn't get to remove the socket, that's why it is still there.

            This is not exactly a duplicate of MDEV-13542. The check for out-of-bounds accesses could be moved earlier and we could return an error to the operation that causes the problematic access, before even invoking buf_page_get_gen().

            Starting with MDEV-12545 in 10.3, we actually do have the fil_space_t::get_size() conveniently available in the higher-level code. MDEV-23855 in 10.5 cleaned up the low-level I/O code quite a bit, but the low-level function fil_report_invalid_page_access() was not removed yet.

            marko Marko Mäkelä added a comment - This is not exactly a duplicate of MDEV-13542 . The check for out-of-bounds accesses could be moved earlier and we could return an error to the operation that causes the problematic access, before even invoking buf_page_get_gen() . Starting with MDEV-12545 in 10.3, we actually do have the fil_space_t::get_size() conveniently available in the higher-level code. MDEV-23855 in 10.5 cleaned up the low-level I/O code quite a bit, but the low-level function fil_report_invalid_page_access() was not removed yet.

            We have not seen anything like this in our internal testing. Apparently the data file was truncated at some point. What happened before the time the log started? Was there some previous crash recovery or restore from a backup? What would SHOW CREATE TABLE report for the table definition? Can you get a fully resolved stack trace of the crash that would identify the current index? If the table contains a secondary index, then this might have been fixed by MDEV-24449.

            marko Marko Mäkelä added a comment - We have not seen anything like this in our internal testing. Apparently the data file was truncated at some point. What happened before the time the log started? Was there some previous crash recovery or restore from a backup? What would SHOW CREATE TABLE report for the table definition? Can you get a fully resolved stack trace of the crash that would identify the current index? If the table contains a secondary index, then this might have been fixed by MDEV-24449 .

            People

              marko Marko Mäkelä
              evertonkopec Everton Barbosa Kopec
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.