Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-25018

[FATAL] InnoDB: Unable to read page (of a dropped tablespace)

    XMLWordPrintable

    Details

      Description

      In a 10.6-based branch, the following crash was observed:

      2021-03-01  9:43:18 22 [ERROR] [FATAL] InnoDB: Unable to read page [page id: space=12, page number=1] into the buffer pool after 100. The most probable cause of this error may be that the table has been corrupted. See https://mariadb.com/kb/en/library/innodb-recovery-modes/
      

      Even though MDEV-19514 removed most causes of change buffer merge in the background, it left one case: change buffer merge while writing something to the change buffer. And exactly this is being triggered:

      #8  0x000055aa24e46893 in buf_page_get_gen (page_id=..., zip_size=0, rw_latch=2, guess=0x0, mode=10, mtr=0x7f4d1c8bce50, err=0x0, allow_ibuf_merge=false) at /Server/bb-10.6-MDEV-24789/storage/innobase/buf/buf0buf.cc:3341
      #9  0x000055aa249b0459 in ibuf_bitmap_get_map_page (page_id=<error reading variable: Cannot access memory at address 0xffffffffffffffc0>, zip_size=0, mtr=0x7f4d1c8bce50) at /Server/bb-10.6-MDEV-24789/storage/innobase/ibuf/ibuf0ibuf.cc:670
      #10 0x000055aa249c2118 in ibuf_reset_bitmap (b=0x7f4d61f6a560, page_id=..., zip_size=0, mtr=0x7f4d1c8bce50) at /Server/bb-10.6-MDEV-24789/storage/innobase/ibuf/ibuf0ibuf.cc:4119
      #11 0x000055aa249c3edc in ibuf_merge_or_delete_for_page (block=0x7f4d61f6a560, page_id=..., zip_size=0) at /Server/bb-10.6-MDEV-24789/storage/innobase/ibuf/ibuf0ibuf.cc:4407
      #12 0x000055aa24e4628a in buf_page_get_low (page_id=..., zip_size=0, rw_latch=2, guess=0x0, mode=16, mtr=0x7f4d1c8bda60, err=0x7f4d1c8bd890, allow_ibuf_merge=true) at /Server/bb-10.6-MDEV-24789/storage/innobase/buf/buf0buf.cc:3266
      #13 0x000055aa24e46893 in buf_page_get_gen (page_id=..., zip_size=0, rw_latch=2, guess=0x0, mode=16, mtr=0x7f4d1c8bda60, err=0x7f4d1c8bd890, allow_ibuf_merge=true) at /Server/bb-10.6-MDEV-24789/storage/innobase/buf/buf0buf.cc:3341
      #14 0x000055aa249b6d0d in ibuf_read_merge_pages (space_ids=0x7f4d1c8be320, page_nos=0x7f4d1c8be360, n_stored=1) at /Server/bb-10.6-MDEV-24789/storage/innobase/ibuf/ibuf0ibuf.cc:2272
      #15 0x000055aa249bc9b2 in ibuf_insert_low (mode=36, op=IBUF_OP_INSERT, no_counter=0, entry=0x617000298e70, entry_size=13, index=0x61600bdfdc20, page_id=..., zip_size=0, thr=0x6210003a4100) at /Server/bb-10.6-MDEV-24789/storage/innobase/ibuf/ibuf0ibuf.cc:3421
      #16 0x000055aa249bd31d in ibuf_insert (op=IBUF_OP_INSERT, entry=0x617000298e70, index=0x61600bdfdc20, page_id=..., zip_size=0, thr=0x6210003a4100) at /Server/bb-10.6-MDEV-24789/storage/innobase/ibuf/ibuf0ibuf.cc:3544
      

      In the core dump, we see that space->n_pending is STOPPING|1. This means that ibuf_merge_or_delete_for_page() is holding the only remaining reference to the tablespace, and that any further reads or writes to that tablespace is forbidden.

      The problem is that the this call stack is not passing BUF_GET_POSSIBLY_FREED to buf_page_get_low() and handling the possibly NULL return value:

      reset_bit:
      	if (space) {
      		ibuf_reset_bitmap(block, page_id, zip_size, &mtr);
      	}
      

      Note: Checking for space->is_stopping() before the call to buf_page_get_low() would not fix anything, because the flag could be set right after the check was performed.

      This bug can be worked around by

      SET GLOBAL innodb_change_buffering=none;
      

      or by avoiding any DDL operations when the change buffer could be heavily used.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              thiru Thirunarayanan Balathandayuthapani
              Reporter:
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration