Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-8901

InnoDB: Punch hole is incorrecty done also to log files causing assertion and database corruption

Details

    Description

      8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
      ---Type <return> to continue, or q <return> to quit---
      #2  0x0000555555fed2ff in os_file_trim (slot=0x7ffff5779000)
          at /home/jplindst/10.1/storage/xtradb/os/os0file.cc:6313
      #3  0x0000555555feb4ac in os_aio_linux_collect (array=0x7ffff57fe0b0, segment=0, seg_size=256)
          at /home/jplindst/10.1/storage/xtradb/os/os0file.cc:5395
      #4  0x0000555555feb73c in os_aio_linux_handle (global_seg=1, message1=0x7fffd7ffdde8, 
          message2=0x7fffd7ffddf0, type=0x7fffd7ffddf8, space_id=0x7fffd7ffde00)
          at /home/jplindst/10.1/storage/xtradb/os/os0file.cc:5515
      #5  0x00005555561c229a in fil_aio_wait (segment=1)
          at /home/jplindst/10.1/storage/xtradb/fil/fil0fil.cc:5992
      #6  0x00005555560b4eb0 in io_handler_thread (arg=0x55555750a5a8 <n+8>)
          at /home/jplindst/10.1/storage/xtradb/srv/srv0start.cc:542
      #7  0x00007ffff68f9e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
      #8  0x00007ffff6026cbd in clone () from /lib/x86_64-linux-gnu/libc.so.6
      #9  0x0000000000000000 in ?? ()

      Attachments

        Activity

          jplindst Jan Lindström (Inactive) created issue -

          After two bug fixes I see page compressed table to use less actual disk space:

          jplindst@perro:~/db$ ../10.1/client/mysql -u root -S mysql.sock2
          Welcome to the MariaDB monitor.  Commands end with ; or \g.
          Your MariaDB connection id is 2
          Server version: 10.1.8-MariaDB-debug Source distribution
           
          Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others.
           
          Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
           
          MariaDB [(none)]> use test;
          Database changed
          MariaDB [test]> create table a(c int not null default 0,t char(255) not null default '') engine=innodb;insert into a VALUES(912312,'asdljkashdkhqgerhwvqefasqeldkjasdljadkljasdasdd');
          Query OK, 0 rows affected (0.91 sec)
           
          Query OK, 1 row affected (0.07 sec)
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 1 row affected (0.70 sec)
          Records: 1  Duplicates: 0  Warnings: 0
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 2 rows affected (0.06 sec)
          Records: 2  Duplicates: 0  Warnings: 0
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 4 rows affected (0.05 sec)
          Records: 4  Duplicates: 0  Warnings: 0
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 8 rows affected (0.04 sec)
          Records: 8  Duplicates: 0  Warnings: 0
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 16 rows affected (0.04 sec)
          Records: 16  Duplicates: 0  Warnings: 0
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 32 rows affected (0.06 sec)
          Records: 32  Duplicates: 0  Warnings: 0
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 64 rows affected (0.10 sec)
          Records: 64  Duplicates: 0  Warnings: 0
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 128 rows affected (0.21 sec)
          Records: 128  Duplicates: 0  Warnings: 0
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 256 rows affected (0.32 sec)
          Records: 256  Duplicates: 0  Warnings: 0
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 512 rows affected (0.58 sec)
          Records: 512  Duplicates: 0  Warnings: 0
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 1024 rows affected (1.35 sec)
          Records: 1024  Duplicates: 0  Warnings: 0
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 2048 rows affected (0.34 sec)
          Records: 2048  Duplicates: 0  Warnings: 0
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 4096 rows affected (0.60 sec)
          Records: 4096  Duplicates: 0  Warnings: 0
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 8192 rows affected (1.77 sec)
          Records: 8192  Duplicates: 0  Warnings: 0
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 16384 rows affected (2.24 sec)
          Records: 16384  Duplicates: 0  Warnings: 0
           
          MariaDB [test]>  insert into a select * from a;
          Query OK, 32768 rows affected (3.90 sec)
          Records: 32768  Duplicates: 0  Warnings: 0
           
          MariaDB [test]> create table b(c int not null default 0,t char(255) not null default '') page_compressed=1 engine=innodb;
          Query OK, 0 rows affected (0.31 sec)
           
          MariaDB [test]> insert into b select * from a;
          Query OK, 65536 rows affected (13.49 sec)
          Records: 65536  Duplicates: 0  Warnings: 0
           
          MariaDB [test]> show status;
          +--------------------------------------------------------------+----------------------------------------+
          | Variable_name                                                | Value                                  |
          +--------------------------------------------------------------+----------------------------------------+
          <cut>
          | Innodb_page_compression_saved                                | 14766080                               |
          | Innodb_page_compression_trim_sect512                         | 0                                      |
          | Innodb_page_compression_trim_sect1024                        | 0                                      |
          | Innodb_page_compression_trim_sect2048                        | 0                                      |
          | Innodb_page_compression_trim_sect4096                        | 3605                                   |
          | Innodb_page_compression_trim_sect8192                        | 0                                      |
          | Innodb_page_compression_trim_sect16384                       | 0                                      |
          | Innodb_page_compression_trim_sect32768                       | 0                                      |
          | Innodb_num_index_pages_written                               | 1280                                   |
          | Innodb_num_non_index_pages_written                           | 1483                                   |
          | Innodb_num_pages_page_compressed                             | 1203                                   |
          | Innodb_num_page_compressed_trim_op                           | 1203                                   |
          | Innodb_num_page_compressed_trim_op_saved                     | 0                                      |
          | Innodb_num_pages_page_decompressed                           | 1203                                   |
          | Innodb_num_pages_page_compression_error                      | 0                                      |
          <cut>
          +--------------------------------------------------------------+----------------------------------------+
          515 rows in set (0.01 sec)
           
          MariaDB [test]> exit
          Bye
          jplindst@perro:~/db$ killall mysqld
          jplindst@perro:~/db$ ls
          aria_log.00000001  err.log  ib_logfile0  multi-master.info  my.cnf~  performance_schema
          aria_log_control   ibdata1  ib_logfile1  my.cnf             mysql    test
          [1]+  Done                    10.1/sql/mysqld --defaults-file=/home/jplindst/db/my.cnf  (wd: ~)
          (wd now: ~/db)
          jplindst@perro:~/db$ cd test
          jplindst@perro:~/db/test$ ls -la
          total 40428
          drwx------ 2 jplindst my     4096 Oct  6 09:53 .
          drwxr-xr-x 5 jplindst my     4096 Oct  6 09:55 ..
          -rw-rw---- 1 jplindst my      706 Oct  6 09:52 a.frm
          -rw-rw---- 1 jplindst my 28311552 Oct  6 09:53 a.ibd
          -rw-rw---- 1 jplindst my      730 Oct  6 09:53 b.frm
          -rw-rw---- 1 jplindst my 28311552 Oct  6 09:54 b.ibd
          jplindst@perro:~/db/test$ du
          40424   .
          jplindst@perro:~/db/test$ du a.ibd
          27652   a.ibd
          jplindst@perro:~/db/test$ du b.ibd
          12760   b.ibd

          jplindst Jan Lindström (Inactive) added a comment - After two bug fixes I see page compressed table to use less actual disk space: jplindst@perro:~/db$ ../10.1/client/mysql -u root -S mysql.sock2 Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 2 Server version: 10.1.8-MariaDB-debug Source distribution   Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others.   Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.   MariaDB [(none)]> use test; Database changed MariaDB [test]> create table a(c int not null default 0,t char(255) not null default '') engine=innodb;insert into a VALUES(912312,'asdljkashdkhqgerhwvqefasqeldkjasdljadkljasdasdd'); Query OK, 0 rows affected (0.91 sec)   Query OK, 1 row affected (0.07 sec)   MariaDB [test]> insert into a select * from a; Query OK, 1 row affected (0.70 sec) Records: 1 Duplicates: 0 Warnings: 0   MariaDB [test]> insert into a select * from a; Query OK, 2 rows affected (0.06 sec) Records: 2 Duplicates: 0 Warnings: 0   MariaDB [test]> insert into a select * from a; Query OK, 4 rows affected (0.05 sec) Records: 4 Duplicates: 0 Warnings: 0   MariaDB [test]> insert into a select * from a; Query OK, 8 rows affected (0.04 sec) Records: 8 Duplicates: 0 Warnings: 0   MariaDB [test]> insert into a select * from a; Query OK, 16 rows affected (0.04 sec) Records: 16 Duplicates: 0 Warnings: 0   MariaDB [test]> insert into a select * from a; Query OK, 32 rows affected (0.06 sec) Records: 32 Duplicates: 0 Warnings: 0   MariaDB [test]> insert into a select * from a; Query OK, 64 rows affected (0.10 sec) Records: 64 Duplicates: 0 Warnings: 0   MariaDB [test]> insert into a select * from a; Query OK, 128 rows affected (0.21 sec) Records: 128 Duplicates: 0 Warnings: 0   MariaDB [test]> insert into a select * from a; Query OK, 256 rows affected (0.32 sec) Records: 256 Duplicates: 0 Warnings: 0   MariaDB [test]> insert into a select * from a; Query OK, 512 rows affected (0.58 sec) Records: 512 Duplicates: 0 Warnings: 0   MariaDB [test]> insert into a select * from a; Query OK, 1024 rows affected (1.35 sec) Records: 1024 Duplicates: 0 Warnings: 0   MariaDB [test]> insert into a select * from a; Query OK, 2048 rows affected (0.34 sec) Records: 2048 Duplicates: 0 Warnings: 0   MariaDB [test]> insert into a select * from a; Query OK, 4096 rows affected (0.60 sec) Records: 4096 Duplicates: 0 Warnings: 0   MariaDB [test]> insert into a select * from a; Query OK, 8192 rows affected (1.77 sec) Records: 8192 Duplicates: 0 Warnings: 0   MariaDB [test]> insert into a select * from a; Query OK, 16384 rows affected (2.24 sec) Records: 16384 Duplicates: 0 Warnings: 0   MariaDB [test]> insert into a select * from a; Query OK, 32768 rows affected (3.90 sec) Records: 32768 Duplicates: 0 Warnings: 0   MariaDB [test]> create table b(c int not null default 0,t char(255) not null default '') page_compressed=1 engine=innodb; Query OK, 0 rows affected (0.31 sec)   MariaDB [test]> insert into b select * from a; Query OK, 65536 rows affected (13.49 sec) Records: 65536 Duplicates: 0 Warnings: 0   MariaDB [test]> show status; +--------------------------------------------------------------+----------------------------------------+ | Variable_name | Value | +--------------------------------------------------------------+----------------------------------------+ <cut> | Innodb_page_compression_saved | 14766080 | | Innodb_page_compression_trim_sect512 | 0 | | Innodb_page_compression_trim_sect1024 | 0 | | Innodb_page_compression_trim_sect2048 | 0 | | Innodb_page_compression_trim_sect4096 | 3605 | | Innodb_page_compression_trim_sect8192 | 0 | | Innodb_page_compression_trim_sect16384 | 0 | | Innodb_page_compression_trim_sect32768 | 0 | | Innodb_num_index_pages_written | 1280 | | Innodb_num_non_index_pages_written | 1483 | | Innodb_num_pages_page_compressed | 1203 | | Innodb_num_page_compressed_trim_op | 1203 | | Innodb_num_page_compressed_trim_op_saved | 0 | | Innodb_num_pages_page_decompressed | 1203 | | Innodb_num_pages_page_compression_error | 0 | <cut> +--------------------------------------------------------------+----------------------------------------+ 515 rows in set (0.01 sec)   MariaDB [test]> exit Bye jplindst@perro:~/db$ killall mysqld jplindst@perro:~/db$ ls aria_log.00000001 err.log ib_logfile0 multi-master.info my.cnf~ performance_schema aria_log_control ibdata1 ib_logfile1 my.cnf mysql test [1]+ Done 10.1/sql/mysqld --defaults-file=/home/jplindst/db/my.cnf (wd: ~) (wd now: ~/db) jplindst@perro:~/db$ cd test jplindst@perro:~/db/test$ ls -la total 40428 drwx------ 2 jplindst my 4096 Oct 6 09:53 . drwxr-xr-x 5 jplindst my 4096 Oct 6 09:55 .. -rw-rw---- 1 jplindst my 706 Oct 6 09:52 a.frm -rw-rw---- 1 jplindst my 28311552 Oct 6 09:53 a.ibd -rw-rw---- 1 jplindst my 730 Oct 6 09:53 b.frm -rw-rw---- 1 jplindst my 28311552 Oct 6 09:54 b.ibd jplindst@perro:~/db/test$ du 40424 . jplindst@perro:~/db/test$ du a.ibd 27652 a.ibd jplindst@perro:~/db/test$ du b.ibd 12760 b.ibd

          Confirmed that innodb-use-trim=1 causes database corruption at least for row compressed tables because actual page size (zip_size) is not known on os_file_trim.

          jplindst Jan Lindström (Inactive) added a comment - Confirmed that innodb-use-trim=1 causes database corruption at least for row compressed tables because actual page size (zip_size) is not known on os_file_trim.
          jplindst Jan Lindström (Inactive) made changes -
          Field Original Value New Value
          Status Open [ 1 ] Confirmed [ 10101 ]
          jplindst Jan Lindström (Inactive) made changes -
          Status Confirmed [ 10101 ] In Progress [ 3 ]

          Ok, fix found.

          jplindst Jan Lindström (Inactive) added a comment - Ok, fix found.

          commit 21adad000a00a62357c0da63ff9ae98850475618
          Author: Jan Lindström <jan.lindstrom@mariadb.com>
          Date: Tue Oct 6 15:40:26 2015 +0300

          MDEV-8901: InnoDB: Punch hole is incorrecty done also to log files causing assertion and database corruption

          Analysis: Problem is that punch hole does not know the actual page size
          of the page and does the page belong to an data file or to a log file.

          Fix: Pass down the file type and page size to os layer to be used
          when trim is called. Also fix unsafe null pointer access to
          actual write_size.

          jplindst Jan Lindström (Inactive) added a comment - commit 21adad000a00a62357c0da63ff9ae98850475618 Author: Jan Lindström <jan.lindstrom@mariadb.com> Date: Tue Oct 6 15:40:26 2015 +0300 MDEV-8901 : InnoDB: Punch hole is incorrecty done also to log files causing assertion and database corruption Analysis: Problem is that punch hole does not know the actual page size of the page and does the page belong to an data file or to a log file. Fix: Pass down the file type and page size to os layer to be used when trim is called. Also fix unsafe null pointer access to actual write_size.
          jplindst Jan Lindström (Inactive) made changes -
          Resolution Fixed [ 1 ]
          Status In Progress [ 3 ] Closed [ 6 ]
          serg Sergei Golubchik made changes -
          Workflow MariaDB v3 [ 71887 ] MariaDB v4 [ 149690 ]

          People

            jplindst Jan Lindström (Inactive)
            jplindst Jan Lindström (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.