Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-14183

aria_pack segfaults in compress_maria_file

Details

    Description

      I am currently running MariaDB version 10.1.26-MariaDB-0+deb9u1 on Debian 9.2. aria_pack is failing with the "Segmentation fault" when the table as 39639 or more, aria_pack fails as per below output.

      aria_pack -t /var/lib/mysql/mygoweekly/term_pkonly
      Compressing /var/lib/mysql/mygoweekly/term_pkonly.MAD: (39639 records)
      Calculating statistics

      normal: 3 empty-space: 0 empty-zero: 0 empty-fill: 1
      pre-space: 0 end-space: 0 intervall-fields: 1 zero: 0
      Original trees: 7 After join: 7
      Compressing file
      Segmentation fault

      Test case:

      • create term table and load data from attachment (term.sql & term.txt)
      • create test table using term_pkonly.sql (it creates table and populates it from table term)
      • shutdown mariadb
      • aria_chk -z /var/lib/mysql/$db/term_pkonly
      • aria_pack -t /var/lib/mysql/$db/term_pkonly

      Server config:
      --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/run/mysqld/mysqld.sock --port=3306 --basedir=/usr --datadir=/var/lib/mysql --tmpdir=/tmp --lc-messages-dir=/usr/share/mysql --skip-external-locking --bind-address=127.0.0.1 --key_buffer_size=256M --max_allowed_packet=16M --thread_stack=192K --thread_cache_size=8 --tmpdir=/dev/shm/mysqltmp --myisam_recover_options=BACKUP --query_cache_limit=1M --query_cache_size=16M --log_error=/var/log/mysql/error.log --expire_logs_days=10 --max_binlog_size=100M --character-set-server=utf8mb4 --collation-server=utf8mb4_general_ci --sort_buffer_size=256M --aria-pagecache-buffer_size=512M --read_buffer_size=256M --aria_sort_buffer_size=256M

      Attachments

        Activity

          kpenza Kenneth Penza added a comment -

          I have replicated the problem on Ubuntu 16.10 running "Server version: 10.1.26-MariaDB-1~yakkety mariadb.org binary distribution". Executed the test case as below and generated the backtrace using gdb.

          Test case:
          *create term table and load data from attachment (term.sql & term.txt ). Loading performed using (mysqlimport -l -L $db *.txt -u $user -p)
          *create test table using term_pkonly.sql (it creates table and populates it from table term)
          *shutdown mariadb
          *aria_chk -z /var/lib/mysql/$db/term_pkonly
          *aria_pack -t /var/lib/mysql/$db/term_pkonly

          (gdb) bt
          #0 0x000055d22a898c01 in compress_maria_file (mrg=0x7fffcf0adfc0, huff_counts=0x55d22c225408) at /home/buildbot/buildbot/build/mariadb-10.1.26/storage/maria/maria_pack.c:2653
          #1 0x000055d22a89a074 in compress (mrg=<optimized out>, result_table=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.1.26/storage/maria/maria_pack.c:650
          #2 0x000055d22a894d35 in main (argc=<optimized out>, argv=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.1.26/storage/maria/maria_pack.c:234

          kpenza Kenneth Penza added a comment - I have replicated the problem on Ubuntu 16.10 running "Server version: 10.1.26-MariaDB-1~yakkety mariadb.org binary distribution". Executed the test case as below and generated the backtrace using gdb. Test case: *create term table and load data from attachment (term.sql & term.txt ). Loading performed using (mysqlimport -l -L $db *.txt -u $user -p) *create test table using term_pkonly.sql (it creates table and populates it from table term) *shutdown mariadb *aria_chk -z /var/lib/mysql/$db/term_pkonly *aria_pack -t /var/lib/mysql/$db/term_pkonly (gdb) bt #0 0x000055d22a898c01 in compress_maria_file (mrg=0x7fffcf0adfc0, huff_counts=0x55d22c225408) at /home/buildbot/buildbot/build/mariadb-10.1.26/storage/maria/maria_pack.c:2653 #1 0x000055d22a89a074 in compress (mrg=<optimized out>, result_table=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.1.26/storage/maria/maria_pack.c:650 #2 0x000055d22a894d35 in main (argc=<optimized out>, argv=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.1.26/storage/maria/maria_pack.c:234

          Thanks for the report and test case.

          Reproducible on all of 5.5-10.3. The test cases below are based on the provided one, just slightly reduced and converted into MTR-friendly form. They are only applicable to 10.x, because 5.5 does not have include/shutdown_mysqld.inc, but 5.5 is also affected.

          Not reproducible with MyISAM.

          Test case 1, causes crash on debug and non-debug builds

          CREATE TABLE `term_pkonly` (
            `id` int(11) NOT NULL AUTO_INCREMENT,
            `name` varchar(255) NOT NULL DEFAULT '',
            `acc` varchar(255) NOT NULL,
            `is_obsolete` int(11) NOT NULL DEFAULT '0',
            `is_root` int(11) NOT NULL DEFAULT '0',
            `is_relation` int(11) NOT NULL DEFAULT '0',
            PRIMARY KEY (`id`)
          ) ENGINE=Aria;
           
          insert into term_pkonly values
            (NULL,'foobar','qux',0,0,0),(NULL,'abcdef','qux',0,0,0);
          insert into term_pkonly select NULL, name, acc, 0, 0, 0 from term_pkonly;
          insert into term_pkonly select NULL, name, acc, 0, 0, 0 from term_pkonly;
          insert into term_pkonly select NULL, name, acc, 0, 0, 0 from term_pkonly;
          insert into term_pkonly select NULL, name, acc, 0, 0, 0 from term_pkonly;
          insert into term_pkonly select NULL, name, acc, 0, 0, 0 from term_pkonly;
          select * from term_pkonly limit 0;
           
          --let $datadir= `SELECT @@datadir`
          --source include/shutdown_mysqld.inc
          --exec cd $datadir && $MARIA_PACK -t test/term_pkonly
           
          # Cleanup
          --source include/start_mysqld.inc
          DROP TABLE term_pkonly;
          

          10.0 c3592ca7b8864a

          Program terminated with signal SIGSEGV, Segmentation fault.
          #0  0x0000000000409931 in compress_maria_file (mrg=0x7ffdbff244f0, huff_counts=0x2ef6840) at /data/src/10.0/storage/maria/maria_pack.c:2651
          2651	          DBUG_PRINT("fields", ("index: %4u code: 0x%s  bits: %2u",
          #0  0x0000000000409931 in compress_maria_file (mrg=0x7ffdbff244f0, huff_counts=0x2ef6840) at /data/src/10.0/storage/maria/maria_pack.c:2651
          #1  0x000000000040448c in compress (mrg=0x7ffdbff244f0, result_table=0x0) at /data/src/10.0/storage/maria/maria_pack.c:650
          #2  0x000000000040348a in main (argc=0, argv=0x2ec51b8) at /data/src/10.0/storage/maria/maria_pack.c:234
          

          The second test case only differs from the first one in the amount of data, it inserts less into the table. Instead of SIGSEGV, it causes an assertion failure on debug builds.

          Test case 2, causes assertion failure

          CREATE TABLE `term_pkonly` (
            `id` int(11) NOT NULL AUTO_INCREMENT,
            `name` varchar(255) NOT NULL DEFAULT '',
            `acc` varchar(255) NOT NULL,
            `is_obsolete` int(11) NOT NULL DEFAULT '0',
            `is_root` int(11) NOT NULL DEFAULT '0',
            `is_relation` int(11) NOT NULL DEFAULT '0',
            PRIMARY KEY (`id`)
          ) ENGINE=Aria;
           
          insert into term_pkonly values
            (NULL,'foobar','qux',0,0,0),(NULL,'abcdef','qux',0,0,0);
          insert into term_pkonly select NULL, name, acc, 0, 0, 0 from term_pkonly;
          insert into term_pkonly select NULL, name, acc, 0, 0, 0 from term_pkonly;
          select * from term_pkonly limit 0;
           
          --let $datadir= `SELECT @@datadir`
          --source include/shutdown_mysqld.inc
          --exec cd $datadir && $MARIA_PACK -t test/term_pkonly
           
          # Cleanup
          --source include/start_mysqld.inc
          DROP TABLE term_pkonly;
          

          10.0 c3592ca7b

          aria_pack: /data/src/10.0/storage/maria/maria_pack.c:2884: write_bits: Assertion `((bits < 8 * sizeof(value)) && ! (value >> bits)) || (bits == 8 * sizeof(value))' failed.
           
          #3  0x00007fc26b329ee2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
          #4  0x000000000040a4e1 in write_bits (value=248, bits=1) at /data/src/10.0/storage/maria/maria_pack.c:2883
          #5  0x0000000000409d51 in compress_maria_file (mrg=0x7ffff9a885d0, huff_counts=0x1367840) at /data/src/10.0/storage/maria/maria_pack.c:2716
          #6  0x000000000040448c in compress (mrg=0x7ffff9a885d0, result_table=0x0) at /data/src/10.0/storage/maria/maria_pack.c:650
          #7  0x000000000040348a in main (argc=0, argv=0x13361b8) at /data/src/10.0/storage/maria/maria_pack.c:234
          

          elenst Elena Stepanova added a comment - Thanks for the report and test case. Reproducible on all of 5.5-10.3. The test cases below are based on the provided one, just slightly reduced and converted into MTR-friendly form. They are only applicable to 10.x, because 5.5 does not have include/shutdown_mysqld.inc , but 5.5 is also affected. Not reproducible with MyISAM. Test case 1, causes crash on debug and non-debug builds CREATE TABLE `term_pkonly` ( `id` int (11) NOT NULL AUTO_INCREMENT, ` name ` varchar (255) NOT NULL DEFAULT '' , `acc` varchar (255) NOT NULL , `is_obsolete` int (11) NOT NULL DEFAULT '0' , `is_root` int (11) NOT NULL DEFAULT '0' , `is_relation` int (11) NOT NULL DEFAULT '0' , PRIMARY KEY (`id`) ) ENGINE=Aria;   insert into term_pkonly values ( NULL , 'foobar' , 'qux' ,0,0,0),( NULL , 'abcdef' , 'qux' ,0,0,0); insert into term_pkonly select NULL , name , acc, 0, 0, 0 from term_pkonly; insert into term_pkonly select NULL , name , acc, 0, 0, 0 from term_pkonly; insert into term_pkonly select NULL , name , acc, 0, 0, 0 from term_pkonly; insert into term_pkonly select NULL , name , acc, 0, 0, 0 from term_pkonly; insert into term_pkonly select NULL , name , acc, 0, 0, 0 from term_pkonly; select * from term_pkonly limit 0;   --let $datadir= `SELECT @@datadir` --source include/shutdown_mysqld.inc --exec cd $datadir && $MARIA_PACK -t test/term_pkonly   # Cleanup --source include/start_mysqld.inc DROP TABLE term_pkonly; 10.0 c3592ca7b8864a Program terminated with signal SIGSEGV, Segmentation fault. #0 0x0000000000409931 in compress_maria_file (mrg=0x7ffdbff244f0, huff_counts=0x2ef6840) at /data/src/10.0/storage/maria/maria_pack.c:2651 2651 DBUG_PRINT("fields", ("index: %4u code: 0x%s bits: %2u", #0 0x0000000000409931 in compress_maria_file (mrg=0x7ffdbff244f0, huff_counts=0x2ef6840) at /data/src/10.0/storage/maria/maria_pack.c:2651 #1 0x000000000040448c in compress (mrg=0x7ffdbff244f0, result_table=0x0) at /data/src/10.0/storage/maria/maria_pack.c:650 #2 0x000000000040348a in main (argc=0, argv=0x2ec51b8) at /data/src/10.0/storage/maria/maria_pack.c:234 The second test case only differs from the first one in the amount of data, it inserts less into the table. Instead of SIGSEGV, it causes an assertion failure on debug builds. Test case 2, causes assertion failure CREATE TABLE `term_pkonly` ( `id` int (11) NOT NULL AUTO_INCREMENT, ` name ` varchar (255) NOT NULL DEFAULT '' , `acc` varchar (255) NOT NULL , `is_obsolete` int (11) NOT NULL DEFAULT '0' , `is_root` int (11) NOT NULL DEFAULT '0' , `is_relation` int (11) NOT NULL DEFAULT '0' , PRIMARY KEY (`id`) ) ENGINE=Aria;   insert into term_pkonly values ( NULL , 'foobar' , 'qux' ,0,0,0),( NULL , 'abcdef' , 'qux' ,0,0,0); insert into term_pkonly select NULL , name , acc, 0, 0, 0 from term_pkonly; insert into term_pkonly select NULL , name , acc, 0, 0, 0 from term_pkonly; select * from term_pkonly limit 0;   --let $datadir= `SELECT @@datadir` --source include/shutdown_mysqld.inc --exec cd $datadir && $MARIA_PACK -t test/term_pkonly   # Cleanup --source include/start_mysqld.inc DROP TABLE term_pkonly; 10.0 c3592ca7b aria_pack: /data/src/10.0/storage/maria/maria_pack.c:2884: write_bits: Assertion `((bits < 8 * sizeof(value)) && ! (value >> bits)) || (bits == 8 * sizeof(value))' failed.   #3 0x00007fc26b329ee2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6 #4 0x000000000040a4e1 in write_bits (value=248, bits=1) at /data/src/10.0/storage/maria/maria_pack.c:2883 #5 0x0000000000409d51 in compress_maria_file (mrg=0x7ffff9a885d0, huff_counts=0x1367840) at /data/src/10.0/storage/maria/maria_pack.c:2716 #6 0x000000000040448c in compress (mrg=0x7ffff9a885d0, result_table=0x0) at /data/src/10.0/storage/maria/maria_pack.c:650 #7 0x000000000040348a in main (argc=0, argv=0x13361b8) at /data/src/10.0/storage/maria/maria_pack.c:234

          10.4 fails with

          10.4 81f6a3b0

          test/term_pkonly gave error 132 on open
           
          User time 0.00, System time 0.00
          Maximum resident set size 5316, Integral resident set size 0
          Non-physical pagefaults 290, Physical pagefaults 0, Swaps 0
          Blocks in 0 out 0, Messages in 0 out 0, Signals 0
          Voluntary context switches 0, Involuntary context switches 2
          mysqltest: At line 19: exec of 'cd /data/bld/10.4-backup-debug/mysql-test/var/mysqld.1/data/ && /data/bld/10.4-backup-debug/bin/aria_pack -t test/term_pkonly' failed, error: 512, status: 2, errno: 32
          Output from before failure:
          id	name	acc	is_obsolete	is_root	is_relation
          

          elenst Elena Stepanova added a comment - 10.4 fails with 10.4 81f6a3b0 test/term_pkonly gave error 132 on open   User time 0.00, System time 0.00 Maximum resident set size 5316, Integral resident set size 0 Non-physical pagefaults 290, Physical pagefaults 0, Swaps 0 Blocks in 0 out 0, Messages in 0 out 0, Signals 0 Voluntary context switches 0, Involuntary context switches 2 mysqltest: At line 19: exec of 'cd /data/bld/10.4-backup-debug/mysql-test/var/mysqld.1/data/ && /data/bld/10.4-backup-debug/bin/aria_pack -t test/term_pkonly' failed, error: 512, status: 2, errno: 32 Output from before failure: id name acc is_obsolete is_root is_relation

          Pushed the fix for testing.

          vlad.lesin Vladislav Lesin added a comment - Pushed the fix for testing.

          Pushed the fix in 10.2.

          vlad.lesin Vladislav Lesin added a comment - Pushed the fix in 10.2.

          People

            vlad.lesin Vladislav Lesin
            kpenza Kenneth Penza
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.