[MDEV-14183] aria_pack segfaults in compress_maria_file Created: 2017-10-28  Updated: 2020-08-25  Resolved: 2020-01-21

Status: Closed
Project: MariaDB Server
Component/s: Scripts & Clients, Storage Engine - Aria
Affects Version/s: 5.5, 10.0, 10.1, 10.1.26, 10.2, 10.3, 10.4
Fix Version/s: 10.2.31, 10.3.22, 10.4.12, 10.5.1

Type: Bug Priority: Critical
Reporter: Kenneth Penza Assignee: Vladislav Lesin
Resolution: Fixed Votes: 2
Labels: None
Environment:

Debian 9.2 running on AMD Phenom(tm) II X4 955 Processor


Attachments: File termtab.tar.xz    

 Description   

I am currently running MariaDB version 10.1.26-MariaDB-0+deb9u1 on Debian 9.2. aria_pack is failing with the "Segmentation fault" when the table as 39639 or more, aria_pack fails as per below output.

aria_pack -t /var/lib/mysql/mygoweekly/term_pkonly
Compressing /var/lib/mysql/mygoweekly/term_pkonly.MAD: (39639 records)
Calculating statistics

normal: 3 empty-space: 0 empty-zero: 0 empty-fill: 1
pre-space: 0 end-space: 0 intervall-fields: 1 zero: 0
Original trees: 7 After join: 7
Compressing file
Segmentation fault

Test case:

  • create term table and load data from attachment (term.sql & term.txt)
  • create test table using term_pkonly.sql (it creates table and populates it from table term)
  • shutdown mariadb
  • aria_chk -z /var/lib/mysql/$db/term_pkonly
  • aria_pack -t /var/lib/mysql/$db/term_pkonly

Server config:
--user=mysql --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/run/mysqld/mysqld.sock --port=3306 --basedir=/usr --datadir=/var/lib/mysql --tmpdir=/tmp --lc-messages-dir=/usr/share/mysql --skip-external-locking --bind-address=127.0.0.1 --key_buffer_size=256M --max_allowed_packet=16M --thread_stack=192K --thread_cache_size=8 --tmpdir=/dev/shm/mysqltmp --myisam_recover_options=BACKUP --query_cache_limit=1M --query_cache_size=16M --log_error=/var/log/mysql/error.log --expire_logs_days=10 --max_binlog_size=100M --character-set-server=utf8mb4 --collation-server=utf8mb4_general_ci --sort_buffer_size=256M --aria-pagecache-buffer_size=512M --read_buffer_size=256M --aria_sort_buffer_size=256M

 Comments   
Comment by Kenneth Penza [ 2017-10-29 ]

I have replicated the problem on Ubuntu 16.10 running "Server version: 10.1.26-MariaDB-1~yakkety mariadb.org binary distribution". Executed the test case as below and generated the backtrace using gdb.

Test case:
*create term table and load data from attachment (term.sql & term.txt ). Loading performed using (mysqlimport -l -L $db *.txt -u $user -p)
*create test table using term_pkonly.sql (it creates table and populates it from table term)
*shutdown mariadb
*aria_chk -z /var/lib/mysql/$db/term_pkonly
*aria_pack -t /var/lib/mysql/$db/term_pkonly

(gdb) bt
#0 0x000055d22a898c01 in compress_maria_file (mrg=0x7fffcf0adfc0, huff_counts=0x55d22c225408) at /home/buildbot/buildbot/build/mariadb-10.1.26/storage/maria/maria_pack.c:2653
#1 0x000055d22a89a074 in compress (mrg=<optimized out>, result_table=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.1.26/storage/maria/maria_pack.c:650
#2 0x000055d22a894d35 in main (argc=<optimized out>, argv=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.1.26/storage/maria/maria_pack.c:234

Comment by Elena Stepanova [ 2017-10-31 ]

Thanks for the report and test case.

Reproducible on all of 5.5-10.3. The test cases below are based on the provided one, just slightly reduced and converted into MTR-friendly form. They are only applicable to 10.x, because 5.5 does not have include/shutdown_mysqld.inc, but 5.5 is also affected.

Not reproducible with MyISAM.

Test case 1, causes crash on debug and non-debug builds

 
CREATE TABLE `term_pkonly` (
 
  `id` int(11) NOT NULL AUTO_INCREMENT,
 
  `name` varchar(255) NOT NULL DEFAULT '',
 
  `acc` varchar(255) NOT NULL,
 
  `is_obsolete` int(11) NOT NULL DEFAULT '0',
 
  `is_root` int(11) NOT NULL DEFAULT '0',
 
  `is_relation` int(11) NOT NULL DEFAULT '0',
 
  PRIMARY KEY (`id`)
 
) ENGINE=Aria;
 
 
 
insert into term_pkonly values
 
  (NULL,'foobar','qux',0,0,0),(NULL,'abcdef','qux',0,0,0);
 
insert into term_pkonly select NULL, name, acc, 0, 0, 0 from term_pkonly;
 
insert into term_pkonly select NULL, name, acc, 0, 0, 0 from term_pkonly;
 
insert into term_pkonly select NULL, name, acc, 0, 0, 0 from term_pkonly;
 
insert into term_pkonly select NULL, name, acc, 0, 0, 0 from term_pkonly;
 
insert into term_pkonly select NULL, name, acc, 0, 0, 0 from term_pkonly;
 
select * from term_pkonly limit 0;
 
 
 
--let $datadir= `SELECT @@datadir`
 
--source include/shutdown_mysqld.inc
 
--exec cd $datadir && $MARIA_PACK -t test/term_pkonly
 
 
 
# Cleanup
 
--source include/start_mysqld.inc
 
DROP TABLE term_pkonly;

10.0 c3592ca7b8864a

 
Program terminated with signal SIGSEGV, Segmentation fault.
 
#0  0x0000000000409931 in compress_maria_file (mrg=0x7ffdbff244f0, huff_counts=0x2ef6840) at /data/src/10.0/storage/maria/maria_pack.c:2651
 
2651	          DBUG_PRINT("fields", ("index: %4u code: 0x%s  bits: %2u",
 
#0  0x0000000000409931 in compress_maria_file (mrg=0x7ffdbff244f0, huff_counts=0x2ef6840) at /data/src/10.0/storage/maria/maria_pack.c:2651
 
#1  0x000000000040448c in compress (mrg=0x7ffdbff244f0, result_table=0x0) at /data/src/10.0/storage/maria/maria_pack.c:650
 
#2  0x000000000040348a in main (argc=0, argv=0x2ec51b8) at /data/src/10.0/storage/maria/maria_pack.c:234

The second test case only differs from the first one in the amount of data, it inserts less into the table. Instead of SIGSEGV, it causes an assertion failure on debug builds.

Test case 2, causes assertion failure

 
CREATE TABLE `term_pkonly` (
 
  `id` int(11) NOT NULL AUTO_INCREMENT,
 
  `name` varchar(255) NOT NULL DEFAULT '',
 
  `acc` varchar(255) NOT NULL,
 
  `is_obsolete` int(11) NOT NULL DEFAULT '0',
 
  `is_root` int(11) NOT NULL DEFAULT '0',
 
  `is_relation` int(11) NOT NULL DEFAULT '0',
 
  PRIMARY KEY (`id`)
 
) ENGINE=Aria;
 
 
 
insert into term_pkonly values
 
  (NULL,'foobar','qux',0,0,0),(NULL,'abcdef','qux',0,0,0);
 
insert into term_pkonly select NULL, name, acc, 0, 0, 0 from term_pkonly;
 
insert into term_pkonly select NULL, name, acc, 0, 0, 0 from term_pkonly;
 
select * from term_pkonly limit 0;
 
 
 
--let $datadir= `SELECT @@datadir`
 
--source include/shutdown_mysqld.inc
 
--exec cd $datadir && $MARIA_PACK -t test/term_pkonly
 
 
 
# Cleanup
 
--source include/start_mysqld.inc
 
DROP TABLE term_pkonly;

10.0 c3592ca7b

 
aria_pack: /data/src/10.0/storage/maria/maria_pack.c:2884: write_bits: Assertion `((bits < 8 * sizeof(value)) && ! (value >> bits)) || (bits == 8 * sizeof(value))' failed.
 
 
 
#3  0x00007fc26b329ee2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
 
#4  0x000000000040a4e1 in write_bits (value=248, bits=1) at /data/src/10.0/storage/maria/maria_pack.c:2883
 
#5  0x0000000000409d51 in compress_maria_file (mrg=0x7ffff9a885d0, huff_counts=0x1367840) at /data/src/10.0/storage/maria/maria_pack.c:2716
 
#6  0x000000000040448c in compress (mrg=0x7ffff9a885d0, result_table=0x0) at /data/src/10.0/storage/maria/maria_pack.c:650
 
#7  0x000000000040348a in main (argc=0, argv=0x13361b8) at /data/src/10.0/storage/maria/maria_pack.c:234

Comment by Elena Stepanova [ 2019-05-01 ]

10.4 fails with

10.4 81f6a3b0

 
test/term_pkonly gave error 132 on open
 
 
 
User time 0.00, System time 0.00
 
Maximum resident set size 5316, Integral resident set size 0
 
Non-physical pagefaults 290, Physical pagefaults 0, Swaps 0
 
Blocks in 0 out 0, Messages in 0 out 0, Signals 0
 
Voluntary context switches 0, Involuntary context switches 2
 
mysqltest: At line 19: exec of 'cd /data/bld/10.4-backup-debug/mysql-test/var/mysqld.1/data/ && /data/bld/10.4-backup-debug/bin/aria_pack -t test/term_pkonly' failed, error: 512, status: 2, errno: 32
 
Output from before failure:
 
id	name	acc	is_obsolete	is_root	is_relation

Comment by Vladislav Lesin [ 2020-01-20 ]

Pushed the fix for testing.

Comment by Vladislav Lesin [ 2020-01-21 ]

Pushed the fix in 10.2.

Generated at Thu Feb 08 08:11:34 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.