Re: RQG: Table corruption after Maria engine recovery - "Wrong data in bitmap"

Vardir before and after recovery
LPexportBug605798_var-bug605798.zip

Re: RQG: Table corruption after Maria engine recovery - "Wrong data in bitmap"
This issues seems to require =--maria-block-size=16K to reproduce. To reproduce with the RQG:

perl runall.pl -basedir=/home/philips/bzr/5.2 --grammar=conf/engines/engine_stress.yy --gendata=conf/engines/engine_stress.zz --queries=1M --engine=Maria --mysqld=log-output=file --reporters=ErrorLog,Backtrace,Recovery --duration=20 --threads=1 --rows=1000 --mask-level=2 --mysqld=maria-block-size=16K --mask=25551 --queries=100000000 --mysqld=loose-skip-innodb --mysqld=-loose-skip-pbxt

Re: RQG: Table corruption after Maria engine recovery - "Wrong data in bitmap"
Fix pushed into 5.1

Re: RQG: Table corruption after Maria engine recovery - "Wrong data in bitmap"
This bug is still repeatable on the latest 5.1 with the patch included.

I am attaching a new tablespace that produces the following error:

[
'test.table1000_maria_int_autoinc',
'check',
'error',
'Page 41: Wrong data in bitmap. Page_type: 0 full: 1 empty_space: 811 Bitmap-bits: 4'
],

This bug is still repeatable on the latest 5.1 with the patch included.

I am attaching a new tablespace that produces the following error:

[
'test.table1000_maria_int_autoinc',
'check',
'error',
'Page 41: Wrong data in bitmap. Page_type: 0 full: 1 empty_space: 811 Bitmap-bits: 4'
],

binary, datadir, binlog
LPexportBug605798_var-bug602169-2.zip

Re: RQG: Table corruption after Maria engine recovery - "Wrong data in bitmap"
Here is an example RQG command line

/home/buildbot/randgen/runall.pl \
--grammar=conf/engines/engine_stress.yy \
--gendata=conf/engines/engine_stress.zz \
--queries=1M \
--engine=Maria \
-mysqld=-log-output=file \
-mysqld=-log-bin \
--reporters=ErrorLog,Backtrace,Recovery,Shutdown \
--duration=240 \
--threads=5 \
--rows=10000 \
--mask-level=2 \
-mysqld=-maria-repair-threads=2 \
-mysqld=-loose-maria-group-commit=soft \
-mysqld=-loose-maria_group_commit_interval=100000000 \
-mysqld=-maria-checkpoint-interval=1 \
-mysqld=-maria-block-size=16K \
--mask=64891 \
--queries=100000000 \
--basedir=. \
--seed=1281526887

Re: RQG: Table corruption after Maria engine recovery - "Wrong data in bitmap"
To repeat:

• get above file var-bug602169-2.zip
  Exceute the following:

cd mysql-test
unzip var-bug602169-2.zip
rm -rf var
mv vardir8 var
MTR_VERSION=1 perl mysql-test-run.pl -start-dirty --mysqld=maria 1st --mysqld=-maria_block_size=1024
maria_chk ./storage/maria/maria_chk -s var/master-data/test/table1000_maria_int_autoinc

You will get the output:

maria_chk: error: Page 41: Wrong data in bitmap. Page_type: 1 full: 0 empty_space: 811 Bitmap-bits: 4

Re: RQG: Table corruption after Maria engine recovery - "Wrong data in bitmap"
To fix this, I need the exact command to repeat this new failure so that I can repeat this.
(I will probably need to generate a debug trace of mysqld to track this bug down)

Note to Philip: The problem is probably a timing issue with maria_checkpoints, so if you execute checkpoints more often it may be easier to repeat this.

Re: RQG: Table corruption after Maria engine recovery - "Wrong data in bitmap"
RQG command line that reproduced this issue with checkpoint interval = 0

/home/buildbot/randgen/runall.pl \
--queries=1M \
--engine=Maria \
-mysqld=-log-output=file \
--reporters=ErrorLog,Backtrace,Recovery,Shutdown \
--duration=240 \
--threads=20 \
--rows=10 \
--mask-level=1 \
-mysqld=-loose-maria-group-commit=hard \
-mysqld=-loose-maria_group_commit_interval=10 \
-mysqld=-maria-checkpoint-interval=0 \
-mysqld=-maria-block-size=16K \
--grammar=conf/engines/many_indexes.yy \
--gendata=conf/engines/many_indexes.zz \
--mask=53801 \
--queries=100000000 \
--basedir=. \
--seed=1283009930

Re: RQG: Table corruption after Maria engine recovery - "Wrong data in bitmap"
This command line reproduces the bug with certainty . It uses -mem and --mysqld=-maria-checkpoint-interval=0

perl runall.pl -queries=1M --engine=Maria --mysqld=log-output=file --reporters=ErrorLog,Backtrace,Recovery,Shutdown --duration=240 --threads=20 --rows=10 --mask-level=1 --mysqld=loose-maria-group-commit=hard --mysqld=loose-maria_group_commit_interval=10 --mysqld=maria-checkpoint-interval=0 --mysqld=maria-block-size=16K --grammar=conf/engines/many_indexes.yy --gendata=conf/engines/many_indexes.zz --mask=53801 --queries=100000000 --seed=1283009930 --basedir=/home/philips/bzr/maria-5.1 --duration=60 --mysqld=-skip-innodb --mem

Please use the following .ZZ file

$tables = {
rows => [10]
};

$fields = {
types => [ 'varchar(255)', 'char(255)' , 'enum', 'set', 'longtext' ],
indexes => [ undef, 'key' ],
charsets => [ 'UTF8', 'UCS2' ]
};

$data = {
strings => [ 'letter', 'english', undef ],
blobs => [ 'letter', 'english' ]
}

Re: RQG: Table corruption after Maria engine recovery - "Wrong data in bitmap"
Still repeatable with

revision-id: monty@askmonty.org-20100907165839-hbgfbzsuytbxtzn2
date: 2010-09-07 19:58:39 +0300
build-date: 2010-09-09 10:49:08 +0300
revno: 2924
branch-nick: 5.1

RQG command line:

1. 2010-09-09T06:21:29 # /home/buildbot/randgen/runall.pl \
2. 2010-09-09T06:21:29 # --queries=1M \
3. 2010-09-09T06:21:29 # --engine=Maria \
4. 2010-09-09T06:21:29 # -mysqld=-log-output=file \
5. 2010-09-09T06:21:29 # -mysqld=-maria_log_purge_type=at_flush \
6. 2010-09-09T06:21:29 # --reporters=ErrorLog,Backtrace,Recovery,Shutdown \
7. 2010-09-09T06:21:29 # --duration=120 \
8. 2010-09-09T06:21:29 # --threads=20 \
9. 2010-09-09T06:21:29 # --rows=10000 \
10. 2010-09-09T06:21:29 # --mask-level=2 \
11. 2010-09-09T06:21:29 # -mysqld=-maria-repair-threads=2 \
12. 2010-09-09T06:21:29 # -mysqld=-loose-maria-group-commit=soft \
13. 2010-09-09T06:21:29 # -mysqld=-loose-maria_group_commit_interval=0 \
14. 2010-09-09T06:21:29 # -mysqld=-maria-checkpoint-interval=1 \
15. 2010-09-09T06:21:29 # -mysqld=-maria-block-size=4K \
16. 2010-09-09T06:21:29 # --grammar=conf/engines/engine_stress.yy \
17. 2010-09-09T06:21:29 # --gendata=conf/engines/engine_stress.zz \
18. 2010-09-09T06:21:29 # --mask=6680 \
19. 2010-09-09T06:21:29 # --queries=100000000 \
20. 2010-09-09T06:21:29 # --mask=6680 \
21. 2010-09-09T06:21:29 # --basedir=. \
22. 2010-09-09T06:21:29 # --seed=time \
23. 2010-09-09T06:21:29 # --vardir=/home/buildbot/rqg-buildbot-slave/rqg-weekend/vardir-maria_recovery_weekend/current

Launchpad bug id: 605798