[MDEV-5621] Server random crash on ALTER TABLE Created: 2014-02-06  Updated: 2014-09-12  Resolved: 2014-09-12

Status: Closed
Project: MariaDB Server
Component/s: None
Affects Version/s: 5.5.35
Fix Version/s: 5.5.40

Type: Bug Priority: Major
Reporter: azurit Assignee: Jan Lindström (Inactive)
Resolution: Cannot Reproduce Votes: 0
Labels: alter, assertion, crash
Environment:

Debian Wheezy, 64bit, Intel Xeon E5620, 4 GB RAM


Attachments: File #sql-6493_f2aac.frm     Text File global_variables.txt     File invoices.ibd     File my.cnf     File syslog.log     File syslog2.log     Text File table.txt     Text File table2.txt    

 Description   

MariaDB server is randomly crashing when ALTERing a table (adding a column), cannot always reproduce. Table is not useble after the crash and data is lost. Command used before the last crash:
ALTER TABLE `invoices`
ADD `proforma_invoice_amount_used` FLOAT NOT NULL,
COMMENT='';

Attaching these files:
syslog.log - crash log from syslog
global_variables.txt - output from 'show global variables'
table.txt - altered table original definition
my.cnf - server config file
invoices.ibd - idb file from table which was altered
#sql-6493_f2aac.frm - temporary frm file from table which was altered (the original one was missing)



 Comments   
Comment by Elena Stepanova [ 2014-02-06 ]

Hi,

Thanks for reporting this.
Is it a production server? If not, would it maybe be possible to turn on general_log for a while, to see if there is another parallel activity related to the table at the time of the crash?

Comment by azurit [ 2014-02-07 ]

Unfortunately it's a production server BUT it was run on testing domain with testing database and there were no other access to the web at that time (so no other queries were running for 99%).

Comment by Elena Stepanova [ 2014-02-07 ]

According to syslog that you provided, there were 16 open connections to the database server at the moment of the crash, so if it wasn't web access, maybe there were some maintenance works going on (backup, statistics, monitoring, anything).

Comment by azurit [ 2014-02-07 ]

Yes, there were other cnnections and queries but to the other databases (i thought you were asking about that one database, not the whole server - you said 'related to the table').

Comment by Elena Stepanova [ 2014-02-07 ]

Yes, I meant those related to the table. I see, thank you.

Comment by azurit [ 2014-05-23 ]

I had another crash with ALTER command, attaching logs. MariaDB version 5.5.37 (Debian Wheezy, 64bit). Command:

ALTER TABLE `orders2`
ADD `used_code` varchar(50) NOT NULL,
COMMENT='';

Comment by Elena Stepanova [ 2014-06-01 ]

Looks like a different one, but both are InnoDB assertion failures. Passing to Jan to his collection of InnoDB sporadic problems, maybe he'll be able to make sense out of it.

Comment by Jan Lindström (Inactive) [ 2014-07-04 ]

Both crashed look like there is a corruption on InnoDB data dictionary that could be caused by HD failure. Can you still repeat this issue with more recent version of MariaDB ?

Comment by azurit [ 2014-07-04 ]

The last crash was with 5.5.37, we are now running 5.5.38. Crashes are very rare and random so i can't tell if it's still reproducable. Crashes occured on different databases/tables (also on new, which were created after first few crashes) and we are using 'innodb_file_per_table = 1'. Also, server is running on RAID1 array and there is no single error logged in smart of all HDDs.

Comment by Jan Lindström (Inactive) [ 2014-07-04 ]

I see. Is it possible to test with following change:

revno: 4224
committer: Jan Lindström <jplindst@mariadb.org>
branch nick: 5.5
timestamp: Fri 2014-07-04 12:25:32 +0300
message:
MDEV-5621: Server random crash on ALTER TABLE

This is not a real fix, instead try to gather additional information
at the point when dictionary content is not what we expect it to be.

Comment by azurit [ 2014-07-04 ]

Where am i supposed to download that patch? thnx

Comment by Jan Lindström (Inactive) [ 2014-07-04 ]

bzr branch lp:maria/5.5 and build from sources at the moment

Generated at Thu Feb 08 07:05:46 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.