[MDEV-2490] LP:814806 - Unclean shutdown corrupted Aria table blocking startup Created: 2011-07-22  Updated: 2015-02-02  Resolved: 2012-10-04

Status: Closed
Project: MariaDB Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Dreas van Donselaar (Inactive) Assignee: Michael Widenius
Resolution: Fixed Votes: 0
Labels: Launchpad

Attachments: XML File LPexportBug814806.xml    

 Description   

On 3 different servers now MariaDB failed to start after a power failure:
====
Jul 22 15:54:38 filt01 mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
Jul 22 15:54:38 filt01 mysqld: 110722 15:54:38 [Warning] '-log_slow_queries' is deprecated and will be removed in a future release. Please use ''slow_query_log'/'-log-slow-file'' instead.
Jul 22 15:54:38 filt01 mysqld: 110722 15:54:38 [Warning] 'master-host' is deprecated and will be removed in a future release. Please use ''CHANGE MASTER'' instead.
Jul 22 15:54:38 filt01 mysqld: 110722 15:54:38 [ERROR] mysqld got signal 11 ;
Jul 22 15:54:38 filt01 mysqld: This could be because you hit a bug. It is also possible that this binary
Jul 22 15:54:38 filt01 mysqld: or one of the libraries it was linked against is corrupt, improperly built,
Jul 22 15:54:38 filt01 mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.
Jul 22 15:54:38 filt01 mysqld: We will try our best to scrape up some info that will hopefully help diagnose
Jul 22 15:54:38 filt01 mysqld: the problem, but since we have already crashed, something is definitely wrong
Jul 22 15:54:38 filt01 mysqld: and this may fail.
Jul 22 15:54:38 filt01 mysqld:
Jul 22 15:54:38 filt01 mysqld: key_buffer_size=268435456
Jul 22 15:54:38 filt01 mysqld: read_buffer_size=131072
Jul 22 15:54:38 filt01 mysqld: max_used_connections=0
Jul 22 15:54:38 filt01 mysqld: max_threads=502
Jul 22 15:54:38 filt01 mysqld: threads_connected=0
Jul 22 15:54:38 filt01 mysqld: It is possible that mysqld could use up to
Jul 22 15:54:38 filt01 mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1359776 K
Jul 22 15:54:38 filt01 mysqld: bytes of memory
Jul 22 15:54:38 filt01 mysqld: Hope that's ok; if not, decrease some variables in the equation.
Jul 22 15:54:38 filt01 mysqld:
Jul 22 15:54:38 filt01 mysqld: Thread pointer: 0x0
Jul 22 15:54:38 filt01 mysqld: Attempting backtrace. You can use the following information to find out
Jul 22 15:54:38 filt01 mysqld: where mysqld died. If you see no messages after this, something went
Jul 22 15:54:38 filt01 mysqld: terribly wrong...
Jul 22 15:54:38 filt01 mysqld: stack_bottom = (nil) thread_stack 0x40000
Jul 22 15:54:38 filt01 mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7f2c65bc3119]
Jul 22 15:54:38 filt01 mysqld: /usr/sbin/mysqld(handle_segfault+0x439)[0x7f2c6580e709]
Jul 22 15:54:38 filt01 mysqld: /lib/libpthread.so.0[0x7f2c63993a80]
Jul 22 15:54:38 filt01 mysqld: /usr/sbin/mysqld(trnman_get_min_safe_trid+0x23)[0x7f2c659ed103]
Jul 22 15:54:38 filt01 mysqld: /usr/sbin/mysqld(maria_open+0x2595)[0x7f2c65a0a265]
Jul 22 15:54:38 filt01 mysqld: /usr/sbin/mysqld[0x7f2c65a1f388]
Jul 22 15:54:38 filt01 mysqld: /usr/sbin/mysqld[0x7f2c65a1fccc]
Jul 22 15:54:38 filt01 mysqld: /usr/sbin/mysqld(maria_apply_log+0x445)[0x7f2c65a241f5]
Jul 22 15:54:38 filt01 mysqld: /usr/sbin/mysqld(maria_recovery_from_log+0x50)[0x7f2c65a248d0]
Jul 22 15:54:38 filt01 mysqld: /usr/sbin/mysqld[0x7f2c659dd821]
Jul 22 15:54:38 filt01 mysqld: /usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x38)[0x7f2c658f59e8]
Jul 22 15:54:38 filt01 mysqld: /usr/sbin/mysqld[0x7f2c65979c4c]
Jul 22 15:54:38 filt01 mysqld: /usr/sbin/mysqld(_Z11plugin_initPiPPci+0x839)[0x7f2c6597c0b9]
Jul 22 15:54:38 filt01 mysqld: /usr/sbin/mysqld[0x7f2c65810637]
Jul 22 15:54:38 filt01 mysqld: /usr/sbin/mysqld(main+0x1f0)[0x7f2c658111e0]
Jul 22 15:54:38 filt01 mysqld: /lib/libc.so.6(__libc_start_main+0xe6)[0x7f2c636501a6]
Jul 22 15:54:38 filt01 mysqld: /usr/sbin/mysqld[0x7f2c65749ae9]
Jul 22 15:54:38 filt01 mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
Jul 22 15:54:38 filt01 mysqld: information that should help you find out what is causing the crash.
Jul 22 15:54:38 filt01 mysqld: Writing a core file
Jul 22 15:54:38 filt01 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
====

We have 1 table in Aria format. As soon as we move out the Aria table files, MariaDB starts fine again.

I've uploaded a copy of such a corrupt Aria table to the MariaDB FTP server (private folder).

 Comments   
Comment by Michael Widenius [ 2011-07-26 ]

Re: Unclean shutdown corrupted Aria table blocking startup
I am uploading the test case just now.
Note that to fix a recovery problem, I would need the following files:

  • The table files (.MAI, .MAD and .frm)
  • The aria_log_control file
  • The aria log files (aria_log.*)
  • Optionally the aria_recovery.trace file

I will come back to you if the tar file doesn't contain all of the above files.

Comment by Dreas van Donselaar (Inactive) [ 2011-07-26 ]

Re: [Bug 814806] Re: Unclean shutdown corrupted Aria table blocking startup
Hi!

On 07/26/2011 06:33 PM, Michael Widenius wrote:
> I am uploading the test case just now.
> Note that to fix a recovery problem, I would need the following files:
> - The table files (.MAI, .MAD and .frm)
> - The aria_log_control file
> - The aria log files (aria_log.*)
> - Optionally the aria_recovery.trace file
>
> I will come back to you if the tar file doesn't contain all of the above
> files.
It does not. I only included the table files. I'm afraid I have to wait
for another incident to include the other files.

Greets,

Dreas

Comment by Dreas van Donselaar (Inactive) [ 2011-08-04 ]

Re: Unclean shutdown corrupted Aria table blocking startup
We had a similar crash now after simply issueing on a server:
===
mysql -e "stop slave; set global sql_slave_skip_counter=1; start slave";
===

I'll upload a new copy including aria_log_control / aria_log.* of the new data to the FTP.

Comment by Dreas van Donselaar (Inactive) [ 2011-08-04 ]

Re: Unclean shutdown corrupted Aria table blocking startup
I just moved out the table files, started MySQL, moved the table files back in, and ran a repair:
===
server:/var/lib/mysql/spambayes$ mysqlcheck --auto-repair spambayes
spambayes.global_tokens
warning : Table is marked as crashed
warning : 1 client is using or hasn't closed the table properly
error : Page 126165: Row 161 is crashed
error : Corrupt
spambayes.hashed OK
spambayes.local_tokens OK

Repairing tables
spambayes.global_tokens
warning : Number of rows changed from 21682848 to 21682752
status : OK
server:/var/lib/mysql/spambayes$
===
Not sure if this provides additional useful information.

Comment by Michael Widenius [ 2011-08-15 ]

Re: Unclean shutdown corrupted Aria table blocking startup
I was able to reproduce and fix this.
Fix is pushed into 5.2

Comment by Dreas van Donselaar (Inactive) [ 2011-08-31 ]

Re: Unclean shutdown corrupted Aria table blocking startup
This bug is not yet fixed. We just hit it again running 5.2.8. Uploading tgz with required files to the FTP server now.

Comment by Dreas van Donselaar (Inactive) [ 2011-10-24 ]

Re: Unclean shutdown corrupted Aria table blocking startup
This issue is still occurring (with 5.2.9). Uploading the requested files to the FTP server now.

Comment by Dreas van Donselaar (Inactive) [ 2011-10-24 ]

Re: Unclean shutdown corrupted Aria table blocking startup
Switching back to in progress because the issue is still occurring.

Comment by Michael Widenius [ 2012-02-23 ]

Re: Unclean shutdown corrupted Aria table blocking startup
I think this is fixed in 5.2.11. If not ,please reopen the bug!

Comment by Rasmus Johansson (Inactive) [ 2012-02-23 ]

Launchpad bug id: 814806

Generated at Thu Feb 08 06:42:12 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.