[#MDEV-3175] LP:917689 - Archive table corruption crashing MariaDB signal 11

[MDEV-3175] LP:917689 - Archive table corruption crashing MariaDB signal 11 Created: 2012-01-17 Updated: 2014-06-20 Resolved: 2012-10-04
Status:	Closed
Project:	MariaDB Server
Component/s:	None
Affects Version/s:	None
Fix Version/s:	None

Type:

Bug

Priority:

Minor

Reporter:

Dreas van Donselaar (Inactive)

Assignee:

Michael Widenius

Resolution:

Fixed

Votes:

Labels:

Launchpad

Attachments:

LPexportBug917689.xml

LPexportBug917689_crash_log.txt

Description

We had a MariaDB 5.2.10 server that was continuously crashing:
{{{
Jan 17 16:17:07 server1 mysqld: 120117 16:17:07 [ERROR] mysqld got signal 11 ;
Jan 17 16:17:07 server1 mysqld: This could be because you hit a bug. It is also possible that this binary
Jan 17 16:17:07 server1 mysqld: or one of the libraries it was linked against is corrupt, improperly built,
Jan 17 16:17:07 server1 mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.
Jan 17 16:17:07 server1 mysqld: We will try our best to scrape up some info that will hopefully help diagnose
Jan 17 16:17:07 server1 mysqld: the problem, but since we have already crashed, something is definitely wrong
Jan 17 16:17:07 server1 mysqld: and this may fail.
Jan 17 16:17:07 server1 mysqld:
Jan 17 16:17:07 server1 mysqld: key_buffer_size=0
Jan 17 16:17:07 server1 mysqld: read_buffer_size=131072
Jan 17 16:17:07 server1 mysqld: max_used_connections=17
Jan 17 16:17:07 server1 mysqld: max_threads=502
Jan 17 16:17:07 server1 mysqld: threads_connected=6
Jan 17 16:17:07 server1 mysqld: It is possible that mysqld could use up to
Jan 17 16:17:07 server1 mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1098540 K
Jan 17 16:17:07 server1 mysqld: bytes of memory
Jan 17 16:17:07 server1 mysqld: Hope that's ok; if not, decrease some variables in the equation.
Jan 17 16:17:07 server1 mysqld:
Jan 17 16:17:07 server1 mysqld: Thread pointer: 0x7fd61f517b70
Jan 17 16:17:07 server1 mysqld: Attempting backtrace. You can use the following information to find out
Jan 17 16:17:07 server1 mysqld: where mysqld died. If you see no messages after this, something went
Jan 17 16:17:07 server1 mysqld: terribly wrong...
Jan 17 16:17:07 server1 mysqld: stack_bottom = 0x2020202020202020 thread_stack 0x40000
Jan 17 16:17:07 server1 mysqld: *** glibc detected *** /usr/sbin/mysqld: malloc(): memory corruption: 0x00007fd62147d000 ***
}}}
Initially we thought this would be caused by hardware issues, but it's a repair of an archive table that triggers this. I've uploaded the table causing this crash to the FTP server.

Comments

Comment by Dreas van Donselaar (Inactive) [ 2012-01-17 ]

Re: Archive table corruption crashing MariaDB signal 11

Comment by Dreas van Donselaar (Inactive) [ 2012-01-17 ]

crash_log.txt
LPexportBug917689_crash_log.txt

Comment by Elena Stepanova [ 2012-01-19 ]

Re: Archive table corruption crashing MariaDB signal 11
Hi,

You are saying that it's crashing on 5.2.10. Do you have a server where this table works okay?
From what I see, any action on this table also causes crashes on MySQL-5.1.60, MySQL-5.6.4, MariaDB-5.2.8.

When did the crashes start? On which server version was this table updated for the last time?

Comment by Dreas van Donselaar (Inactive) [ 2012-01-19 ]

Re: Archive table corruption crashing MariaDB signal 11
Hi,

> You are saying that it's crashing on 5.2.10. Do you have a server where this table works okay?
No, it seems to fail everywhere (fortunately the data inside the table is not important to us though).

> When did the crashes start? On which server version was this table updated for the last time?
The crash started after an unclean shutdown of the machine. It was last updated in 5.2.10.

Comment by Elena Stepanova [ 2012-01-26 ]

Re: Archive table corruption crashing MariaDB signal 11
So, as said before, any action on the provided table causes a server crash. Apparently, the table got corrupted due to the unclean shutdown as Dreas mentioned in his comment, but hopefully there is still a way to avoid a crash and report a table corruption instead.

Comment by Dreas van Donselaar (Inactive) [ 2012-03-12 ]

Re: Archive table corruption crashing MariaDB signal 11
This is reoccurring on one of our servers. Including backtrace output:
===
mysqlcheck: Got error: 2013: Lost connection to MySQL server during query when executing 'CHECK TABLE ... FAST'
root@mysql2 /var/lib/mysql # *** glibc detected *** /usr/sbin/mysqld: corrupted double-linked list: 0x00007f85c1bb3530 ***
======= Backtrace: =========
/lib/libc.so.6(+0x71bd6)[0x7f86d481dbd6]
/lib/libc.so.6(+0x73608)[0x7f86d481f608]

- - glibc detected *** /usr/sbin/mysqld: malloc(): memory corruption: 0x00007ff00189f440 ***
    ======= Backtrace: =========
    /lib/libc.so.6(+0x71bd6)[0x7feffc5a7bd6]
    /lib/libc.so.6(+0x74c6d)[0x7feffc5aac6d]
    /lib/libc.so.6(__libc_malloc+0x70)[0x7feffc5aca30]
    /lib/libc.so.6(__backtrace_symbols+0x11f)[0x7feffc61a28f]
    /usr/sbin/mysqld(my_print_stacktrace+0x5f)[0x7feffee1868f]
    /usr/sbin/mysqld(handle_segfault+0x405)[0x7feffea53f05]
    /lib/libpthread.so.0(+0xeff0)[0x7feffd044ff0]
    /lib/libc.so.6(memset+0xacf)[0x7feffc5b4faf]
    /usr/sbin/mysqld(_ZN12Field_string6unpackEPhPKhjb+0x86)[0x7feffea2d9e6]
    /usr/sbin/mysqld(_ZN10ha_archive10unpack_rowEP11azio_streamPh+0x187)[0x7feffec1ca07]
    /usr/sbin/mysqld(_ZN10ha_archive8optimizeEP3THDP15st_ha_check_opt+0x1c7)[0x7feffec1db87]
    /usr/sbin/mysqld(_ZN10ha_archive6repairEP3THDP15st_ha_check_opt+0x15)[0x7feffec1be45]
    /usr/sbin/mysqld(_ZN10ha_archive16check_and_repairEP3THD+0x38)[0x7feffec1bfa8]
    /usr/sbin/mysqld(+0x3aa729)[0x7feffea9f729]
    /usr/sbin/mysqld(_Z10open_tableP3THDP10TABLE_LISTP11st_mem_rootPbj+0x808)[0x7feffeaa2198]
    /usr/sbin/mysqld(_Z11open_tablesP3THDPP10TABLE_LISTPjj+0x5f7)[0x7feffeaa2b27]
    /usr/sbin/mysqld(_Z28open_and_lock_tables_derivedP3THDP10TABLE_LISTb+0x67)[0x7feffeaa4b57]
    /usr/sbin/mysqld(_Z12mysql_insertP3THDP10TABLE_LISTR4ListI4ItemERS3_IS5_ES6_S6_15enum_duplicatesb+0xd8)[0x7feffeadb518]
    /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0xaf8)[0x7feffea61eb8]
    /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjPPKc+0x2c1)[0x7feffea66eb1]
    /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xd0b)[0x7feffea6801b]
    /usr/sbin/mysqld(_Z10do_commandP3THD+0xf4)[0x7feffea68b34]
    /usr/sbin/mysqld(handle_one_connection+0x14b)[0x7feffea5aebb]
    /lib/libpthread.so.0(+0x68ca)[0x7feffd03c8ca]
    /lib/libc.so.6(clone+0x6d)[0x7feffc60586d]
    ======= Memory map: ========
    7feec4000000-7feec52b6000 rw-p 00000000 00:00 0
    7feec52b6000-7feec8000000 ---p 00000000 00:00 0
    7feecb75e000-7feecb75f000 ---p 00000000 00:00 0
    7feecb75f000-7feecb79f000 rw-p 00000000 00:00 0
    7feecb79f000-7feecb7a0000 ---p 00000000 00:00 0
    7feecb7a0000-7feecb7e0000 rw-p 00000000 00:00 0
    7feecb7e0000-7feecb7e1000 ---p 00000000 00:00 0
    7feecb7e1000-7feecb821000 rw-p 00000000 00:00 0
    7feecb821000-7feecb822000 ---p 00000000 00:00 0
    7feecb822000-7feecb862000 rw-p 00000000 00:00 0
    7feecb862000-7feecb863000 ---p 00000000 00:00 0
    7feecb863000-7feecb8a3000 rw-p 00000000 00:00 0
    7feecb8a3000-7feecb8a4000 ---p 00000000 00:00 0
    7feecb8a4000-7feecb8e4000 rw-p 00000000 00:00 0
    7feecb8e4000-7feecb8e5000 ---p 00000000 00:00 0
    7feecb8e5000-7feecb925000 rw-p 00000000 00:00 0
    7feecb925000-7feecb926000 ---p 00000000 00:00 0
    7feecb926000-7feecb966000 rw-p 00000000 00:00 0
    7feecb966000-7feecb967000 ---p 00000000 00:00 0
    7feecb967000-7feecb9a7000 rw-p 00000000 00:00 0
    7feecb9a7000-7feecb9a8000 ---p 00000000 00:00 0
    7feecb9a8000-7feecb9e8000 rw-p 00000000 00:00 0
    7feecb9e8000-7feecb9e9000 ---p 00000000 00:00 0
    7feecb9e9000-7feecba29000 rw-p 00000000 00:00 0
    7feecba29000-7feecba2a000 ---p 00000000 00:00 0
    7feecba2a000-7feecba6a000 rw-p 00000000 00:00 0
    7feecba6a000-7feecba6b000 ---p 00000000 00:00 0
    7feecba6b000-7feecbaab000 rw-p 00000000 00:00 0
    7feecbaab000-7feecbaac000 ---p 00000000 00:00 0
    7feecbaac000-7feecbaec000 rw-p 00000000 00:00 0
    7feecbaec000-7feecbaed000 ---p 00000000 00:00 0
    7feecbaed000-7feecbb2d000 rw-p 00000000 00:00 0
    7feecbb2d000-7feecbb2e000 ---p 00000000 00:00 0
    7feecbb2e000-7feecbb6e000 rw-p 00000000 00:00 0
    7feecbb6e000-7feecbb6f000 ---p 00000000 00:00 0
    7feecbb6f000-7feecbbaf000 rw-p 00000000 00:00 0
    7feecbbaf000-7feecbbb0000 ---p 00000000 00:00 0
    7feecbbb0000-7feecbbf0000 rw-p 00000000 00:00 0
    7feecbbf0000-7feecbbf1000 ---p 00000000 00:00 0
    7feecbbf1000-7feecbc31000 rw-p 00000000 00:00 0
    7feecbc31000-7feecbc32000 ---p 00000000 00:00 0
    7feecbc32000-7feecbc72000 rw-p 00000000 00:00 0
    7feecbc72000-7feecbc73000 ---p 00000000 00:00 0
    7feecbc73000-7feecbcb3000 rw-p 00000000 00:00 0
    7feecbcb3000-7feecbcb4000 ---p 00000000 00:00 0
    7feecbcb4000-7feecbcf4000 rw-p 00000000 00:00 0
    7feecbcf4000-7feecbcf5000 ---p 00000000 00:00 0
    7feecbcf5000-7feecbd35000 rw-p 00000000 00:00 0
    7feecbd35000-7feecbd36000 ---p 00000000 00:00 0
    7feecbd36000-7feecbd76000 rw-p 00000000 00:00 0
    7feecbd76000-7feecbd77000 ---p 00000000 00:00 0
    7feecbd77000-7feecbdb7000 rw-p 00000000 00:00 0
    7feecbdb7000-7feecbdb8000 ---p 00000000 00:00 0
    7feecbdb8000-7feecbdf8000 rw-p 00000000 00:00 0
    7feecbdf8000-7feecbdf9000 ---p 00000000 00:00 0
    7feecbdf9000-7feecbe39000 rw-p 00000000 00:00 0
    7feecbe39000-7feecbe3a000 ---p 00000000 00:00 0
    7feecbe3a000-7feecbe7a000 rw-p 00000000 00:00 0
    7feecbe7a000-7feecbe7b000 ---p 00000000 00:00 0
    7feecbe7b000-7feecbebb000 rw-p 00000000 00:00 0
    7feecbebb000-7feecbebc000 ---p 00000000 00:00 0
    7feecbebc000-7feecbefc000 rw-p 00000000 00:00 0
    7feecbefc000-7feecbefd000 ---p 00000000 00:00 0
    7feecbefd000-7feecbf3d000 rw-p 00000000 00:00 0
    7feecbf3d000-7feecbf3e000 ---p 00000000 00:00 0
    7feecbf3e000-7feecbf7e000 rw-p 00000000 00:00 0
    7feecbf7e000-7feecbf7f000 ---p 00000000 00:00 0
    7feecbf7f000-7feecbfbf000 rw-p 00000000 00:00 0
    7feecbfbf000-7feecbfc0000 ---p 00000000 00:00 0
    7feecbfc0000-7feecc7f3000 rw-p 00000000 00:00 0
    7feecc7f3000-7feed0000000 ---p 00000000 00:00 0
    7feed0004000-7feed0005000 ---p 00000000 00:00 0
    7feed0005000-7feed0045000 rw-p 00000000 00:00 0
    7feed0045000-7feed0046000 ---p 00000000 00:00 0
    7feed0046000-7feed0086000 rw-p 00000000 00:00 0
    7feed0086000-7feed0087000 ---p 00000000 00:00 0
    7feed0087000-7feed0887000 rw-p 00000000 00:00 0
    7feed0887000-7feed0888000 ---p 00000000 00:00 0
    7feed0888000-7feed1088000 rw-p 00000000 00:00 0
    7feed1088000-7feed1089000 ---p 00000000 00:00 0
    7feed1089000-7feed1889000 rw-p 00000000 00:00 0
    7feed1889000-7feed188a000 ---p 00000000 00:00 0
    7feed188a000-7feed208a000 rw-p 00000000 00:00 0
    7feed208a000-7feed208b000 ---p 00000000 00:00 0
    7feed208b000-7feed288b000 rw-p 00000000 00:00 0
    7feed288b000-7feed288c000 ---p 00000000 00:00 0
    7feed288c000-7feed3292000 rw-p 00000000 00:00 0
    7feed32a7000-7feed32a8000 ---p 00000000 00:00 0
    7feed32a8000-7feed32e8000 rw-p 00000000 00:00 0
    7feed32e8000-7feed32e9000 ---p 00000000 00:00 0
    7feed32e9000-7feed3329000 rw-p 00000000 00:00 0
    7feed3329000-7feed332a000 ---p 00000000 00:00 0
    7feed332a000-7feed336a000 rw-p 00000000 00:00 0
    7feed336a000-7feed336b000 ---p 00000000 00:00 0
    7feed336b000-7feed33ab000 rw-p 00000000 00:00 0
    7feed33ab000-7feed33ac000 ---p 00000000 00:00 0
    7feed33ac000-7feed33ec000 rw-p 00000000 00:00 0
    7feed33ec000-7feed33ed000 ---p 00000000 00:00 0
    7feed33ed000-7feed342d000 rw-p 00000000 00:00 0
    7feed342d000-7feed342e000 ---p 00000000 00:00 0
    7feed342e000-7feed346e000 rw-p 00000000 00:00 0
    7feed346e000-7feed346f000 ---p 00000000 00:00 0
    7feed346f000-7feed34af000 rw-p 00000000 00:00 0
    7feed34af000-7feed34b0000 ---p 00000000 00:00 0
    7feed34b0000-7feedc492000 rw-p 00000000 00:00 0
    7feedc492000-7feee0000000 ---p 00000000 00:00 0
    7feee001a000-7feee001b000 ---p 00000000 00:00 0
    7feee001b000-7feee005b000 rw-p 00000000 00:00 0
    7feee005b000-7feee005c000 ---p 00000000 00:00 0
    7feee005c000-7feee009c000 rw-p 00000000 00:00 0
    7feee009c000-7feee009d000 ---p 00000000 00:00 0
    7feee009d000-7feee00dd000 rw-p 00000000 00:00 0
    7feee00dd000-7feee00de000 ---p 00000000 00:00 0
    7feee00de000-7feee011e000 rw-p 00000000 00:00 0
    7feee011e000-7feee011f000 ---p 00000000 00:00 0
    7feee011f000-7feee015f000 rw-p 00000000 00:00 0
    7feee015f000-7feee0160000 ---p 00000000 00:00 0
    7feee0160000-7feee01a0000 rw-p 00000000 00:00 0
    7feee01a0000-7feee01a1000 ---p 00000000 00:00 0
    7feee01a1000-7feee01e1000 rw-p 00000000 00:00 0
    7feee01e1000-7feee01e2000 ---p 00000000 00:00 0
    7feee01e2000-7feee0222000 rw-p 00000000 00:00 0
    7feee0222000-7feee0223000 ---p 00000000 00:00 0
    7feee0223000-7feee0263000 rw-p 00000000 00:00 0
    7feee0263000-7feee0264000 ---p 00000000 00:00 0
    7feee0264000-7feee02a4000 rw-p 00000000 00:00 0
    7feee02a4000-7feee02a5000 ---p 00000000 00:00 0
    7feee02a5000-7feee02e5000 rw-p 00000000 00:00 0
    7feee02e5000-7feee02e6000 ---p 00000000 00:00 0
    7feee02e6000-7feee0326000 rw-p 00000000 00:00 0
    7feee0326000-7feee0327000 ---p 00000000 00:00 0
    7feee0327000-7feee0367000 rw-p 00000000 00:00 0
    7feee0367000-7feee0368000 ---p 00000000 00:00 0
    7feee0368000-7feee03a8000 rw-p 00000000 00:00 0
    7feee03a8000-7feee03a9000 ---p 00000000 00:00 0
    7feee03a9000-7feee03e9000 rw-p 00000000 00:00 0
    7feee03e9000-7feee03ea000 ---p 00000000 00:00 0
    7feee03ea000-7feee042a000 rw-p 00000000 00:00 0
    7feee042a000-7feee042b000 ---p 00000000 00:00 0
    7feee042b000-7feee046b000 rw-p 00000000 00:00 0
    7feee046b000-7feee046c000 ---p 00000000 00:00 0
    7feee046c000-7feee04ac000 rw-p 00000000 00:00 0
    ===

Comment by Rasmus Johansson (Inactive) [ 2012-03-13 ]

Launchpad bug id: 917689

Generated at Thu Feb 08 06:46:49 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.

[MDEV-3175] LP:917689 - Archive table corruption crashing MariaDB signal 11 Created: 2012-01-17 Updated: 2014-06-20 Resolved: 2012-10-04