Re: Failing assertion: (ulint) index == offsets[3] at rem0rec.ic line 973
Also fails on Oracle MySQL 5.5.19 and 5.5.20, but could not reproduce on 5.6.4.

Re: Failing assertion: (ulint) index == offsets[3] at rem0rec.ic line 973
Elena,

Did you use any special build options / scripts / etc?
I've been testing and have not been able to repeat with Percona-Server or MySQL (5.5.19 and 5.5.20).
The only difference between my test and yours is that I am starting the server(s) with kewpie vs. runall.pl.
I'll investigate this further / try to duplicate the issue with just runall.pl, but thought this might be of interest / was curious about any additional information you could offer.

Thanks,
Patrick

Re: Failing assertion: (ulint) index == offsets[3] at rem0rec.ic line 973
Patrick,

For MySQL server, I used the release packages, e.g. mysql-5.5.20-linux2.6-i686.tar.gz, only mysqld-debug binary instead of mysqld.
Percona server was built with DCMAKE_BUILD_TYPE=Debug without any other options.

I used the standard RQG with options, grammar and data file as described above. Please note innodb-flush-log-at-trx-commit=0 – on this particular grammar and data I couldn't reproduce the failure with the default value 1 (although i've seen it with the default value on other, more complicated tests). Please make sure this option is picked up by your server startup utility.

The tests were run on a 1-core i686 machine and on a 8-core x86_64 box, both with openSUSE 11.3; in each case, the assertion would fail soon after the test start.

Re: Failing assertion: (ulint) index == offsets[3] at rem0rec.ic line 973
i hit this constantly when running own tests on 5.5-debug. have filed an oracle bug for it now.

Re: Failing assertion: (ulint) index == offsets[3] at rem0rec.ic line 973
FYI, Oracle Bug 13614558 - DEBUG BUILD, INNODB_LOCK_MONITOR: FAILING ASSERTION: (ULINT) INDEX == OFFSETS[3]

Re: Failing assertion: (ulint) index == offsets[3] at rem0rec.ic line 973
Ok, thanks for this additional input. I hope to take a deeper look soon and will try OpenSuse. I've been running on Ubuntu 64 bit for my tests so far.
I am setting the innodb-flush-log-at-trx-commit value, but not seeing this crash
Thanks again for the update.

Re: Failing assertion: (ulint) index == offsets[3] at rem0rec.ic line 973
Trying running SHOW ENGINE INNODB STATUS in a loop over and over. It has same problem as lock monitor, except lock monitor only runs every 15 seconds. I repeat it reliably on windows, so OS is no problem.. If speed is an issue, put everything on ramdisk

Re: Failing assertion: (ulint) index == offsets[3] at rem0rec.ic line 973
Also filed as JIRA issue MDEV-109

Re: Failing assertion: (ulint) index == offsets[3] at rem0rec.ic line 973
Hi,

This bug has been hanging in 'Incomplete' for quite a while (in percona-server project). Do you expect any more information from my side? The problem still exists in current percona-server, revno 258. If you have a problem reproducing it, please try the grammar with the addition suggested by sbester above:

1. assert.yy

query:
UPDATE _table SET _field_no_pk = _digit ;

thread1:
SHOW ENGINE INNODB STATUS ;

1. end of assert.yy

Also, please try to increase the number of threads and the duration of the test, e.g.:

perl runall.pl \
--queries=10M \
--duration=600 \
-mysqld=-loose-innodb-flush-log-at-trx-commit=0 \
--threads=4 \
--grammar=assert.yy \
--gendata=assert.zz \
--engine=InnoDB \
--basedir=<path to the debug server> \
--vardir=<path to vardir>

It fails for me every time I run it.

As said before, I build percona-server as
cmake . -DCMAKE_BUILD_TYPE=Debug && make
No other build options.

Re: Failing assertion: (ulint) index == offsets[3] at rem0rec.ic line 973
Hi Elena,

No, I don't think this bug is waiting on you now, I'm changing the status to Confirmed, it looks like the available information should be sufficient for whoever will be working on this bug. Thank you for your help with it.

Re: Failing assertion: (ulint) index == offsets[3] at rem0rec.ic line 973
i haven't been able to repeat this on official mysql 5.5.23 or higher. 5.5.21 and 5.5.22 crashed in seconds though.

Re: Failing assertion: (ulint) index == offsets[3] at rem0rec.ic line 973
Shane,

Thanks for the update. We should re-verify this against recent PS versions then.

Re: Failing assertion: (ulint) index == offsets[3] at rem0rec.ic line 973
Crash confirmed with latest bzr trunk (290.2.6) with runall.pl and details provided above.

Creating Directories
Installing Master Database
=======================================================

TEST RESULT TIME (ms)
-------------------------------------------------------

Servers started, exiting

1. 2012-09-06T01:38:09 Starting: gentest.pl --gendata=/home/raghu/assert.zz --engine=InnoDB --threads=4 --queries=10M --duration=600 --dsn=dbi:mysql:host=127.0.0.1:port=19300:user=root:database=test --grammar=/home/raghu/assert.yy
2. 2012-09-06T01:38:09 -------------------------------
3. 2012-09-06T01:38:09 Configuration
4. 2012-09-06T01:38:09 dsn => ['dbi:mysql:host=127.0.0.1:port=19300:user=root:database=test']
5. 2012-09-06T01:38:09 duration => 600
6. 2012-09-06T01:38:09 engine => InnoDB
7. 2012-09-06T01:38:09 gendata => /home/raghu/assert.zz
8. 2012-09-06T01:38:09 generator => FromGrammar
9. 2012-09-06T01:38:09 grammar => /home/raghu/assert.yy
10. 2012-09-06T01:38:09 queries => 10M
11. 2012-09-06T01:38:09 reporters => ['']
12. 2012-09-06T01:38:09 seed => 1
13. 2012-09-06T01:38:09 threads => 4
14. 2012-09-06T01:38:09 transformers => ['']
15. 2012-09-06T01:38:09 validators => ['']
16. 2012-09-06T01:38:09 # Creating MySQL table: test.table10_innodb_key_pk_parts_2_int_autoinc; engine: innodb; rows: 10 .
17. 2012-09-06T01:38:09 # Creating MySQL table: test.table100_innodb_key_pk_parts_2_int_autoinc; engine: innodb; rows: 100 .
18. 2012-09-06T01:38:09 Loading Generator GenTest::Generator::FromGrammar.
19. 2012-09-06T01:38:10 Caching schema metadata for dbi:mysql:host=127.0.0.1:port=19300:user=root:database=test
20. 2012-09-06T01:38:10 Reporters: ErrorLog, Backtrace
21. 2012-09-06T01:38:10 Validators: ErrorMessageCorruption
22. 2012-09-06T01:38:10 Starting 4 processes, 10M queries each, duration 600 seconds.
23. 2012-09-06T01:38:10 GenTest::ErrorFilter(2627) started
24. 2012-09-06T01:38:10 Loading Validator GenTest::Validator::ErrorMessageCorruption.
25. 2012-09-06T01:38:10 Loading Validator GenTest::Validator::ErrorMessageCorruption.
26. 2012-09-06T01:38:11 Loading Validator GenTest::Validator::ErrorMessageCorruption.
27. 2012-09-06T01:38:11 Loading Validator GenTest::Validator::ErrorMessageCorruption.
28. 2012-09-06T01:38:11 Started periodic reporting process...
29. 2012-09-06T01:42:44 Query: SHOW ENGINE INNODB STATUS failed: 2013 Lost connection to MySQL server during query
30. 2012-09-06T01:42:44 Server crash reported at dsn dbi:mysql:host=127.0.0.1:port=19300:user=root:database=test
31. 2012-09-06T01:42:44 Query: UPDATE `table100_innodb_key_pk_parts_2_int_autoinc` SET `col_int_key` = 4 failed: 2013 Lost connection to MySQL server during query
32. 2012-09-06T01:42:44 Server crash reported at dsn dbi:mysql:host=127.0.0.1:port=19300:user=root:database=test
33. 2012-09-06T01:42:44 Query: UPDATE `table100_innodb_key_pk_parts_2_int_autoinc` SET `col_int_key` = 1 failed: 2013 Lost connection to MySQL server during query
34. 2012-09-06T01:42:44 Server crash reported at dsn dbi:mysql:host=127.0.0.1:port=19300:user=root:database=test
35. 2012-09-06T01:42:44 Query: UPDATE `table100_innodb_key_pk_parts_2_int_autoinc` SET `col_int_key` = 7 failed: 2013 Lost connection to MySQL server during query
36. 2012-09-06T01:42:44 Server crash reported at dsn dbi:mysql:host=127.0.0.1:port=19300:user=root:database=test
37. 2012-09-06T01:42:44 Killing child process with pid 2631...
38. 2012-09-06T01:42:44 Killing child process with pid 2635...
39. 2012-09-06T01:42:44 Killing child process with pid 2633...
40. 2012-09-06T01:42:45 Killing periodic reporting process with pid 2628...
41. 2012-09-06T01:42:45 Kill GenTest::ErrorFilter(2627)
42. 2012-09-06T01:42:45 Server crash reported, initiating post-crash analysis...
43. 2012-09-06T01:42:45 The last 100 lines from /home/raghu/mysql/master-data//../log/master.err :
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table100_innodb_key_pk_parts_2_int_autoinc#P#p1.
    InnoDB: DEBUG: update_statistics for test/table10_innodb_key_pk_parts_2_int_autoinc#P#p0.
    InnoDB: DEBUG: update_statistics for test/table10_innodb_key_pk_parts_2_int_autoinc#P#p1.
    120905 23:12:37 InnoDB: Assertion failure in thread 140543818106624 in file rem0rec.ic line 973
    InnoDB: Failing assertion: (ulint) index == offsets[3]
    InnoDB: We intentionally generate a memory trap.
    InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
    InnoDB: If you get repeated assertion failures or crashes, even
    InnoDB: immediately after the mysqld startup, there may be
    InnoDB: corruption in the InnoDB tablespace. Please refer to
    InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
    InnoDB: about forcing recovery.
    20:12:37 UTC - mysqld got signal 6 ;
    This could be because you hit a bug. It is also possible that this binary
    or one of the libraries it was linked against is corrupt, improperly built,
    or misconfigured. This error can also be caused by malfunctioning hardware.
    We will try our best to scrape up some info that will hopefully help
    diagnose the problem, but since we have already crashed,
    something is definitely wrong and this may fail.
    Please help us make Percona Server better by reporting any
    bugs at http://bugs.percona.com/

key_buffer_size=1048576
read_buffer_size=131072
max_used_connections=5
max_threads=151
thread_count=5
connection_count=5
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 60862 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x2ced230
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7fd2e85a7e68 thread_stack 0x40000
/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(my_print_stacktrace+0x2e)[0x7d7195]
/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(handle_fatal_signal+0x2f1)[0x6ad0a1]
/lib/libpthread.so.0(+0xf170)[0x7fd2ec09f170]
/lib/libc.so.6(gsignal+0x35)[0x7fd2ea655fa5]
/lib/libc.so.6(abort+0x148)[0x7fd2ea657428]
/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld[0x986ff5]
/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld[0x98a017]
/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld[0x990a5a]
/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld[0x88f112]
/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld[0x839db7]
/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(_Z14ha_show_statusP3THDP10handlerton12ha_stat_type+0x3fe)[0x6b5876]
/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(_Z21mysql_execute_commandP3THD+0xe9b)[0x595bfa]
/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x223)[0x59c7ea]
/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x7dd)[0x59d28a]
/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(_Z10do_commandP3THD+0x282)[0x59e75f]
/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(_Z24do_handle_one_connectionP3THD+0x10c)[0x648061]
/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld(handle_one_connection+0x52)[0x648157]
/lib/libpthread.so.0(+0x7e0f)[0x7fd2ec097e0f]
/lib/libc.so.6(clone+0x6d)[0x7fd2ea70504d]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7fd2d000e780): SHOW ENGINE INNODB STATUS
Connection ID (thread ID): 7
Status: NOT_KILLED

You may download the Percona Server operations manual by visiting
http://www.percona.com/software/percona-server/. You may find information
in the manual which will help you identify the cause of the crash.
Writing a core file

1. 2012-09-06T01:42:45 datadir is /home/raghu/mysql/master-data/
2. 2012-09-06T01:42:45 binary is /media/Vone/percona-server/raghu/Percona-Server/mysql-test/../sql/mysqld
3. 2012-09-06T01:42:45 bindir is /media/Vone/percona-server/raghu/Percona-Server/mysql-test/../sql
4. 2012-09-06T01:42:45 core is /home/raghu/mysql/master-data/core

warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?

warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fffe276f000
15 /media/Vone/percona-server/origin/Percona-Server/sql/main.cc: No such file or directory.

1. 2012-09-06T01:42:57 [New LWP 2630]
2. 2012-09-06T01:42:57 [New LWP 2636]
3. 2012-09-06T01:42:57 [New LWP 2598]
4. 2012-09-06T01:42:57 [New LWP 2600]
5. 2012-09-06T01:42:57 [New LWP 2601]
6. 2012-09-06T01:42:57 [New LWP 2603]
7. 2012-09-06T01:42:57 [New LWP 2604]
8. 2012-09-06T01:42:57 [New LWP 2608]
9. 2012-09-06T01:42:57 [New LWP 2597]
10. 2012-09-06T01:42:57 [New LWP 2611]
11. 2012-09-06T01:42:57 [New LWP 2610]
12. 2012-09-06T01:42:57 [New LWP 2593]
13. 2012-09-06T01:42:57 [New LWP 2606]
14. 2012-09-06T01:42:57 [New LWP 2615]
15. 2012-09-06T01:42:57 [New LWP 2602]
16. 2012-09-06T01:42:57 [New LWP 2634]
17. 2012-09-06T01:42:57 [New LWP 2609]
18. 2012-09-06T01:42:57 [New LWP 2632]
19. 2012-09-06T01:42:57 [New LWP 2596]
20. 2012-09-06T01:42:57 [New LWP 2607]
21. 2012-09-06T01:42:57 [New LWP 2595]
22. 2012-09-06T01:42:57 [New LWP 2599]
23. 2012-09-06T01:42:57 [New LWP 2612]
24. 2012-09-06T01:42:57 [Thread debugging using libthread_db enabled]
25. 2012-09-06T01:42:57 Using host libthread_db library "/lib/libthread_db.so.1".
26. 2012-09-06T01:42:57 Core was generated by `/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld --no-defaults --base'.
27. 2012-09-06T01:42:57 Program terminated with signal 6, Aborted.
28. 2012-09-06T01:42:57 #0 0x00007fd2ec09c9cc in pthread_kill () from /lib/libpthread.so.0
29. 2012-09-06T01:42:57 #0 0x00007fd2ec09c9cc in pthread_kill () from /lib/libpthread.so.0
30. 2012-09-06T01:42:57 #1 0x00000000007d6c58 in my_write_core (sig=6) at /media/Vone/percona-server/origin/Percona-Server/mysys/stacktrace.c:433
31. 2012-09-06T01:42:57 #2 0x00000000006ad21c in handle_fatal_signal (sig=6) at /media/Vone/percona-server/origin/Percona-Server/sql/signal_handler.cc:249
32. 2012-09-06T01:42:57 #3 <signal handler called>
33. 2012-09-06T01:42:57 #4 0x00007fd2ea655fa5 in raise () from /lib/libc.so.6
34. 2012-09-06T01:42:57 #5 0x00007fd2ea657428 in abort () from /lib/libc.so.6
35. 2012-09-06T01:42:57 #6 0x0000000000986ff5 in rec_offs_validate (rec=rec@entry=0x7fd2e3d904ec "\200", index=index@entry=0x7fd2d0024298, offsets=offsets@entry=0x7fd2e85a5900) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/rem0rec.ic:973
36. 2012-09-06T01:42:57 #7 0x000000000098a017 in lock_rec_queue_validate (block=block@entry=0x7fd2dbfe9740, rec=rec@entry=0x7fd2e3d904ec "\200", index=0x7fd2d0024298, offsets=offsets@entry=0x7fd2e85a5900) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:4756
37. 2012-09-06T01:42:57 #8 0x0000000000990a5a in lock_rec_validate_page (block=0x7fd2dbfe9740) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:4965
38. 2012-09-06T01:42:57 #9 lock_rec_block_validate (page_no=<optimized out>, space=0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:5056
39. 2012-09-06T01:42:57 #10 lock_validate () at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:5109
40. 2012-09-06T01:42:57 #11 lock_print_info_all_transactions (file=file@entry=0x294d5a0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:4580
41. 2012-09-06T01:42:57 #12 0x000000000088f112 in srv_printf_innodb_monitor (file=0x294d5a0, nowait=0, trx_start=0x7fd2e85a6780, trx_end=0x7fd2e85a6788) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:2243
42. 2012-09-06T01:42:57 #13 0x0000000000839db7 in innodb_show_status (stat_print=0x6ad7aa <stat_print(THD*, char const*, uint, char const*, uint, char const*, uint)>, thd=0x2ced230, hton=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:10374
43. 2012-09-06T01:42:57 #14 innobase_show_status (hton=<optimized out>, thd=0x2ced230, stat_print=0x6ad7aa <stat_print(THD*, char const*, uint, char const*, uint, char const*, uint)>, stat_type=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:10614
44. 2012-09-06T01:42:57 #15 0x00000000006b5876 in ha_show_status (thd=0x2ced230, db_type=0x254adb0, stat=HA_ENGINE_STATUS) at /media/Vone/percona-server/origin/Percona-Server/sql/handler.cc:5066
45. 2012-09-06T01:42:57 #16 0x0000000000595bfa in mysql_execute_command (thd=thd@entry=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:2480
46. 2012-09-06T01:42:57 #17 0x000000000059c7ea in mysql_parse (thd=thd@entry=0x2ced230, rawbuf=<optimized out>, length=25, parser_state=parser_state@entry=0x7fd2e85a7760) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:5811
47. 2012-09-06T01:42:57 #18 0x000000000059d28a in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x2ced230, packet=packet@entry=0x2d78151 "", packet_length=packet_length@entry=25) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:1061
48. 2012-09-06T01:42:57 #19 0x000000000059e75f in do_command (thd=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:788
49. 2012-09-06T01:42:57 #20 0x0000000000648061 in do_handle_one_connection (thd_arg=thd_arg@entry=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1484
50. 2012-09-06T01:42:57 #21 0x0000000000648157 in handle_one_connection (arg=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1391
51. 2012-09-06T01:42:57 #22 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
52. 2012-09-06T01:42:57 #23 0x00007fd2ea70504d in clone () from /lib/libc.so.6
53. 2012-09-06T01:42:57 #5 0x00007fd2ea657428 in abort () from /lib/libc.so.6

warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?

warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fffe276f000

1. 2012-09-06T01:42:59 [New LWP 2630]
2. 2012-09-06T01:42:59 [New LWP 2636]
3. 2012-09-06T01:42:59 [New LWP 2598]
4. 2012-09-06T01:42:59 [New LWP 2600]
5. 2012-09-06T01:42:59 [New LWP 2601]
6. 2012-09-06T01:42:59 [New LWP 2603]
7. 2012-09-06T01:42:59 [New LWP 2604]
8. 2012-09-06T01:42:59 [New LWP 2608]
9. 2012-09-06T01:42:59 [New LWP 2597]
10. 2012-09-06T01:42:59 [New LWP 2611]
11. 2012-09-06T01:42:59 [New LWP 2610]
12. 2012-09-06T01:42:59 [New LWP 2593]
13. 2012-09-06T01:42:59 [New LWP 2606]
14. 2012-09-06T01:42:59 [New LWP 2615]
15. 2012-09-06T01:42:59 [New LWP 2602]
16. 2012-09-06T01:42:59 [New LWP 2634]
17. 2012-09-06T01:42:59 [New LWP 2609]
18. 2012-09-06T01:42:59 [New LWP 2632]
19. 2012-09-06T01:42:59 [New LWP 2596]
20. 2012-09-06T01:42:59 [New LWP 2607]
21. 2012-09-06T01:42:59 [New LWP 2595]
22. 2012-09-06T01:42:59 [New LWP 2599]
23. 2012-09-06T01:42:59 [New LWP 2612]
24. 2012-09-06T01:42:59 [Thread debugging using libthread_db enabled]
25. 2012-09-06T01:42:59 Using host libthread_db library "/lib/libthread_db.so.1".
26. 2012-09-06T01:42:59 Core was generated by `/media/Vone/percona-server/raghu/Percona-Server/sql/mysqld --no-defaults --base'.
27. 2012-09-06T01:42:59 Program terminated with signal 6, Aborted.
28. 2012-09-06T01:42:59 #0 0x00007fd2ec09c9cc in pthread_kill () from /lib/libpthread.so.0
29. 2012-09-06T01:42:59
30. 2012-09-06T01:42:59 Thread 23 (Thread 0x7fd2e85e9700 (LWP 2612)):
31. 2012-09-06T01:42:59 #0 0x00007fd2ec09eea9 in do_sigwait () from /lib/libpthread.so.0
32. 2012-09-06T01:42:59 #1 0x00007fd2ec09ef3b in sigwait () from /lib/libpthread.so.0
33. 2012-09-06T01:42:59 #2 0x0000000000516ab1 in signal_hand (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/sql/mysqld.cc:2732
34. 2012-09-06T01:42:59 #3 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
35. 2012-09-06T01:42:59 #4 0x00007fd2ea70504d in clone () from /lib/libc.so.6
36. 2012-09-06T01:42:59
37. 2012-09-06T01:42:59 Thread 22 (Thread 0x7fd2dab5d700 (LWP 2599)):
38. 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1
39. 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=2, array=0x296b110) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646
40. 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=4, message1=message1@entry=0x7fd2dab5ce68, message2=message2@entry=0x7fd2dab5ce60, type=type@entry=0x7fd2dab5ce58, space_id=space_id@entry=0x7fd2dab5ce50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791
41. 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=4) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535
42. 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483
43. 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
44. 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6
45. 2012-09-06T01:42:59
46. 2012-09-06T01:42:59 Thread 21 (Thread 0x7fd2ea406700 (LWP 2595)):
47. 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1
48. 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=0, array=0x294ec10) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646
49. 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=0, message1=message1@entry=0x7fd2ea405e68, message2=message2@entry=0x7fd2ea405e60, type=type@entry=0x7fd2ea405e58, space_id=space_id@entry=0x7fd2ea405e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791
50. 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535
51. 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483
52. 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
53. 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6
54. 2012-09-06T01:42:59
55. 2012-09-06T01:42:59 Thread 20 (Thread 0x7fd2d6caa700 (LWP 2607)):
56. 2012-09-06T01:42:59 #0 0x00007fd2ec09bcc1 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
57. 2012-09-06T01:42:59 #1 0x00000000009b04cd in os_cond_wait_timed (abstime=0x7fd2d6ca9da0, mutex=0x27fa3e0, cond=0x27fa418) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:171
58. 2012-09-06T01:42:59 #2 os_event_wait_time_low (event=0x27fa3e0, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=1) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:716
59. 2012-09-06T01:42:59 #3 0x0000000000890c18 in srv_error_monitor_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:2887
60. 2012-09-06T01:42:59 #4 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
61. 2012-09-06T01:42:59 #5 0x00007fd2ea70504d in clone () from /lib/libc.so.6
62. 2012-09-06T01:42:59
63. 2012-09-06T01:42:59 Thread 19 (Thread 0x7fd2e8dea700 (LWP 2596)):
64. 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1
65. 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=0, array=0x295ce90) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646
66. 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=1, message1=message1@entry=0x7fd2e8de9e68, message2=message2@entry=0x7fd2e8de9e60, type=type@entry=0x7fd2e8de9e58, space_id=space_id@entry=0x7fd2e8de9e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791
67. 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=1) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535
68. 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483
69. 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
70. 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6
71. 2012-09-06T01:42:59
72. 2012-09-06T01:42:59 Thread 18 (Thread 0x7fd2e8526700 (LWP 2632)):
73. 2012-09-06T01:42:59 #0 0x00007fd2ec09b954 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
74. 2012-09-06T01:42:59 #1 0x00000000009b05c5 in os_cond_wait (mutex=0x2c88100, cond=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:207
75. 2012-09-06T01:42:59 #2 os_event_wait_low (event=event@entry=0x2c88100, reset_sig_count=4209) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:609
76. 2012-09-06T01:42:59 #3 0x000000000089823e in sync_array_wait_event (arr=0x254a440, index=2) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0arr.c:458
77. 2012-09-06T01:42:59 #4 0x0000000000898e71 in rw_lock_s_lock_spin (lock=lock@entry=0x7fd2dbfe98b8, pass=pass@entry=0, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0rw.c:453
78. 2012-09-06T01:42:59 #5 0x00000000008f8960 in rw_lock_s_lock_func (line=253, file_name=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", pass=0, lock=0x7fd2dbfe98b8) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:419
79. 2012-09-06T01:42:59 #6 pfs_rw_lock_s_lock_func (lock=lock@entry=0x7fd2dbfe98b8, pass=0, pass@entry=140543342471336, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:779
80. 2012-09-06T01:42:59 #7 0x0000000000900d24 in buf_page_get_gen (space=2550926246, space@entry=0, zip_size=zip_size@entry=0, offset=offset@entry=314, rw_latch=rw_latch@entry=1, guess=guess@entry=0x0, mode=mode@entry=10, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2e8523dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/buf/buf0buf.c:2942
81. 2012-09-06T01:42:59 #8 0x00000000008d7ba2 in btr_block_get_func (space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, mode=mode@entry=1, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2e8523dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0btr.ic:59
82. 2012-09-06T01:42:59 #9 0x00000000008dc24e in btr_cur_latch_leaves (page=page@entry=0x7fd2e3d90000 "\213\363f\241", space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x7fd2cc016cb8, mtr=mtr@entry=0x7fd2e8523dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:252
83. 2012-09-06T01:42:59 #10 0x00000000008dd45d in btr_cur_search_to_nth_level (index=index@entry=0x7fd2d001b728, level=level@entry=0, tuple=tuple@entry=0x7fd2cc02a4e8, mode=4, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x7fd2cc016cb8, has_search_latch=has_search_latch@entry=0, file=file@entry=0xb1b1c0 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0sel.c", line=line@entry=3097, mtr=mtr@entry=0x7fd2e8523dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:756
84. 2012-09-06T01:42:59 #11 0x00000000008e9279 in btr_pcur_open_with_no_init_func (mtr=0x7fd2e8523dc0, line=3097, file=0xb1b1c0 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0sel.c", has_search_latch=0, cursor=0x7fd2cc016cb8, latch_mode=1, mode=<optimized out>, tuple=0x7fd2cc02a4e8, index=0x7fd2d001b728) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0pcur.ic:482
85. 2012-09-06T01:42:59 #12 btr_pcur_restore_position_func (latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x7fd2cc016cb8, file=file@entry=0xb1b1c0 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0sel.c", line=line@entry=3097, mtr=mtr@entry=0x7fd2e8523dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0pcur.c:351
86. 2012-09-06T01:42:59 #13 0x0000000000870b0e in sel_restore_position_for_mysql (same_user_rec=same_user_rec@entry=0x7fd2e85242a0, latch_mode=latch_mode@entry=1, pcur=pcur@entry=0x7fd2cc016cb8, moves_up=moves_up@entry=1, mtr=mtr@entry=0x7fd2e8523dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0sel.c:3097
87. 2012-09-06T01:42:59 #14 0x0000000000878963 in row_search_for_mysql (buf=0x7fd2cc012008 "\377", mode=1, prebuilt=0x7fd2cc016c48, match_mode=0, direction=0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0sel.c:4759
88. 2012-09-06T01:42:59 #15 0x00000000008349ae in ha_innobase::index_read (this=0x7fd2cc0123e0, buf=0x7fd2cc012008 "\377", key_ptr=<optimized out>, key_len=0, find_flag=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:6665
89. 2012-09-06T01:42:59 #16 0x0000000000832071 in ha_innobase::index_first (this=0x7fd2cc0123e0, buf=0x7fd2cc012008 "\377") at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:7015
90. 2012-09-06T01:42:59 #17 0x000000000083f703 in ha_innobase::rnd_next (this=0x7fd2cc0123e0, buf=0x7fd2cc012008 "\377") at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:7112
91. 2012-09-06T01:42:59 #18 0x0000000000a18506 in ha_partition::rnd_next (this=0x7fd2cc011aa0, buf=0x7fd2cc012008 "\377") at /media/Vone/percona-server/origin/Percona-Server/sql/ha_partition.cc:4032
92. 2012-09-06T01:42:59 #19 0x0000000000777233 in rr_sequential (info=0x7fd2e8524590) at /media/Vone/percona-server/origin/Percona-Server/sql/records.cc:455
93. 2012-09-06T01:42:59 #20 0x0000000000613f85 in mysql_update (thd=0x2d83e40, table_list=0x7fd2cc004e38, fields=..., values=..., conds=0x0, order_num=<optimized out>, order=0x0, limit=18446744073709551615, handle_duplicates=DUP_ERROR, ignore=false, found_return=0x7fd2e85248f0, updated_return=0x7fd2e85248f8) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_update.cc:653
94. 2012-09-06T01:42:59 #21 0x000000000059785c in mysql_execute_command (thd=thd@entry=0x2d83e40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:2940
95. 2012-09-06T01:42:59 #22 0x000000000059c7ea in mysql_parse (thd=thd@entry=0x2d83e40, rawbuf=<optimized out>, length=73, parser_state=parser_state@entry=0x7fd2e8525760) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:5811
96. 2012-09-06T01:42:59 #23 0x000000000059d28a in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x2d83e40, packet=packet@entry=0x2d87581 "UPDATE `table100_innodb_key_pk_parts_2_int_autoinc` SET `col_int_key` = 4", packet_length=packet_length@entry=73) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:1061
97. 2012-09-06T01:42:59 #24 0x000000000059e75f in do_command (thd=0x2d83e40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:788
98. 2012-09-06T01:42:59 #25 0x0000000000648061 in do_handle_one_connection (thd_arg=thd_arg@entry=0x2d83e40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1484
99. 2012-09-06T01:42:59 #26 0x0000000000648157 in handle_one_connection (arg=0x2d83e40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1391
100. 2012-09-06T01:42:59 #27 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
101. 2012-09-06T01:42:59 #28 0x00007fd2ea70504d in clone () from /lib/libc.so.6
102. 2012-09-06T01:42:59
103. 2012-09-06T01:42:59 Thread 17 (Thread 0x7fd2d5ca8700 (LWP 2609)):
104. 2012-09-06T01:42:59 #0 0x00007fd2ea6fec63 in select () from /lib/libc.so.6
105. 2012-09-06T01:42:59 #1 0x00000000009b0cdc in os_thread_sleep (tm=tm@entry=5000000) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0thread.c:259
106. 2012-09-06T01:42:59 #2 0x0000000000890c78 in srv_LRU_dump_restore_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:2933
107. 2012-09-06T01:42:59 #3 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
108. 2012-09-06T01:42:59 #4 0x00007fd2ea70504d in clone () from /lib/libc.so.6
109. 2012-09-06T01:42:59
110. 2012-09-06T01:42:59 Thread 16 (Thread 0x7fd2e84e5700 (LWP 2634)):
111. 2012-09-06T01:42:59 #0 0x00007fd2ec09b954 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
112. 2012-09-06T01:42:59 #1 0x00000000009b05c5 in os_cond_wait (mutex=0x2c88100, cond=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:207
113. 2012-09-06T01:42:59 #2 os_event_wait_low (event=event@entry=0x2c88100, reset_sig_count=4209) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:609
114. 2012-09-06T01:42:59 #3 0x000000000089823e in sync_array_wait_event (arr=0x254a440, index=3) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0arr.c:458
115. 2012-09-06T01:42:59 #4 0x0000000000898e71 in rw_lock_s_lock_spin (lock=lock@entry=0x7fd2dbfe98b8, pass=pass@entry=0, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0rw.c:453
116. 2012-09-06T01:42:59 #5 0x00000000008f8960 in rw_lock_s_lock_func (line=253, file_name=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", pass=0, lock=0x7fd2dbfe98b8) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:419
117. 2012-09-06T01:42:59 #6 pfs_rw_lock_s_lock_func (lock=lock@entry=0x7fd2dbfe98b8, pass=0, pass@entry=140543208226584, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:779
118. 2012-09-06T01:42:59 #7 0x0000000000900d24 in buf_page_get_gen (space=2550926246, space@entry=0, zip_size=zip_size@entry=0, offset=offset@entry=314, rw_latch=rw_latch@entry=1, guess=guess@entry=0x0, mode=mode@entry=10, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2e84e2dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/buf/buf0buf.c:2942
119. 2012-09-06T01:42:59 #8 0x00000000008d7ba2 in btr_block_get_func (space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, mode=mode@entry=1, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2e84e2dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0btr.ic:59
120. 2012-09-06T01:42:59 #9 0x00000000008dc24e in btr_cur_latch_leaves (page=0x7fd2e3d90000 "\213\363f\241", space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x7fd2cc028678, mtr=mtr@entry=0x7fd2e84e2dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:252
121. 2012-09-06T01:42:59 #10 0x00000000008ddf3d in btr_cur_open_at_index_side_func (from_left=from_left@entry=1, index=index@entry=0x7fd2d001b728, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x7fd2cc028678, file=file@entry=0xb160b8 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0pcur.ic", line=line@entry=519, mtr=mtr@entry=0x7fd2e84e2dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:941
122. 2012-09-06T01:42:59 #11 0x0000000000876aee in btr_pcur_open_at_index_side (mtr=0x7fd2e84e2dc0, do_init=0, pcur=0x7fd2cc028678, latch_mode=1, index=0x7fd2d001b728, from_left=1) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0pcur.ic:518
123. 2012-09-06T01:42:59 #12 row_search_for_mysql (buf=0x7fd2cc0239c8 "\377", mode=1, prebuilt=0x7fd2cc028608, match_mode=0, direction=0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0sel.c:3976
124. 2012-09-06T01:42:59 #13 0x00000000008349ae in ha_innobase::index_read (this=0x7fd2cc023da0, buf=0x7fd2cc0239c8 "\377", key_ptr=<optimized out>, key_len=0, find_flag=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:6665
125. 2012-09-06T01:42:59 #14 0x0000000000832071 in ha_innobase::index_first (this=0x7fd2cc023da0, buf=0x7fd2cc0239c8 "\377") at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:7015
126. 2012-09-06T01:42:59 #15 0x000000000083f703 in ha_innobase::rnd_next (this=0x7fd2cc023da0, buf=0x7fd2cc0239c8 "\377") at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:7112
127. 2012-09-06T01:42:59 #16 0x0000000000a18506 in ha_partition::rnd_next (this=0x7fd2cc023460, buf=0x7fd2cc0239c8 "\377") at /media/Vone/percona-server/origin/Percona-Server/sql/ha_partition.cc:4032
128. 2012-09-06T01:42:59 #17 0x0000000000777233 in rr_sequential (info=0x7fd2e84e3590) at /media/Vone/percona-server/origin/Percona-Server/sql/records.cc:455
129. 2012-09-06T01:42:59 #18 0x0000000000613f85 in mysql_update (thd=0x2d8b6c0, table_list=0x7fd2c4004e38, fields=..., values=..., conds=0x0, order_num=<optimized out>, order=0x0, limit=18446744073709551615, handle_duplicates=DUP_ERROR, ignore=false, found_return=0x7fd2e84e38f0, updated_return=0x7fd2e84e38f8) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_update.cc:653
130. 2012-09-06T01:42:59 #19 0x000000000059785c in mysql_execute_command (thd=thd@entry=0x2d8b6c0) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:2940
131. 2012-09-06T01:42:59 #20 0x000000000059c7ea in mysql_parse (thd=thd@entry=0x2d8b6c0, rawbuf=<optimized out>, length=73, parser_state=parser_state@entry=0x7fd2e84e4760) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:5811
132. 2012-09-06T01:42:59 #21 0x000000000059d28a in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x2d8b6c0, packet=packet@entry=0x2d8ee01 "UPDATE `table100_innodb_key_pk_parts_2_int_autoinc` SET `col_int_key` = 7", packet_length=packet_length@entry=73) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:1061
133. 2012-09-06T01:42:59 #22 0x000000000059e75f in do_command (thd=0x2d8b6c0) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:788
134. 2012-09-06T01:42:59 #23 0x0000000000648061 in do_handle_one_connection (thd_arg=thd_arg@entry=0x2d8b6c0) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1484
135. 2012-09-06T01:42:59 #24 0x0000000000648157 in handle_one_connection (arg=0x2d8b6c0) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1391
136. 2012-09-06T01:42:59 #25 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
137. 2012-09-06T01:42:59 #26 0x00007fd2ea70504d in clone () from /lib/libc.so.6
138. 2012-09-06T01:42:59
139. 2012-09-06T01:42:59 Thread 15 (Thread 0x7fd2d935a700 (LWP 2602)):
140. 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1
141. 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=1, array=0x2973390) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646
142. 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=7, message1=message1@entry=0x7fd2d9359e68, message2=message2@entry=0x7fd2d9359e60, type=type@entry=0x7fd2d9359e58, space_id=space_id@entry=0x7fd2d9359e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791
143. 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=7) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535
144. 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483
145. 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
146. 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6
147. 2012-09-06T01:42:59
148. 2012-09-06T01:42:59 Thread 14 (Thread 0x7fd2e8567700 (LWP 2615)):
149. 2012-09-06T01:42:59 #0 0x00007fd2ec09e37d in read () from /lib/libpthread.so.0
150. 2012-09-06T01:42:59 #1 0x0000000000a3abda in vio_read (vio=0x2d7fb10, buf=0x2d7fd00 "\a", size=4) at /media/Vone/percona-server/origin/Percona-Server/vio/viosocket.c:56
151. 2012-09-06T01:42:59 #2 0x000000000051f33f in my_real_read (net=net@entry=0x2d7c4c0, complen=complen@entry=0x7fd2e8566d88) at /media/Vone/percona-server/origin/Percona-Server/sql/net_serv.cc:832
152. 2012-09-06T01:42:59 #3 0x000000000051fd41 in my_net_read (net=0x2d7c4c0) at /media/Vone/percona-server/origin/Percona-Server/sql/net_serv.cc:1029
153. 2012-09-06T01:42:59 #4 0x000000000059e5f5 in do_command (thd=0x2d7c290) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:734
154. 2012-09-06T01:42:59 #5 0x0000000000648061 in do_handle_one_connection (thd_arg=thd_arg@entry=0x2d7c290) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1484
155. 2012-09-06T01:42:59 #6 0x0000000000648157 in handle_one_connection (arg=0x2d7c290) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1391
156. 2012-09-06T01:42:59 #7 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
157. 2012-09-06T01:42:59 #8 0x00007fd2ea70504d in clone () from /lib/libc.so.6
158. 2012-09-06T01:42:59
159. 2012-09-06T01:42:59 Thread 13 (Thread 0x7fd2d74ab700 (LWP 2606)):
160. 2012-09-06T01:42:59 #0 0x00007fd2ec09bcc1 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
161. 2012-09-06T01:42:59 #1 0x00000000009b04cd in os_cond_wait_timed (abstime=0x7fd2d74aadc0, mutex=0x27fa460, cond=0x27fa498) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:171
162. 2012-09-06T01:42:59 #2 os_event_wait_time_low (event=0x27fa460, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=1) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:716
163. 2012-09-06T01:42:59 #3 0x000000000089001d in srv_lock_timeout_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:2674
164. 2012-09-06T01:42:59 #4 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
165. 2012-09-06T01:42:59 #5 0x00007fd2ea70504d in clone () from /lib/libc.so.6
166. 2012-09-06T01:42:59
167. 2012-09-06T01:42:59 Thread 12 (Thread 0x7fd2ea408740 (LWP 2593)):
168. 2012-09-06T01:42:59 #0 0x00007fd2ea6fcebd in poll () from /lib/libc.so.6
169. 2012-09-06T01:42:59 #1 0x000000000051c231 in poll (__timeout=-1, __nfds=2, __fds=0x7fffe26b9850) at /usr/include/bits/poll2.h:41
170. 2012-09-06T01:42:59 #2 handle_connections_sockets () at /media/Vone/percona-server/origin/Percona-Server/sql/mysqld.cc:5361
171. 2012-09-06T01:42:59 #3 0x000000000051e100 in mysqld_main (argc=47, argv=0x254a298) at /media/Vone/percona-server/origin/Percona-Server/sql/mysqld.cc:4800
172. 2012-09-06T01:42:59 #4 0x0000000000513975 in main (argc=<optimized out>, argv=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/sql/main.cc:25
173. 2012-09-06T01:42:59
174. 2012-09-06T01:42:59 Thread 11 (Thread 0x7fd2d54a7700 (LWP 2610)):
175. 2012-09-06T01:42:59 #0 0x00007fd2ea6fec63 in select () from /lib/libc.so.6
176. 2012-09-06T01:42:59 #1 0x00000000009b0cdc in os_thread_sleep (tm=tm@entry=899000) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0thread.c:259
177. 2012-09-06T01:42:59 #2 0x0000000000891a36 in srv_master_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:3298
178. 2012-09-06T01:42:59 #3 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
179. 2012-09-06T01:42:59 #4 0x00007fd2ea70504d in clone () from /lib/libc.so.6
180. 2012-09-06T01:42:59
181. 2012-09-06T01:42:59 Thread 10 (Thread 0x7fd2d4ca6700 (LWP 2611)):
182. 2012-09-06T01:42:59 #0 0x00007fd2ec09b954 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
183. 2012-09-06T01:42:59 #1 0x00000000009b05c5 in os_cond_wait (mutex=0x2c88100, cond=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:207
184. 2012-09-06T01:42:59 #2 os_event_wait_low (event=event@entry=0x2c88100, reset_sig_count=4209) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:609
185. 2012-09-06T01:42:59 #3 0x000000000089823e in sync_array_wait_event (arr=0x254a440, index=1) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0arr.c:458
186. 2012-09-06T01:42:59 #4 0x0000000000898e71 in rw_lock_s_lock_spin (lock=lock@entry=0x7fd2dbfe98b8, pass=pass@entry=0, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0rw.c:453
187. 2012-09-06T01:42:59 #5 0x00000000008f8960 in rw_lock_s_lock_func (line=253, file_name=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", pass=0, lock=0x7fd2dbfe98b8) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:419
188. 2012-09-06T01:42:59 #6 pfs_rw_lock_s_lock_func (lock=lock@entry=0x7fd2dbfe98b8, pass=pass@entry=0, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:779
189. 2012-09-06T01:42:59 #7 0x0000000000900d24 in buf_page_get_gen (space=2550926246, space@entry=0, zip_size=zip_size@entry=0, offset=offset@entry=314, rw_latch=rw_latch@entry=1, guess=guess@entry=0x0, mode=mode@entry=10, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2d4ca4b40) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/buf/buf0buf.c:2942
190. 2012-09-06T01:42:59 #8 0x00000000008d7ba2 in btr_block_get_func (space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, mode=mode@entry=1, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2d4ca4b40) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0btr.ic:59
191. 2012-09-06T01:42:59 #9 0x00000000008dc24e in btr_cur_latch_leaves (page=page@entry=0x7fd2e3d90000 "\213\363f\241", space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x2cc1068, mtr=mtr@entry=0x7fd2d4ca4b40) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:252
192. 2012-09-06T01:42:59 #10 0x00000000008dd45d in btr_cur_search_to_nth_level (index=0x7fd2d001b728, level=level@entry=0, tuple=tuple@entry=0x2cc15a8, mode=mode@entry=4, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x2cc1068, has_search_latch=has_search_latch@entry=0, file=file@entry=0xb1ac58 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0row.c", line=line@entry=716, mtr=mtr@entry=0x7fd2d4ca4b40) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:756
193. 2012-09-06T01:42:59 #11 0x000000000086c136 in btr_pcur_open_func (mtr=0x7fd2d4ca4b40, line=716, file=0xb1ac58 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0row.c", cursor=0x2cc1068, latch_mode=1, mode=4, tuple=0x2cc15a8, index=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0pcur.ic:438
194. 2012-09-06T01:42:59 #12 row_search_on_row_ref (pcur=pcur@entry=0x2cc1068, mode=mode@entry=1, table=0x7fd2d0026858, ref=0x2cc15a8, mtr=mtr@entry=0x7fd2d4ca4b40) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0row.c:716
195. 2012-09-06T01:42:59 #13 0x00000000009e9a56 in row_purge_reposition_pcur (mode=mode@entry=1, node=node@entry=0x2cc1008, mtr=mtr@entry=0x7fd2d4ca4b40) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0purge.c:101
196. 2012-09-06T01:42:59 #14 0x00000000009ea871 in row_purge_poss_sec (node=node@entry=0x2cc1008, index=index@entry=0x7fd2d001aa48, entry=entry@entry=0x7fd2c0000ca8) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0purge.c:256
197. 2012-09-06T01:42:59 #15 0x00000000009eab2c in row_purge_remove_sec_if_poss_leaf (node=node@entry=0x2cc1008, index=index@entry=0x7fd2d001aa48, entry=entry@entry=0x7fd2c0000ca8) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0purge.c:383
198. 2012-09-06T01:42:59 #16 0x00000000009eb305 in row_purge_remove_sec_if_poss (entry=0x7fd2c0000ca8, index=0x7fd2d001aa48, node=0x2cc1008) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0purge.c:433
199. 2012-09-06T01:42:59 #17 row_purge_upd_exist_or_extern_func (thr=thr@entry=0x2cc0e50, node=node@entry=0x2cc1008) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0purge.c:541
200. 2012-09-06T01:42:59 #18 0x00000000009ec69c in row_purge (thr=0x2cc0e50, node=0x2cc1008) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0purge.c:770
201. 2012-09-06T01:42:59 #19 row_purge_step (thr=thr@entry=0x2cc0e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0purge.c:805
202. 2012-09-06T01:42:59 #20 0x00000000009d468f in que_thr_step (thr=0x2cc0e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/que/que0que.c:1259
203. 2012-09-06T01:42:59 #21 que_run_threads_low (thr=0x2cc0e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/que/que0que.c:1319
204. 2012-09-06T01:42:59 #22 que_run_threads (thr=thr@entry=0x2cc0e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/que/que0que.c:1356
205. 2012-09-06T01:42:59 #23 0x00000000008a2a42 in trx_purge (limit=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/trx/trx0purge.c:1195
206. 2012-09-06T01:42:59 #24 0x0000000000893d26 in srv_purge_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:3967
207. 2012-09-06T01:42:59 #25 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
208. 2012-09-06T01:42:59 #26 0x00007fd2ea70504d in clone () from /lib/libc.so.6
209. 2012-09-06T01:42:59
210. 2012-09-06T01:42:59 Thread 9 (Thread 0x7fd2dbb5f700 (LWP 2597)):
211. 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1
212. 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=0, array=0x296b110) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646
213. 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=2, message1=message1@entry=0x7fd2dbb5ee68, message2=message2@entry=0x7fd2dbb5ee60, type=type@entry=0x7fd2dbb5ee58, space_id=space_id@entry=0x7fd2dbb5ee50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791
214. 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=2) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535
215. 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483
216. 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
217. 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6
218. 2012-09-06T01:42:59
219. 2012-09-06T01:42:59 Thread 8 (Thread 0x7fd2d64a9700 (LWP 2608)):
220. 2012-09-06T01:42:59 #0 0x00007fd2ec09bcc1 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
221. 2012-09-06T01:42:59 #1 0x00000000009b04cd in os_cond_wait_timed (abstime=0x7fd2d64a8e10, mutex=0x27fa4e0, cond=0x27fa518) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:171
222. 2012-09-06T01:42:59 #2 os_event_wait_time_low (event=0x27fa4e0, time_in_usec=time_in_usec@entry=5000000, reset_sig_count=1) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:716
223. 2012-09-06T01:42:59 #3 0x000000000088fd14 in srv_monitor_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:2524
224. 2012-09-06T01:42:59 #4 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
225. 2012-09-06T01:42:59 #5 0x00007fd2ea70504d in clone () from /lib/libc.so.6
226. 2012-09-06T01:42:59
227. 2012-09-06T01:42:59 Thread 7 (Thread 0x7fd2d8358700 (LWP 2604)):
228. 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1
229. 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=3, array=0x2973390) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646
230. 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=9, message1=message1@entry=0x7fd2d8357e68, message2=message2@entry=0x7fd2d8357e60, type=type@entry=0x7fd2d8357e58, space_id=space_id@entry=0x7fd2d8357e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791
231. 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=9) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535
232. 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483
233. 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
234. 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6
235. 2012-09-06T01:42:59
236. 2012-09-06T01:42:59 Thread 6 (Thread 0x7fd2d8b59700 (LWP 2603)):
237. 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1
238. 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=2, array=0x2973390) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646
239. 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=8, message1=message1@entry=0x7fd2d8b58e68, message2=message2@entry=0x7fd2d8b58e60, type=type@entry=0x7fd2d8b58e58, space_id=space_id@entry=0x7fd2d8b58e50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791
240. 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=8) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535
241. 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483
242. 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
243. 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6
244. 2012-09-06T01:42:59
245. 2012-09-06T01:42:59 Thread 5 (Thread 0x7fd2d9b5b700 (LWP 2601)):
246. 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1
247. 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=0, array=0x2973390) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646
248. 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=6, message1=message1@entry=0x7fd2d9b5ae68, message2=message2@entry=0x7fd2d9b5ae60, type=type@entry=0x7fd2d9b5ae58, space_id=space_id@entry=0x7fd2d9b5ae50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791
249. 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=6) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535
250. 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483
251. 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
252. 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6
253. 2012-09-06T01:42:59
254. 2012-09-06T01:42:59 Thread 4 (Thread 0x7fd2da35c700 (LWP 2600)):
255. 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1
256. 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=3, array=0x296b110) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646
257. 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=5, message1=message1@entry=0x7fd2da35be68, message2=message2@entry=0x7fd2da35be60, type=type@entry=0x7fd2da35be58, space_id=space_id@entry=0x7fd2da35be50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791
258. 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=5) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535
259. 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483
260. 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
261. 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6
262. 2012-09-06T01:42:59
263. 2012-09-06T01:42:59 Thread 3 (Thread 0x7fd2db35e700 (LWP 2598)):
264. 2012-09-06T01:42:59 #0 0x00007fd2ebe8f664 in ?? () from /lib/libaio.so.1
265. 2012-09-06T01:42:59 #1 0x00000000009adb5c in os_aio_linux_collect (seg_size=256, segment=1, array=0x296b110) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4646
266. 2012-09-06T01:42:59 #2 os_aio_linux_handle (global_seg=global_seg@entry=3, message1=message1@entry=0x7fd2db35de68, message2=message2@entry=0x7fd2db35de60, type=type@entry=0x7fd2db35de58, space_id=space_id@entry=0x7fd2db35de50) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0file.c:4791
267. 2012-09-06T01:42:59 #3 0x000000000095398d in fil_aio_wait (segment=segment@entry=3) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/fil/fil0fil.c:5535
268. 2012-09-06T01:42:59 #4 0x0000000000894323 in io_handler_thread (arg=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0start.c:483
269. 2012-09-06T01:42:59 #5 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
270. 2012-09-06T01:42:59 #6 0x00007fd2ea70504d in clone () from /lib/libc.so.6
271. 2012-09-06T01:42:59
272. 2012-09-06T01:42:59 Thread 2 (Thread 0x7fd2e84a4700 (LWP 2636)):
273. 2012-09-06T01:42:59 #0 0x00007fd2ec09b954 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
274. 2012-09-06T01:42:59 #1 0x00000000009b05c5 in os_cond_wait (mutex=0x2c88100, cond=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:207
275. 2012-09-06T01:42:59 #2 os_event_wait_low (event=event@entry=0x2c88100, reset_sig_count=4209) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/os/os0sync.c:609
276. 2012-09-06T01:42:59 #3 0x000000000089823e in sync_array_wait_event (arr=0x254a440, index=0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0arr.c:458
277. 2012-09-06T01:42:59 #4 0x0000000000898e71 in rw_lock_s_lock_spin (lock=lock@entry=0x7fd2dbfe98b8, pass=pass@entry=0, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/sync/sync0rw.c:453
278. 2012-09-06T01:42:59 #5 0x00000000008f8960 in rw_lock_s_lock_func (line=253, file_name=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", pass=0, lock=0x7fd2dbfe98b8) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:419
279. 2012-09-06T01:42:59 #6 pfs_rw_lock_s_lock_func (lock=lock@entry=0x7fd2dbfe98b8, pass=0, pass@entry=140543006899528, file_name=file_name@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/sync0rw.ic:779
280. 2012-09-06T01:42:59 #7 0x0000000000900d24 in buf_page_get_gen (space=2550926246, space@entry=0, zip_size=zip_size@entry=0, offset=offset@entry=314, rw_latch=rw_latch@entry=1, guess=guess@entry=0x0, mode=mode@entry=10, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2e84a1dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/buf/buf0buf.c:2942
281. 2012-09-06T01:42:59 #8 0x00000000008d7ba2 in btr_block_get_func (space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, mode=mode@entry=1, file=file@entry=0xb28250 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c", line=line@entry=253, mtr=mtr@entry=0x7fd2e84a1dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0btr.ic:59
282. 2012-09-06T01:42:59 #9 0x00000000008dc24e in btr_cur_latch_leaves (page=0x7fd2e3d90000 "\213\363f\241", space=space@entry=0, zip_size=zip_size@entry=0, page_no=page_no@entry=314, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x7fd2b800e978, mtr=mtr@entry=0x7fd2e84a1dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:252
283. 2012-09-06T01:42:59 #10 0x00000000008ddf3d in btr_cur_open_at_index_side_func (from_left=from_left@entry=1, index=index@entry=0x7fd2d001b728, latch_mode=latch_mode@entry=1, cursor=cursor@entry=0x7fd2b800e978, file=file@entry=0xb160b8 "/media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0pcur.ic", line=line@entry=519, mtr=mtr@entry=0x7fd2e84a1dc0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/btr/btr0cur.c:941
284. 2012-09-06T01:42:59 #11 0x0000000000876aee in btr_pcur_open_at_index_side (mtr=0x7fd2e84a1dc0, do_init=0, pcur=0x7fd2b800e978, latch_mode=1, index=0x7fd2d001b728, from_left=1) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/btr0pcur.ic:518
285. 2012-09-06T01:42:59 #12 row_search_for_mysql (buf=0x7fd2b8008678 "\377", mode=1, prebuilt=0x7fd2b800e908, match_mode=0, direction=0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/row/row0sel.c:3976
286. 2012-09-06T01:42:59 #13 0x00000000008349ae in ha_innobase::index_read (this=0x7fd2b800a0a0, buf=0x7fd2b8008678 "\377", key_ptr=<optimized out>, key_len=0, find_flag=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:6665
287. 2012-09-06T01:42:59 #14 0x0000000000832071 in ha_innobase::index_first (this=0x7fd2b800a0a0, buf=0x7fd2b8008678 "\377") at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:7015
288. 2012-09-06T01:42:59 #15 0x000000000083f703 in ha_innobase::rnd_next (this=0x7fd2b800a0a0, buf=0x7fd2b8008678 "\377") at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:7112
289. 2012-09-06T01:42:59 #16 0x0000000000a18506 in ha_partition::rnd_next (this=0x7fd2b8009b50, buf=0x7fd2b8008678 "\377") at /media/Vone/percona-server/origin/Percona-Server/sql/ha_partition.cc:4032
290. 2012-09-06T01:42:59 #17 0x0000000000777233 in rr_sequential (info=0x7fd2e84a2590) at /media/Vone/percona-server/origin/Percona-Server/sql/records.cc:455
291. 2012-09-06T01:42:59 #18 0x0000000000613f85 in mysql_update (thd=0x2d92f40, table_list=0x7fd2b8004e38, fields=..., values=..., conds=0x0, order_num=<optimized out>, order=0x0, limit=18446744073709551615, handle_duplicates=DUP_ERROR, ignore=false, found_return=0x7fd2e84a28f0, updated_return=0x7fd2e84a28f8) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_update.cc:653
292. 2012-09-06T01:42:59 #19 0x000000000059785c in mysql_execute_command (thd=thd@entry=0x2d92f40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:2940
293. 2012-09-06T01:42:59 #20 0x000000000059c7ea in mysql_parse (thd=thd@entry=0x2d92f40, rawbuf=<optimized out>, length=73, parser_state=parser_state@entry=0x7fd2e84a3760) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:5811
294. 2012-09-06T01:42:59 #21 0x000000000059d28a in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x2d92f40, packet=packet@entry=0x2d96681 "UPDATE `table100_innodb_key_pk_parts_2_int_autoinc` SET `col_int_key` = 1", packet_length=packet_length@entry=73) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:1061
295. 2012-09-06T01:42:59 #22 0x000000000059e75f in do_command (thd=0x2d92f40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:788
296. 2012-09-06T01:42:59 #23 0x0000000000648061 in do_handle_one_connection (thd_arg=thd_arg@entry=0x2d92f40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1484
297. 2012-09-06T01:42:59 #24 0x0000000000648157 in handle_one_connection (arg=0x2d92f40) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1391
298. 2012-09-06T01:42:59 #25 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
299. 2012-09-06T01:42:59 #26 0x00007fd2ea70504d in clone () from /lib/libc.so.6
300. 2012-09-06T01:42:59
301. 2012-09-06T01:42:59 Thread 1 (Thread 0x7fd2e85a8700 (LWP 2630)):
302. 2012-09-06T01:42:59 #0 0x00007fd2ec09c9cc in pthread_kill () from /lib/libpthread.so.0
303. 2012-09-06T01:42:59 #1 0x00000000007d6c58 in my_write_core (sig=6) at /media/Vone/percona-server/origin/Percona-Server/mysys/stacktrace.c:433
304. 2012-09-06T01:42:59 #2 0x00000000006ad21c in handle_fatal_signal (sig=6) at /media/Vone/percona-server/origin/Percona-Server/sql/signal_handler.cc:249
305. 2012-09-06T01:42:59 #3 <signal handler called>
306. 2012-09-06T01:42:59 #4 0x00007fd2ea655fa5 in raise () from /lib/libc.so.6
307. 2012-09-06T01:42:59 #5 0x00007fd2ea657428 in abort () from /lib/libc.so.6
308. 2012-09-06T01:42:59 #6 0x0000000000986ff5 in rec_offs_validate (rec=rec@entry=0x7fd2e3d904ec "\200", index=index@entry=0x7fd2d0024298, offsets=offsets@entry=0x7fd2e85a5900) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/include/rem0rec.ic:973
309. 2012-09-06T01:42:59 #7 0x000000000098a017 in lock_rec_queue_validate (block=block@entry=0x7fd2dbfe9740, rec=rec@entry=0x7fd2e3d904ec "\200", index=0x7fd2d0024298, offsets=offsets@entry=0x7fd2e85a5900) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:4756
310. 2012-09-06T01:42:59 #8 0x0000000000990a5a in lock_rec_validate_page (block=0x7fd2dbfe9740) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:4965
311. 2012-09-06T01:42:59 #9 lock_rec_block_validate (page_no=<optimized out>, space=0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:5056
312. 2012-09-06T01:42:59 #10 lock_validate () at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:5109
313. 2012-09-06T01:42:59 #11 lock_print_info_all_transactions (file=file@entry=0x294d5a0) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/lock/lock0lock.c:4580
314. 2012-09-06T01:42:59 #12 0x000000000088f112 in srv_printf_innodb_monitor (file=0x294d5a0, nowait=0, trx_start=0x7fd2e85a6780, trx_end=0x7fd2e85a6788) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/srv/srv0srv.c:2243
315. 2012-09-06T01:42:59 #13 0x0000000000839db7 in innodb_show_status (stat_print=0x6ad7aa <stat_print(THD*, char const*, uint, char const*, uint, char const*, uint)>, thd=0x2ced230, hton=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:10374
316. 2012-09-06T01:42:59 #14 innobase_show_status (hton=<optimized out>, thd=0x2ced230, stat_print=0x6ad7aa <stat_print(THD*, char const*, uint, char const*, uint, char const*, uint)>, stat_type=<optimized out>) at /media/Vone/percona-server/origin/Percona-Server/storage/innobase/handler/ha_innodb.cc:10614
317. 2012-09-06T01:42:59 #15 0x00000000006b5876 in ha_show_status (thd=0x2ced230, db_type=0x254adb0, stat=HA_ENGINE_STATUS) at /media/Vone/percona-server/origin/Percona-Server/sql/handler.cc:5066
318. 2012-09-06T01:42:59 #16 0x0000000000595bfa in mysql_execute_command (thd=thd@entry=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:2480
319. 2012-09-06T01:42:59 #17 0x000000000059c7ea in mysql_parse (thd=thd@entry=0x2ced230, rawbuf=<optimized out>, length=25, parser_state=parser_state@entry=0x7fd2e85a7760) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:5811
320. 2012-09-06T01:42:59 #18 0x000000000059d28a in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x2ced230, packet=packet@entry=0x2d78151 "", packet_length=packet_length@entry=25) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:1061
321. 2012-09-06T01:42:59 #19 0x000000000059e75f in do_command (thd=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_parse.cc:788
322. 2012-09-06T01:42:59 #20 0x0000000000648061 in do_handle_one_connection (thd_arg=thd_arg@entry=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1484
323. 2012-09-06T01:42:59 #21 0x0000000000648157 in handle_one_connection (arg=0x2ced230) at /media/Vone/percona-server/origin/Percona-Server/sql/sql_connect.cc:1391
324. 2012-09-06T01:42:59 #22 0x00007fd2ec097e0f in start_thread () from /lib/libpthread.so.0
325. 2012-09-06T01:42:59 #23 0x00007fd2ea70504d in clone () from /lib/libc.so.6
326. 2012-09-06T01:42:59 Test completed with failure status STATUS_SERVER_CRASHED (101)
327. 2012-09-06T01:42:59 gentest.pl exited with exit status STATUS_SERVER_CRASHED (101)
     2012-09-06T01:42:59 [2527] runall.pl will exit with exit status STATUS_SERVER_CRASHED (101)

It is crashing on ha_show_status which may be related to bug 1043620 and/or to the issue http://bugs.mysql.com/bug.php?id=60682 (which also deals with ha_show_status associated locking, also used as test case in https://bugs.launchpad.net/percona-server/+bug/1040735)

Launchpad bug id: 916205

After several runs, I conclude that this bug is already fixed.