[MDEV-8691] Multiple test failures on mips and mipsel platforms, suites federated and rpl Created: 2015-08-28  Updated: 2022-12-15  Resolved: 2022-12-15

Status: Closed
Project: MariaDB Server
Component/s: Platform Debian
Affects Version/s: 10.0.21
Fix Version/s: N/A

Type: Bug Priority: Minor
Reporter: Otto Kekäläinen Assignee: VicenČ›iu Ciorbaru
Resolution: Cannot Reproduce Votes: 0
Labels: foundation, mips, mipsel

Issue Links:
Relates
relates to MDEV-10417 Mips64el Debian - Test Failures Closed
Sprint: 10.0.22, 10.1.9-1, 10.1.9-2

 Description   

On Debian build system, architectures mips and mipsel fail to build properly due to multiple failing tests.

Failing test(s): federated.federated_partition federated.federated_transactions federated.federated_innodb multi_source.gtid multi_source.multisource rpl.rpl_auto_increment rpl.rpl_invoked_features rpl.rpl_partition_innodb

See full logs at

Builds overview (latest package at the moment is 10.0.21-2 in Debian experimental):

Examples of failures:

worker[1] mysql-test-run: WARNING: Process [mysqld.1 - pid: 22599, winpid: 22599, exit: 256] died after mysql-test-run waited 3.1 seconds for /«PKGBUILDDIR»/builddir/mysql-test/var/run/mysqld.1.pid to be created.
 
federated.federated_partition 'X,innodb_plugin' [ fail ]
 
        Test ended at 2015-08-28 03:51:35
 
 
 
CURRENT_TEST: federated.federated_partition
 
 
 
 
 
Failed to start mysqld.1
 
mysqltest failed but provided no output
 
 
 
 
 
 - saving '/«PKGBUILDDIR»/builddir/mysql-test/var/log/federated.federated_partition-X,innodb_plugin/' to '/«PKGBUILDDIR»/builddir/mysql-test/var/log/federated.federated_partition-X,innodb_plugin/'
 
 - found 'core' (0/5)
 
 
 
Trying 'dbx' to get a backtrace
 
 
 
Trying 'gdb' to get a backtrace
 
Compressed file /«PKGBUILDDIR»/builddir/mysql-test/var/log/federated.federated_partition-X,innodb_plugin/mysqld.2/data/core
 
 - found 'core' (1/5)
 
 
 
Trying 'dbx' to get a backtrace
 
 
 
Trying 'gdb' to get a backtrace
 
Compressed file /«PKGBUILDDIR»/builddir/mysql-test/var/log/federated.federated_partition-X,innodb_plugin/mysqld.1/data/core
 
***Warnings generated in error logs during shutdown after running tests: federated.federated_partition
 
 
 
150828  3:51:31 [ERROR] mysqld got signal 11 ;
 
Attempting backtrace. You can use the following information to find out
 
150828  3:51:31 [ERROR] mysqld got signal 11 ;
 
Attempting backtrace. You can use the following information to find out
 
 
 
worker[1] mysql-test-run: WARNING: Process [mysqld.1 - pid: 22647, winpid: 22647, exit: 256] died after mysql-test-run waited 2.3 seconds for /«PKGBUILDDIR»/builddir/mysql-test/var/run/mysqld.1.pid to be created.
 
federated.federated_transactions 'X,innodb_plugin' [ fail ]
 
        Test ended at 2015-08-28 03:52:06
 
 
 
CURRENT_TEST: federated.federated_partition
 
CURRENT_TEST: federated.federated_transactions
 
 
 
 
 
Failed to start mysqld.1
 
 
 
 
 
 - saving '/«PKGBUILDDIR»/builddir/mysql-test/var/log/federated.federated_transactions-X,innodb_plugin/' to '/«PKGBUILDDIR»/builddir/mysql-test/var/log/federated.federated_transactions-X,innodb_plugin/'
 
 - found 'core' (2/5)
 
 
 
Trying 'dbx' to get a backtrace
 
 
 
Trying 'gdb' to get a backtrace
 
worker[1] mysql-test-run: WARNING: Process [mysqld.1 - pid: 22677, winpid: 22677, exit: 256] died after mysql-test-run waited 3.8 seconds for /«PKGBUILDDIR»/builddir/mysql-test/var/run/mysqld.1.pid to be created.
 
Compressed file /«PKGBUILDDIR»/builddir/mysql-test/var/log/federated.federated_transactions-X,innodb_plugin/mysqld.2/data/core
 
 - found 'core' (3/5)
 
 
 
Trying 'dbx' to get a backtrace
 
 
 
Trying 'gdb' to get a backtrace
 
Compressed file /«PKGBUILDDIR»/builddir/mysql-test/var/log/federated.federated_transactions-X,innodb_plugin/mysqld.1/data/core
 
***Warnings generated in error logs during shutdown after running tests: federated.federated_transactions
 
 
 
150828  3:52:04 [ERROR] mysqld got signal 11 ;
 
Attempting backtrace. You can use the following information to find out
 
150828  3:52:04 [ERROR] mysqld got signal 11 ;
 
Attempting backtrace. You can use the following information to find out
 
 
 
federated.federated_innodb 'X,innodb_plugin' [ fail ]
 
        Test ended at 2015-08-28 03:52:36
 
 
 
CURRENT_TEST: federated.federated_partition
 
CURRENT_TEST: federated.federated_transactions
 
CURRENT_TEST: federated.federated_innodb
 
 
 
 
 
Failed to start mysqld.1
 
 
 
 
 
 - saving '/«PKGBUILDDIR»/builddir/mysql-test/var/log/federated.federated_innodb-X,innodb_plugin/' to '/«PKGBUILDDIR»/builddir/mysql-test/var/log/federated.federated_innodb-X,innodb_plugin/'
 
 - found 'core' (4/5)
 
 
 
Trying 'dbx' to get a backtrace
 
 
 
Trying 'gdb' to get a backtrace
 
Compressed file /«PKGBUILDDIR»/builddir/mysql-test/var/log/federated.federated_innodb-X,innodb_plugin/mysqld.2/data/core
 
 - found 'core' (5/5)
 
 
 
Trying 'dbx' to get a backtrace
 
 
 
Trying 'gdb' to get a backtrace
 
 - deleting it, already saved 5
 
***Warnings generated in error logs during shutdown after running tests: federated.federated_innodb
 
 
 
150828  3:52:08 [ERROR] mysqld got signal 11 ;
 
Attempting backtrace. You can use the following information to find out
 
150828  3:52:08 [ERROR] mysqld got signal 11 ;
 
Attempting backtrace. You can use the following information to find out
 
 
 
worker[1] mysql-test-run: WARNING: Process [mysqld.1 - pid: 25115, winpid: 25115, exit: 256] died after mysql-test-run waited 2.5 seconds for /«PKGBUILDDIR»/builddir/mysql-test/var/run/mysqld.1.pid to be created.
 
rpl.rpl_auto_increment 'innodb_plugin,row' [ fail ]
 
        Test ended at 2015-08-28 03:54:38
 
 
 
CURRENT_TEST: rpl.rpl_auto_increment
 
 
 
 
 
Failed to start mysqld.1
 
mysqltest failed but provided no output
 
The result from queries just before the failure was:
 
< snip >
 
DROP TABLE t_ignored1;
 
DROP TABLE t_ignored2;
 
CREATE TABLE t1 (id SMALLINT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY, data INT) ENGINE=innodb;
 
BEGIN;
 
# Set sql_mode with NO_AUTO_VALUE_ON_ZERO for allowing
 
# zero to fill the auto_increment field.
 
SET SQL_MODE=NO_AUTO_VALUE_ON_ZERO;
 
INSERT INTO t1(id,data) VALUES(0,2);
 
# Resetting sql_mode without NO_AUTO_VALUE_ON_ZERO to
 
# affect the execution of the transaction on slave.
 
SET SQL_MODE=0;
 
COMMIT;
 
SELECT * FROM t1;
 
id	data
 
0	2
 
SELECT * FROM t1;
 
id	data
 
0	2
 
DROP TABLE t1;
 
include/rpl_end.inc
 
 
 
 
 
 
 
 - saving '/«PKGBUILDDIR»/builddir/mysql-test/var/log/rpl.rpl_auto_increment-innodb_plugin,row/' to '/«PKGBUILDDIR»/builddir/mysql-test/var/log/rpl.rpl_auto_increment-innodb_plugin,row/'
 
 - found 'core' (15/5)
 
 
 
Trying 'dbx' to get a backtrace
 
 
 
Trying 'gdb' to get a backtrace
 
 - deleting it, already saved 5
 
 - found 'core' (16/5)
 
 
 
Trying 'dbx' to get a backtrace
 
 
 
Trying 'gdb' to get a backtrace
 
 - deleting it, already saved 5
 
***Warnings generated in error logs during shutdown after running tests: rpl.rpl_auto_increment
 
 
 
150828  3:54:36 [ERROR] mysqld got signal 11 ;
 
Attempting backtrace. You can use the following information to find out
 
150828  3:54:36 [ERROR] mysqld got signal 11 ;
 
Attempting backtrace. You can use the following information to find out
 
 
 
rpl.rpl_auto_increment 'row,xtradb'      [ pass ]   2712

Notes:

  • mips and mipsel logs seem identical
  • all test failures occur because mysqld crashes
  • 37 tests did succeed so the mips(el) build is not completely broken
  • this is not an regression, even 10.0.14 build logs for mips show same failures

 Comments   
Comment by Elena Stepanova [ 2015-08-30 ]

I don't think any magical fix is possible here, we'll need at least stack traces from those coredumps, and preferably also error log files.

Comment by Elena Stepanova [ 2015-09-27 ]

If I found the right link, it seems now the tests aren't running at all:
https://buildd.debian.org/status/fetch.php?pkg=mariadb-10.0&arch=mips&ver=10.0.21-3&stamp=1442810351

cd /«PKGBUILDDIR»/builddir/mysql-test && export MTR_BUILD_THREAD=auto
 
cd /«PKGBUILDDIR»/builddir/mysql-test && perl ./mysql-test-run.pl --force --testcase-timeout=30 --suite-timeout=540
 
Logging: /«PKGBUILDDIR»/mysql-test/mysql-test-run.pl  --force --testcase-timeout=30 --suite-timeout=540
 
vardir: /«PKGBUILDDIR»/builddir/mysql-test/var
 
Removing old var directory...
 
Creating var directory '/«PKGBUILDDIR»/builddir/mysql-test/var'...
 
Checking supported features...
 
Use of uninitialized value in pattern match (m//) at /«PKGBUILDDIR»/mysql-test/mysql-test-run.pl line 1883.
 
mysql-test-run: *** ERROR: Could not find version of MariaDB
 
make[5]: *** [mysql-test/CMakeFiles/test-force] Error 1
 
mysql-test/CMakeFiles/test-force.dir/build.make:57: recipe for target 'mysql-test/CMakeFiles/test-force' failed
 
make[5]: Leaving directory '/«PKGBUILDDIR»/builddir'
 
make[4]: *** [mysql-test/CMakeFiles/test-force.dir/all] Error 2
 
CMakeFiles/Makefile2:9438: recipe for target 'mysql-test/CMakeFiles/test-force.dir/all' failed
 
make[3]: *** [mysql-test/CMakeFiles/test-force.dir/rule] Error 2
 
make[4]: Leaving directory '/«PKGBUILDDIR»/builddir'
 
CMakeFiles/Makefile2:9445: recipe for target 'mysql-test/CMakeFiles/test-force.dir/rule' failed
 
make[3]: Leaving directory '/«PKGBUILDDIR»/builddir'
 
make[2]: *** [test-force] Error 2
 
Makefile:2860: recipe for target 'test-force' failed
 
make[2]: Leaving directory '/«PKGBUILDDIR»/builddir'
 
make[1]: *** [override_dh_auto_test] Error 2
 
debian/rules:105: recipe for target 'override_dh_auto_test' failed
 
make[1]: Leaving directory '/«PKGBUILDDIR»'
 
make: *** [build-arch] Error 2
 
dpkg-buildpackage: error: debian/rules build-arch gave error exit status 2
 
debian/rules:156: recipe for target 'build-arch' failed

Comment by Alexander Barkov [ 2015-10-22 ]

Non of the mentioned tests fail on this Scaleway virtual machine
Linux scw-4b87ad 3.2.34-30 #17 SMP Mon Apr 13 15:53:45 UTC 2015 armv7l armv7l armv7l GNU/Linux

bar@scw-4b87ad:~/maria-git/server/builddir/mysql-test$ ./mtr --force federated.federated_partition federated.federated_transactions federated.federated_innodb multi_source.gtid multi_source.multisource rpl.rpl_auto_increment rpl.rpl_invoked_features rpl.rpl_partition_innodb
 
Logging: /home/bar/maria-git/server/mysql-test/mysql-test-run.pl  --force federated.federated_partition federated.federated_transactions federated.federated_innodb multi_source.gtid multi_source.multisource rpl.rpl_auto_increment rpl.rpl_invoked_features rpl.rpl_partition_innodb
 
vardir: /home/bar/maria-git/server/builddir/mysql-test/var
 
Checking leftover processes...
 
 - found old pid 4412 in 'mysqld.1.pid', killing it...
 
   process did not exist!
 
Removing old var directory...
 
Creating var directory '/home/bar/maria-git/server/builddir/mysql-test/var'...
 
Checking supported features...
 
MariaDB Version 10.0.22-MariaDB-debug
 
 - SSL connections supported
 
 - binaries are debug compiled
 
Collecting tests...
 
Installing system database...
 
 
 
==============================================================================
 
 
 
TEST                                      RESULT   TIME (ms) or COMMENT
 
--------------------------------------------------------------------------
 
 
 
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
 
federated.federated_partition 'X,innodb_plugin' [ pass ]    729
 
federated.federated_transactions 'X,innodb_plugin' [ pass ]    456
 
federated.federated_innodb 'X,innodb_plugin' [ pass ]    461
 
federated.federated_innodb 'innodb_plugin,old' [ pass ]    888
 
federated.federated_partition 'X,xtradb' [ pass ]    262
 
federated.federated_transactions 'X,xtradb' [ pass ]    723
 
federated.federated_innodb 'X,xtradb'    [ pass ]    488
 
federated.federated_innodb 'old,xtradb'  [ pass ]    832
 
multi_source.gtid 'innodb_plugin'        [ pass ]   3773
 
multi_source.gtid 'xtradb'               [ pass ]   3729
 
multi_source.multisource 'innodb_plugin' [ pass ]   3744
 
multi_source.multisource 'xtradb'        [ pass ]   3373
 
rpl.rpl_auto_increment 'innodb_plugin,mix' [ pass ]   5202
 
rpl.rpl_auto_increment 'mix,xtradb'      [ pass ]   4756
 
rpl.rpl_auto_increment 'innodb_plugin,row' [ pass ]   5043
 
rpl.rpl_auto_increment 'row,xtradb'      [ pass ]   4721
 
rpl.rpl_auto_increment 'innodb_plugin,stmt' [ pass ]   5004
 
rpl.rpl_auto_increment 'stmt,xtradb'     [ pass ]   4728
 
rpl.rpl_invoked_features 'innodb_plugin,mix' [ pass ]   6030
 
rpl.rpl_invoked_features 'mix,xtradb'    [ pass ]   4117
 
rpl.rpl_partition_innodb 'innodb_plugin,mix' [ pass ]   9397
 
rpl.rpl_partition_innodb 'mix,xtradb'    [ pass ]   8901
 
rpl.rpl_invoked_features 'innodb_plugin,row' [ pass ]   4023
 
rpl.rpl_invoked_features 'row,xtradb'    [ pass ]   3851
 
rpl.rpl_partition_innodb 'innodb_plugin,row' [ pass ]   9722
 
rpl.rpl_partition_innodb 'row,xtradb'    [ pass ]   9531
 
rpl.rpl_invoked_features 'innodb_plugin,stmt' [ pass ]   5062
 
rpl.rpl_invoked_features 'stmt,xtradb'   [ pass ]   4276
 
rpl.rpl_partition_innodb 'innodb_plugin,stmt' [ pass ]   9595
 
rpl.rpl_partition_innodb 'stmt,xtradb'   [ pass ]   8831
 
--------------------------------------------------------------------------
 
The servers were restarted 29 times
 
Spent 132.248 of 327 seconds executing testcases
 
 
 
Completed: All 30 tests were successful.

Comment by Otto Kekäläinen [ 2016-01-26 ]

Hello bar!

I managed to get the Debian builds so that if mysqld crashes during test suite, a stack trace is displayed automatically. The log https://buildd.debian.org/status/fetch.php?pkg=mariadb-10.0&arch=mips64el&ver=10.0.23-2&stamp=1453705657 now shows:

rpl.rpl_semi_sync 'innodb_plugin,mix'    w4 [ fail ]
 
        Test ended at 2016-01-25 07:04:50
 
 
 
CURRENT_TEST: rpl.rpl_innodb_bug30888
 
CURRENT_TEST: rpl.rpl_innodb_bug30888
 
CURRENT_TEST: rpl.rpl_semi_sync
 
 
 
 
 
Failed to start mysqld.1
 
 
 
 
 
 - saving '/«PKGBUILDDIR»/builddir/mysql-test/var/4/log/rpl.rpl_semi_sync-innodb_plugin,mix/' to '/«PKGBUILDDIR»/builddir/mysql-test/var/log/rpl.rpl_semi_sync-innodb_plugin,mix/'
 
 - found 'core' (14/5)
 
 
 
Trying 'dbx' to get a backtrace
 
worker[2] mysql-test-run: WARNING: Test reserved for w3 picked up by w2
 
 
 
Trying 'gdb' to get a backtrace
 
Core generated by '/«PKGBUILDDIR»/builddir/sql/mysqld'
 
worker[3] mysql-test-run: WARNING: Process [mysqld.1 - pid: 5381, winpid: 5381, exit: 256] died after mysql-test-run waited 6.4 seconds for /«PKGBUILDDIR»/builddir/mysql-test/var/3/run/mysqld.1.pid to be created.
 
worker[2] mysql-test-run: WARNING: Process [mysqld.1 - pid: 5383, winpid: 5383, exit: 256] died after mysql-test-run waited 6.6 seconds for /«PKGBUILDDIR»/builddir/mysql-test/var/2/run/mysqld.1.pid to be created.
 
Output from gdb follows. The first stack trace is from the failing thread.
 
The following stack traces are from all threads (so the failing one is
 
duplicated).
 
--------------------------
 
BFD: Warning: /«PKGBUILDDIR»/builddir/mysql-test/var/log/rpl.rpl_semi_sync-innodb_plugin,mix/mysqld.2/data/core is truncated: expected core file size >= 196067328, found: 4702208.
 
[New LWP 5068]
 
[New LWP 5096]
 
[New LWP 5097]
 
[New LWP 5102]
 
[New LWP 5101]
 
[New LWP 5098]
 
[New LWP 5100]
 
[New LWP 5099]
 
Cannot access memory at address 0xfff72df838
 
Cannot access memory at address 0xfff72df830
 
Core was generated by `/«PKGBUILDDIR»/builddir/sql/mysqld --defaults-'.
 
Program terminated with signal SIGSEGV, Segmentation fault.
 
#0  0x000000fff727f210 in ?? ()
 
[Current thread is 1 (LWP 5068)]
 
#0  0x000000fff727f210 in ?? ()
 
#1  0x0000000120a8ada8 in my_write_core (sig=<optimized out>) at /«PKGBUILDDIR»/mysys/stacktrace.c:457
 
Backtrace stopped: Cannot access memory at address 0xfffffd9518
 
 
 
Thread 8 (LWP 5099):
 
#0  0x000000fff7258a8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 7 (LWP 5100):
 
#0  0x000000fff7258a8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 6 (LWP 5098):
 
#0  0x000000fff7258a8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 5 (LWP 5101):
 
#0  0x000000fff7258a8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 4 (LWP 5102):
 
#0  0x000000fff7258a8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 3 (LWP 5097):
 
#0  0x000000fff7258a8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 2 (LWP 5096):
 
#0  0x000000fff727dcc4 in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 1 (LWP 5068):
 
#0  0x000000fff727f210 in ?? ()
 
#1  0x0000000120a8ada8 in my_write_core (sig=<optimized out>) at /«PKGBUILDDIR»/mysys/stacktrace.c:457
 
Backtrace stopped: Cannot access memory at address 0xfffffd9518
 
 
 
 - deleting it, already saved 5
 
 - found 'core' (15/5)
 
 
 
Trying 'dbx' to get a backtrace
 
 
 
Trying 'gdb' to get a backtrace
 
Core generated by '/«PKGBUILDDIR»/builddir/sql/mysqld'
 
Output from gdb follows. The first stack trace is from the failing thread.
 
The following stack traces are from all threads (so the failing one is
 
duplicated).
 
--------------------------
 
[New LWP 5026]
 
[New LWP 5058]
 
[New LWP 5057]
 
[New LWP 5047]
 
[New LWP 5059]
 
[New LWP 5060]
 
[New LWP 5061]
 
[New LWP 5062]
 
[Thread debugging using libthread_db enabled]
 
Using host libthread_db library "/lib/mips64el-linux-gnuabi64/libthread_db.so.1".
 
Core was generated by `/«PKGBUILDDIR»/builddir/sql/mysqld --defaults-'.
 
Program terminated with signal SIGSEGV, Segmentation fault.
 
#0  0x000000ffed5b7210 in pthread_kill () from /lib/mips64el-linux-gnuabi64/libpthread.so.0
 
[Current thread is 1 (LWP 5026)]
 
#0  0x000000ffed5b7210 in pthread_kill () from /lib/mips64el-linux-gnuabi64/libpthread.so.0
 
#1  0x0000000120a8ada8 in my_write_core (sig=<optimized out>) at /«PKGBUILDDIR»/mysys/stacktrace.c:457
 
#2  0x000000012049e838 in handle_fatal_signal (sig=<optimized out>) at /«PKGBUILDDIR»/sql/signal_handler.cc:262
 
#3  <signal handler called>
 
#4  0x000000ffed340e80 in std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*) () from /usr/lib/mips64el-linux-gnuabi64/libstdc++.so.6
 
#5  0x00000001207cf878 in std::__cxx11::list<unsigned char*, std::allocator<unsigned char*> >::_M_insert (__x=<synthetic pointer>, __position=..., this=0x118) at /usr/include/c++/5/bits/stl_list.h:1755
 
#6  std::__cxx11::list<unsigned char*, std::allocator<unsigned char*> >::push_back (__x=<synthetic pointer>, this=0x118) at /usr/include/c++/5/bits/stl_list.h:1089
 
#7  recv_dblwr_t::add (this=0x118, page=0xffe1804000 "\377\060~$") at /«PKGBUILDDIR»/storage/xtradb/log/log0recv.cc:4077
 
#8  0x00000001208df7f8 in buf_dblwr_init_or_load_pages (file=<optimized out>, path=0xffecf83029 <srv_read_only_mode> "", load_corrupt_pages=<optimized out>) at /«PKGBUILDDIR»/storage/xtradb/buf/buf0dblwr.cc:457
 
#9  0x000000ffecedc2bc in open_or_create_data_files (sum_of_new_sizes=<synthetic pointer>, max_flushed_lsn=0xfffff85430, min_flushed_lsn=0xfffff85428, create_new_db=<synthetic pointer>) at /«PKGBUILDDIR»/storage/innobase/srv/srv0start.cc:1033
 
#10 innobase_start_or_create_for_mysql () at /«PKGBUILDDIR»/storage/innobase/srv/srv0start.cc:2105
 
#11 0x000000ffecde8028 in innobase_init (p=<optimized out>) at /«PKGBUILDDIR»/storage/innobase/handler/ha_innodb.cc:3400
 
#12 0x00000001204a17bc in ha_initialize_handlerton (plugin=0xffec4ac9f0) at /«PKGBUILDDIR»/sql/handler.cc:501
 
#13 0x00000001202fcafc in plugin_initialize (tmp_root=0xfffff8d630, tmp_root@entry=0x0, plugin=plugin@entry=0xffec4ac9f0, argc=0x120fdf680 <remaining_argc>, argc@entry=0x6, argv=0xffec474c58, argv@entry=0xfd8, options_only=<optimized out>) at /«PKGBUILDDIR»/sql/sql_plugin.cc:1379
 
#14 0x00000001202fd7b0 in plugin_init (argc=0x6, argv=0xfd8, flags=<optimized out>) at /«PKGBUILDDIR»/sql/sql_plugin.cc:1659
 
#15 0x00000001202347ec in init_server_components () at /«PKGBUILDDIR»/sql/mysqld.cc:4850
 
#16 0x000000012023b83c in mysqld_main (argc=255, argv=0xff00000000) at /«PKGBUILDDIR»/sql/mysqld.cc:5445
 
#17 0x000000012021cf18 in main (argc=<optimized out>, argv=<optimized out>) at /«PKGBUILDDIR»/sql/main.cc:25
 
 
 
Thread 8 (LWP 5062):
 
warning: GDB can't find the start of the function at 0xffed590a8c.
 
 
 
    GDB is unable to find the start of the function at 0xffed590a8c
 
and thus can't determine the size of that function's stack frame.
 
This means that GDB may be unable to access that stack frame, or
 
the frames below it.
 
    This problem is most likely caused by an invalid program counter or
 
stack pointer.
 
    However, if you think GDB should simply search farther back
 
from 0xffed590a8c for code which looks like the beginning of a
 
function, you can increase the range of the search using the `set
 
heuristic-fence-post' command.
 
#0  0x000000ffed590a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 7 (LWP 5061):
 
warning: GDB can't find the start of the function at 0xffed590a8c.
 
#0  0x000000ffed590a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 6 (LWP 5060):
 
warning: GDB can't find the start of the function at 0xffed590a8c.
 
#0  0x000000ffed590a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 5 (LWP 5059):
 
warning: GDB can't find the start of the function at 0xffed590a8c.
 
#0  0x000000ffed590a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 4 (LWP 5047):
 
#0  0x000000ffed5b5cc4 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/mips64el-linux-gnuabi64/libpthread.so.0
 
#1  0x0000000120699a7c in inline_mysql_cond_timedwait (src_file=0x120ba9a40 "/«PKGBUILDDIR»/storage/maria/ma_servicethread.c", src_line=119, abstime=0xffe67fe7d0, mutex=0x1217fea58 <LOCK_checkpoint>, that=<optimized out>) at /«PKGBUILDDIR»/include/mysql/psi/mysql_thread.h:1191
 
#2  my_service_thread_sleep (control=0x120f4b540 <checkpoint_control>, sleep_time=<optimized out>) at /«PKGBUILDDIR»/storage/maria/ma_servicethread.c:118
 
#3  0x000000012068eb78 in ma_checkpoint_background (arg=0xffed5add74 <start_thread+212>, arg@entry=0x1e) at /«PKGBUILDDIR»/storage/maria/ma_checkpoint.c:709
 
#4  0x00000001207367b4 in pfs_spawn_thread (arg=<optimized out>) at /«PKGBUILDDIR»/storage/perfschema/pfs.cc:1860
 
#5  0x000000ffed5add74 in start_thread () from /lib/mips64el-linux-gnuabi64/libpthread.so.0
 
Backtrace stopped: frame did not save the PC
 
 
 
Thread 3 (LWP 5057):
 
warning: GDB can't find the start of the function at 0xffed590a8c.
 
#0  0x000000ffed590a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 2 (LWP 5058):
 
warning: GDB can't find the start of the function at 0xffed590a8c.
 
#0  0x000000ffed590a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 1 (LWP 5026):
 
#0  0x000000ffed5b7210 in pthread_kill () from /lib/mips64el-linux-gnuabi64/libpthread.so.0
 
#1  0x0000000120a8ada8 in my_write_core (sig=<optimized out>) at /«PKGBUILDDIR»/mysys/stacktrace.c:457
 
#2  0x000000012049e838 in handle_fatal_signal (sig=<optimized out>) at /«PKGBUILDDIR»/sql/signal_handler.cc:262
 
#3  <signal handler called>
 
#4  0x000000ffed340e80 in std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*) () from /usr/lib/mips64el-linux-gnuabi64/libstdc++.so.6
 
#5  0x00000001207cf878 in std::__cxx11::list<unsigned char*, std::allocator<unsigned char*> >::_M_insert (__x=<synthetic pointer>, __position=..., this=0x118) at /usr/include/c++/5/bits/stl_list.h:1755
 
#6  std::__cxx11::list<unsigned char*, std::allocator<unsigned char*> >::push_back (__x=<synthetic pointer>, this=0x118) at /usr/include/c++/5/bits/stl_list.h:1089
 
#7  recv_dblwr_t::add (this=0x118, page=0xffe1804000 "\377\060~$") at /«PKGBUILDDIR»/storage/xtradb/log/log0recv.cc:4077
 
#8  0x00000001208df7f8 in buf_dblwr_init_or_load_pages (file=<optimized out>, path=0xffecf83029 <srv_read_only_mode> "", load_corrupt_pages=<optimized out>) at /«PKGBUILDDIR»/storage/xtradb/buf/buf0dblwr.cc:457
 
#9  0x000000ffecedc2bc in open_or_create_data_files (sum_of_new_sizes=<synthetic pointer>, max_flushed_lsn=0xfffff85430, min_flushed_lsn=0xfffff85428, create_new_db=<synthetic pointer>) at /«PKGBUILDDIR»/storage/innobase/srv/srv0start.cc:1033
 
#10 innobase_start_or_create_for_mysql () at /«PKGBUILDDIR»/storage/innobase/srv/srv0start.cc:2105
 
#11 0x000000ffecde8028 in innobase_init (p=<optimized out>) at /«PKGBUILDDIR»/storage/innobase/handler/ha_innodb.cc:3400
 
#12 0x00000001204a17bc in ha_initialize_handlerton (plugin=0xffec4ac9f0) at /«PKGBUILDDIR»/sql/handler.cc:501
 
#13 0x00000001202fcafc in plugin_initialize (tmp_root=0xfffff8d630, tmp_root@entry=0x0, plugin=plugin@entry=0xffec4ac9f0, argc=0x120fdf680 <remaining_argc>, argc@entry=0x6, argv=0xffec474c58, argv@entry=0xfd8, options_only=<optimized out>) at /«PKGBUILDDIR»/sql/sql_plugin.cc:1379
 
#14 0x00000001202fd7b0 in plugin_init (argc=0x6, argv=0xfd8, flags=<optimized out>) at /«PKGBUILDDIR»/sql/sql_plugin.cc:1659
 
#15 0x00000001202347ec in init_server_components () at /«PKGBUILDDIR»/sql/mysqld.cc:4850
 
#16 0x000000012023b83c in mysqld_main (argc=255, argv=0xff00000000) at /«PKGBUILDDIR»/sql/mysqld.cc:5445
 
#17 0x000000012021cf18 in main (argc=<optimized out>, argv=<optimized out>) at /«PKGBUILDDIR»/sql/main.cc:25
 
 
 
 - deleting it, already saved 5
 
 
 
Retrying test rpl.rpl_semi_sync, attempt(2/3)...
 
 
 
***Warnings generated in error logs during shutdown after running tests: rpl.rpl_auto_increment_update_failure
 
 
 
160125  7:02:29 [ERROR] mysqld got signal 11 ;
 
Attempting backtrace. You can use the following information to find out
 
160125  7:02:33 [ERROR] mysqld got signal 11 ;
 
Attempting backtrace. You can use the following information to find out
 
 
 
***Warnings generated in error logs during shutdown after running tests: rpl.rpl_semi_sync_event
 
 
 
160125  7:02:30 [ERROR] mysqld got signal 11 ;
 
Attempting backtrace. You can use the following information to find out
 
160125  7:02:37 [ERROR] mysqld got signal 11 ;
 
Attempting backtrace. You can use the following information to find out
 
 
 
rpl.rpl_deadlock_innodb 'innodb_plugin,mix' w2 [ fail ]
 
        Test ended at 2016-01-25 07:05:24
 
 
 
CURRENT_TEST: rpl.rpl_mdev6020
 
CURRENT_TEST: rpl.rpl_mdev6020
 
CURRENT_TEST: rpl.rpl_deadlock_innodb
 
 
 
 
 
Failed to start mysqld.1
 
 
 
 
 
 - saving '/«PKGBUILDDIR»/builddir/mysql-test/var/2/log/rpl.rpl_deadlock_innodb-innodb_plugin,mix/' to '/«PKGBUILDDIR»/builddir/mysql-test/var/log/rpl.rpl_deadlock_innodb-innodb_plugin,mix/'
 
 - found 'core' (16/5)
 
 
 
Trying 'dbx' to get a backtrace
 
 
 
Trying 'gdb' to get a backtrace
 
Core generated by '/«PKGBUILDDIR»/builddir/sql/mysqld'
 
worker[4] mysql-test-run: WARNING: Process [mysqld.1 - pid: 5471, winpid: 5471, exit: 256] died after mysql-test-run waited 8 seconds for /«PKGBUILDDIR»/builddir/mysql-test/var/4/run/mysqld.1.pid to be created.
 
Output from gdb follows. The first stack trace is from the failing thread.
 
The following stack traces are from all threads (so the failing one is
 
duplicated).
 
--------------------------
 
BFD: Warning: /«PKGBUILDDIR»/builddir/mysql-test/var/log/rpl.rpl_deadlock_innodb-innodb_plugin,mix/mysqld.2/data/core is truncated: expected core file size >= 196001792, found: 117080064.
 
[New LWP 5126]
 
[New LWP 5172]
 
[New LWP 5175]
 
[New LWP 5177]
 
[New LWP 5179]
 
[New LWP 5174]
 
[New LWP 5176]
 
[New LWP 5178]
 
Cannot access memory at address 0xfff3193838
 
Cannot access memory at address 0xfff3193830
 
Core was generated by `/«PKGBUILDDIR»/builddir/sql/mysqld --defaults-'.
 
Program terminated with signal SIGSEGV, Segmentation fault.
 
#0  0x000000fff3133210 in ?? ()
 
[Current thread is 1 (LWP 5126)]
 
#0  0x000000fff3133210 in ?? ()
 
#1  0x0000000120a8ada8 in my_write_core (sig=<optimized out>) at /«PKGBUILDDIR»/mysys/stacktrace.c:457
 
Backtrace stopped: Cannot access memory at address 0xffffd64e98
 
 
 
Thread 8 (LWP 5178):
 
#0  0x000000fff310ca8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 7 (LWP 5176):
 
#0  0x000000fff310ca8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 6 (LWP 5174):
 
#0  0x000000fff310ca8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 5 (LWP 5179):
 
#0  0x000000fff310ca8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 4 (LWP 5177):
 
#0  0x000000fff310ca8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 3 (LWP 5175):
 
#0  0x000000fff310ca8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 2 (LWP 5172):
 
#0  0x000000fff3131cc4 in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 1 (LWP 5126):
 
#0  0x000000fff3133210 in ?? ()
 
#1  0x0000000120a8ada8 in my_write_core (sig=<optimized out>) at /«PKGBUILDDIR»/mysys/stacktrace.c:457
 
Backtrace stopped: Cannot access memory at address 0xffffd64e98
 
 
 
 - deleting it, already saved 5
 
 - found 'core' (17/5)
 
 
 
Trying 'dbx' to get a backtrace
 
 
 
Trying 'gdb' to get a backtrace
 
Core generated by '/«PKGBUILDDIR»/builddir/sql/mysqld'
 
Output from gdb follows. The first stack trace is from the failing thread.
 
The following stack traces are from all threads (so the failing one is
 
duplicated).
 
--------------------------
 
[New LWP 5118]
 
[New LWP 5129]
 
[New LWP 5138]
 
[New LWP 5140]
 
[New LWP 5142]
 
[New LWP 5141]
 
[New LWP 5139]
 
[New LWP 5143]
 
[Thread debugging using libthread_db enabled]
 
Using host libthread_db library "/lib/mips64el-linux-gnuabi64/libthread_db.so.1".
 
Core was generated by `/«PKGBUILDDIR»/builddir/sql/mysqld --defaults-'.
 
Program terminated with signal SIGSEGV, Segmentation fault.
 
#0  0x000000fff4e3b210 in pthread_kill () from /lib/mips64el-linux-gnuabi64/libpthread.so.0
 
[Current thread is 1 (LWP 5118)]
 
#0  0x000000fff4e3b210 in pthread_kill () from /lib/mips64el-linux-gnuabi64/libpthread.so.0
 
#1  0x0000000120a8ada8 in my_write_core (sig=<optimized out>) at /«PKGBUILDDIR»/mysys/stacktrace.c:457
 
#2  0x000000012049e838 in handle_fatal_signal (sig=<optimized out>) at /«PKGBUILDDIR»/sql/signal_handler.cc:262
 
#3  <signal handler called>
 
#4  0x000000fff4bc4e80 in std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*) () from /usr/lib/mips64el-linux-gnuabi64/libstdc++.so.6
 
#5  0x00000001207cf878 in std::__cxx11::list<unsigned char*, std::allocator<unsigned char*> >::_M_insert (__x=<synthetic pointer>, __position=..., this=0x118) at /usr/include/c++/5/bits/stl_list.h:1755
 
#6  std::__cxx11::list<unsigned char*, std::allocator<unsigned char*> >::push_back (__x=<synthetic pointer>, this=0x118) at /usr/include/c++/5/bits/stl_list.h:1089
 
#7  recv_dblwr_t::add (this=0x118, page=0xffe9004000 "\377\060~$") at /«PKGBUILDDIR»/storage/xtradb/log/log0recv.cc:4077
 
#8  0x00000001208df7f8 in buf_dblwr_init_or_load_pages (file=<optimized out>, path=0xffedbef029 <srv_read_only_mode> "", load_corrupt_pages=<optimized out>) at /«PKGBUILDDIR»/storage/xtradb/buf/buf0dblwr.cc:457
 
#9  0x000000ffedb482bc in open_or_create_data_files (sum_of_new_sizes=<synthetic pointer>, max_flushed_lsn=0xffffa951e0, min_flushed_lsn=0xffffa951d8, create_new_db=<synthetic pointer>) at /«PKGBUILDDIR»/storage/innobase/srv/srv0start.cc:1033
 
#10 innobase_start_or_create_for_mysql () at /«PKGBUILDDIR»/storage/innobase/srv/srv0start.cc:2105
 
#11 0x000000ffeda54028 in innobase_init (p=<optimized out>) at /«PKGBUILDDIR»/storage/innobase/handler/ha_innodb.cc:3400
 
#12 0x00000001204a17bc in ha_initialize_handlerton (plugin=0xfff40b49f0) at /«PKGBUILDDIR»/sql/handler.cc:501
 
#13 0x00000001202fcafc in plugin_initialize (tmp_root=0xffffa9d3d0, tmp_root@entry=0x0, plugin=plugin@entry=0xfff40b49f0, argc=0x120fdf680 <remaining_argc>, argc@entry=0x6, argv=0xfff407cc58, argv@entry=0xfd8, options_only=<optimized out>) at /«PKGBUILDDIR»/sql/sql_plugin.cc:1379
 
#14 0x00000001202fd7b0 in plugin_init (argc=0x6, argv=0xfd8, flags=<optimized out>) at /«PKGBUILDDIR»/sql/sql_plugin.cc:1659
 
#15 0x00000001202347ec in init_server_components () at /«PKGBUILDDIR»/sql/mysqld.cc:4850
 
#16 0x000000012023b83c in mysqld_main (argc=255, argv=0xff00000000) at /«PKGBUILDDIR»/sql/mysqld.cc:5445
 
#17 0x000000012021cf18 in main (argc=<optimized out>, argv=<optimized out>) at /«PKGBUILDDIR»/sql/main.cc:25
 
 
 
Thread 8 (LWP 5143):
 
warning: GDB can't find the start of the function at 0xfff4e14a8c.
 
 
 
    GDB is unable to find the start of the function at 0xfff4e14a8c
 
and thus can't determine the size of that function's stack frame.
 
This means that GDB may be unable to access that stack frame, or
 
the frames below it.
 
    This problem is most likely caused by an invalid program counter or
 
stack pointer.
 
    However, if you think GDB should simply search farther back
 
from 0xfff4e14a8c for code which looks like the beginning of a
 
function, you can increase the range of the search using the `set
 
heuristic-fence-post' command.
 
#0  0x000000fff4e14a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 7 (LWP 5139):
 
warning: GDB can't find the start of the function at 0xfff4e14a8c.
 
#0  0x000000fff4e14a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 6 (LWP 5141):
 
warning: GDB can't find the start of the function at 0xfff4e14a8c.
 
#0  0x000000fff4e14a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 5 (LWP 5142):
 
warning: GDB can't find the start of the function at 0xfff4e14a8c.
 
#0  0x000000fff4e14a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 4 (LWP 5140):
 
warning: GDB can't find the start of the function at 0xfff4e14a8c.
 
#0  0x000000fff4e14a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 3 (LWP 5138):
 
warning: GDB can't find the start of the function at 0xfff4e14a8c.
 
#0  0x000000fff4e14a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 2 (LWP 5129):
 
#0  0x000000fff4e39cc4 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/mips64el-linux-gnuabi64/libpthread.so.0
 
#1  0x0000000120699a7c in inline_mysql_cond_timedwait (src_file=0x120ba9a40 "/«PKGBUILDDIR»/storage/maria/ma_servicethread.c", src_line=119, abstime=0xffee3fe7d0, mutex=0x1217fea58 <LOCK_checkpoint>, that=<optimized out>) at /«PKGBUILDDIR»/include/mysql/psi/mysql_thread.h:1191
 
#2  my_service_thread_sleep (control=0x120f4b540 <checkpoint_control>, sleep_time=<optimized out>) at /«PKGBUILDDIR»/storage/maria/ma_servicethread.c:118
 
#3  0x000000012068eb78 in ma_checkpoint_background (arg=0xfff4e31d74 <start_thread+212>, arg@entry=0x1e) at /«PKGBUILDDIR»/storage/maria/ma_checkpoint.c:709
 
#4  0x00000001207367b4 in pfs_spawn_thread (arg=<optimized out>) at /«PKGBUILDDIR»/storage/perfschema/pfs.cc:1860
 
#5  0x000000fff4e31d74 in start_thread () from /lib/mips64el-linux-gnuabi64/libpthread.so.0
 
Backtrace stopped: frame did not save the PC
 
 
 
Thread 1 (LWP 5118):
 
#0  0x000000fff4e3b210 in pthread_kill () from /lib/mips64el-linux-gnuabi64/libpthread.so.0
 
#1  0x0000000120a8ada8 in my_write_core (sig=<optimized out>) at /«PKGBUILDDIR»/mysys/stacktrace.c:457
 
#2  0x000000012049e838 in handle_fatal_signal (sig=<optimized out>) at /«PKGBUILDDIR»/sql/signal_handler.cc:262
 
#3  <signal handler called>
 
#4  0x000000fff4bc4e80 in std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*) () from /usr/lib/mips64el-linux-gnuabi64/libstdc++.so.6
 
#5  0x00000001207cf878 in std::__cxx11::list<unsigned char*, std::allocator<unsigned char*> >::_M_insert (__x=<synthetic pointer>, __position=..., this=0x118) at /usr/include/c++/5/bits/stl_list.h:1755
 
#6  std::__cxx11::list<unsigned char*, std::allocator<unsigned char*> >::push_back (__x=<synthetic pointer>, this=0x118) at /usr/include/c++/5/bits/stl_list.h:1089
 
#7  recv_dblwr_t::add (this=0x118, page=0xffe9004000 "\377\060~$") at /«PKGBUILDDIR»/storage/xtradb/log/log0recv.cc:4077
 
#8  0x00000001208df7f8 in buf_dblwr_init_or_load_pages (file=<optimized out>, path=0xffedbef029 <srv_read_only_mode> "", load_corrupt_pages=<optimized out>) at /«PKGBUILDDIR»/storage/xtradb/buf/buf0dblwr.cc:457
 
#9  0x000000ffedb482bc in open_or_create_data_files (sum_of_new_sizes=<synthetic pointer>, max_flushed_lsn=0xffffa951e0, min_flushed_lsn=0xffffa951d8, create_new_db=<synthetic pointer>) at /«PKGBUILDDIR»/storage/innobase/srv/srv0start.cc:1033
 
#10 innobase_start_or_create_for_mysql () at /«PKGBUILDDIR»/storage/innobase/srv/srv0start.cc:2105
 
#11 0x000000ffeda54028 in innobase_init (p=<optimized out>) at /«PKGBUILDDIR»/storage/innobase/handler/ha_innodb.cc:3400
 
#12 0x00000001204a17bc in ha_initialize_handlerton (plugin=0xfff40b49f0) at /«PKGBUILDDIR»/sql/handler.cc:501
 
#13 0x00000001202fcafc in plugin_initialize (tmp_root=0xffffa9d3d0, tmp_root@entry=0x0, plugin=plugin@entry=0xfff40b49f0, argc=0x120fdf680 <remaining_argc>, argc@entry=0x6, argv=0xfff407cc58, argv@entry=0xfd8, options_only=<optimized out>) at /«PKGBUILDDIR»/sql/sql_plugin.cc:1379
 
#14 0x00000001202fd7b0 in plugin_init (argc=0x6, argv=0xfd8, flags=<optimized out>) at /«PKGBUILDDIR»/sql/sql_plugin.cc:1659
 
#15 0x00000001202347ec in init_server_components () at /«PKGBUILDDIR»/sql/mysqld.cc:4850
 
#16 0x000000012023b83c in mysqld_main (argc=255, argv=0xff00000000) at /«PKGBUILDDIR»/sql/mysqld.cc:5445
 
#17 0x000000012021cf18 in main (argc=<optimized out>, argv=<optimized out>) at /«PKGBUILDDIR»/sql/main.cc:25
 
 
 
 - deleting it, already saved 5
 
 
 
Retrying test rpl.rpl_deadlock_innodb, attempt(2/3)...
 
 
 
***Warnings generated in error logs during shutdown after running tests: rpl.rpl_semi_sync
 
 
 
160125  7:02:30 [ERROR] mysqld got signal 11 ;
 
Attempting backtrace. You can use the following information to find out
 
160125  7:02:40 [ERROR] mysqld got signal 11 ;
 
Attempting backtrace. You can use the following information to find out
 
 
 
rpl.rpl_mdev6386 'innodb_plugin,mix'     w1 [ fail ]
 
        Test ended at 2016-01-25 07:06:01
 
 
 
CURRENT_TEST: rpl.rpl_partition_innodb
 
CURRENT_TEST: rpl.rpl_partition_innodb
 
CURRENT_TEST: rpl.rpl_auto_increment_update_failure
 
CURRENT_TEST: rpl.rpl_mdev6386
 
 
 
 
 
Failed to start mysqld.1
 
 
 
 
 
 - saving '/«PKGBUILDDIR»/builddir/mysql-test/var/1/log/rpl.rpl_mdev6386-innodb_plugin,mix/' to '/«PKGBUILDDIR»/builddir/mysql-test/var/log/rpl.rpl_mdev6386-innodb_plugin,mix/'
 
 - found 'core' (18/5)
 
 
 
Trying 'dbx' to get a backtrace
 
 
 
Trying 'gdb' to get a backtrace
 
worker[4] mysql-test-run: WARNING: Test reserved for w2 picked up by w4
 
Core generated by '/«PKGBUILDDIR»/builddir/sql/mysqld'
 
worker[2] mysql-test-run: WARNING: Process [mysqld.1 - pid: 5664, winpid: 5664, exit: 256] died after mysql-test-run waited 15.9 seconds for /«PKGBUILDDIR»/builddir/mysql-test/var/2/run/mysqld.1.pid to be created.
 
worker[4] mysql-test-run: WARNING: Process [mysqld.1 - pid: 5665, winpid: 5665, exit: 256] died after mysql-test-run waited 15.8 seconds for /«PKGBUILDDIR»/builddir/mysql-test/var/4/run/mysqld.1.pid to be created.
 
Output from gdb follows. The first stack trace is from the failing thread.
 
The following stack traces are from all threads (so the failing one is
 
duplicated).
 
--------------------------
 
BFD: Warning: /«PKGBUILDDIR»/builddir/mysql-test/var/log/rpl.rpl_mdev6386-innodb_plugin,mix/mysqld.2/data/core is truncated: expected core file size >= 196001792, found: 163905536.
 
[New LWP 5127]
 
[New LWP 5184]
 
[New LWP 5183]
 
[New LWP 5173]
 
[New LWP 5182]
 
[New LWP 5186]
 
[New LWP 5181]
 
[New LWP 5185]
 
Cannot access memory at address 0xfff776b838
 
Cannot access memory at address 0xfff776b830
 
Core was generated by `/«PKGBUILDDIR»/builddir/sql/mysqld --defaults-'.
 
Program terminated with signal SIGSEGV, Segmentation fault.
 
#0  0x000000fff770b210 in ?? ()
 
[Current thread is 1 (LWP 5127)]
 
#0  0x000000fff770b210 in ?? ()
 
#1  0x0000000120a8ada8 in my_write_core (sig=<optimized out>) at /«PKGBUILDDIR»/mysys/stacktrace.c:457
 
Backtrace stopped: Cannot access memory at address 0xffffa55b48
 
 
 
Thread 8 (LWP 5185):
 
#0  0x000000fff76e4a8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 7 (LWP 5181):
 
#0  0x000000fff76e4a8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 6 (LWP 5186):
 
#0  0x000000fff76e4a8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 5 (LWP 5182):
 
#0  0x000000fff76e4a8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 4 (LWP 5173):
 
#0  0x000000fff7709cc4 in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 3 (LWP 5183):
 
#0  0x000000fff76e4a8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 2 (LWP 5184):
 
#0  0x000000fff76e4a8c in ?? ()
 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
 
 
 
Thread 1 (LWP 5127):
 
#0  0x000000fff770b210 in ?? ()
 
#1  0x0000000120a8ada8 in my_write_core (sig=<optimized out>) at /«PKGBUILDDIR»/mysys/stacktrace.c:457
 
Backtrace stopped: Cannot access memory at address 0xffffa55b48
 
 
 
 - deleting it, already saved 5
 
 - found 'core' (19/5)
 
 
 
Trying 'dbx' to get a backtrace
 
 
 
Trying 'gdb' to get a backtrace
 
Core generated by '/«PKGBUILDDIR»/builddir/sql/mysqld'
 
Output from gdb follows. The first stack trace is from the failing thread.
 
The following stack traces are from all threads (so the failing one is
 
duplicated).
 
--------------------------
 
[New LWP 5119]
 
[New LWP 5133]
 
[New LWP 5128]
 
[New LWP 5131]
 
[New LWP 5132]
 
[New LWP 5134]
 
[New LWP 5135]
 
[New LWP 5136]
 
[Thread debugging using libthread_db enabled]
 
Using host libthread_db library "/lib/mips64el-linux-gnuabi64/libthread_db.so.1".
 
Core was generated by `/«PKGBUILDDIR»/builddir/sql/mysqld --defaults-'.
 
Program terminated with signal SIGSEGV, Segmentation fault.
 
#0  0x000000fff6dbf210 in pthread_kill () from /lib/mips64el-linux-gnuabi64/libpthread.so.0
 
[Current thread is 1 (LWP 5119)]
 
#0  0x000000fff6dbf210 in pthread_kill () from /lib/mips64el-linux-gnuabi64/libpthread.so.0
 
#1  0x0000000120a8ada8 in my_write_core (sig=<optimized out>) at /«PKGBUILDDIR»/mysys/stacktrace.c:457
 
#2  0x000000012049e838 in handle_fatal_signal (sig=<optimized out>) at /«PKGBUILDDIR»/sql/signal_handler.cc:262
 
#3  <signal handler called>
 
#4  0x000000fff6b48e80 in std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*) () from /usr/lib/mips64el-linux-gnuabi64/libstdc++.so.6
 
#5  0x00000001207cf878 in std::__cxx11::list<unsigned char*, std::allocator<unsigned char*> >::_M_insert (__x=<synthetic pointer>, __position=..., this=0x118) at /usr/include/c++/5/bits/stl_list.h:1755
 
#6  std::__cxx11::list<unsigned char*, std::allocator<unsigned char*> >::push_back (__x=<synthetic pointer>, this=0x118) at /usr/include/c++/5/bits/stl_list.h:1089
 
#7  recv_dblwr_t::add (this=0x118, page=0xffeb004000 "\377\060~$") at /«PKGBUILDDIR»/storage/xtradb/log/log0recv.cc:4077
 
#8  0x00000001208df7f8 in buf_dblwr_init_or_load_pages (file=<optimized out>, path=0xfff678b029 <srv_read_only_mode> "", load_corrupt_pages=<optimized out>) at /«PKGBUILDDIR»/storage/xtradb/buf/buf0dblwr.cc:457
 
#9  0x000000fff66e42bc in open_or_create_data_files (sum_of_new_sizes=<synthetic pointer>, max_flushed_lsn=0xffffa89a50, min_flushed_lsn=0xffffa89a48, create_new_db=<synthetic pointer>) at /«PKGBUILDDIR»/storage/innobase/srv/srv0start.cc:1033
 
#10 innobase_start_or_create_for_mysql () at /«PKGBUILDDIR»/storage/innobase/srv/srv0start.cc:2105
 
#11 0x000000fff65f0028 in innobase_init (p=<optimized out>) at /«PKGBUILDDIR»/storage/innobase/handler/ha_innodb.cc:3400
 
#12 0x00000001204a17bc in ha_initialize_handlerton (plugin=0xfff5cb49f0) at /«PKGBUILDDIR»/sql/handler.cc:501
 
#13 0x00000001202fcafc in plugin_initialize (tmp_root=0xffffa91c40, tmp_root@entry=0x0, plugin=plugin@entry=0xfff5cb49f0, argc=0x120fdf680 <remaining_argc>, argc@entry=0x6, argv=0xfff5c7cc58, argv@entry=0xfd8, options_only=<optimized out>) at /«PKGBUILDDIR»/sql/sql_plugin.cc:1379
 
#14 0x00000001202fd7b0 in plugin_init (argc=0x6, argv=0xfd8, flags=<optimized out>) at /«PKGBUILDDIR»/sql/sql_plugin.cc:1659
 
#15 0x00000001202347ec in init_server_components () at /«PKGBUILDDIR»/sql/mysqld.cc:4850
 
#16 0x000000012023b83c in mysqld_main (argc=255, argv=0xff00000000) at /«PKGBUILDDIR»/sql/mysqld.cc:5445
 
#17 0x000000012021cf18 in main (argc=<optimized out>, argv=<optimized out>) at /«PKGBUILDDIR»/sql/main.cc:25
 
 
 
Thread 8 (LWP 5136):
 
warning: GDB can't find the start of the function at 0xfff6d98a8c.
 
 
 
    GDB is unable to find the start of the function at 0xfff6d98a8c
 
and thus can't determine the size of that function's stack frame.
 
This means that GDB may be unable to access that stack frame, or
 
the frames below it.
 
    This problem is most likely caused by an invalid program counter or
 
stack pointer.
 
    However, if you think GDB should simply search farther back
 
from 0xfff6d98a8c for code which looks like the beginning of a
 
function, you can increase the range of the search using the `set
 
heuristic-fence-post' command.
 
#0  0x000000fff6d98a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 7 (LWP 5135):
 
warning: GDB can't find the start of the function at 0xfff6d98a8c.
 
#0  0x000000fff6d98a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 6 (LWP 5134):
 
warning: GDB can't find the start of the function at 0xfff6d98a8c.
 
#0  0x000000fff6d98a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 5 (LWP 5132):
 
warning: GDB can't find the start of the function at 0xfff6d98a8c.
 
#0  0x000000fff6d98a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 4 (LWP 5131):
 
warning: GDB can't find the start of the function at 0xfff6d98a8c.
 
#0  0x000000fff6d98a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 3 (LWP 5128):
 
#0  0x000000fff6dbdcc4 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/mips64el-linux-gnuabi64/libpthread.so.0
 
#1  0x0000000120699a7c in inline_mysql_cond_timedwait (src_file=0x120ba9a40 "/«PKGBUILDDIR»/storage/maria/ma_servicethread.c", src_line=119, abstime=0xffefffe7d0, mutex=0x1217fea58 <LOCK_checkpoint>, that=<optimized out>) at /«PKGBUILDDIR»/include/mysql/psi/mysql_thread.h:1191
 
#2  my_service_thread_sleep (control=0x120f4b540 <checkpoint_control>, sleep_time=<optimized out>) at /«PKGBUILDDIR»/storage/maria/ma_servicethread.c:118
 
#3  0x000000012068eb78 in ma_checkpoint_background (arg=0xfff6db5d74 <start_thread+212>, arg@entry=0x1e) at /«PKGBUILDDIR»/storage/maria/ma_checkpoint.c:709
 
#4  0x00000001207367b4 in pfs_spawn_thread (arg=<optimized out>) at /«PKGBUILDDIR»/storage/perfschema/pfs.cc:1860
 
#5  0x000000fff6db5d74 in start_thread () from /lib/mips64el-linux-gnuabi64/libpthread.so.0
 
Backtrace stopped: frame did not save the PC
 
 
 
Thread 2 (LWP 5133):
 
warning: GDB can't find the start of the function at 0xfff6d98a8c.
 
#0  0x000000fff6d98a8c in ?? () from /lib/mips64el-linux-gnuabi64/libaio.so.1
 
 
 
Thread 1 (LWP 5119):
 
#0  0x000000fff6dbf210 in pthread_kill () from /lib/mips64el-linux-gnuabi64/libpthread.so.0
 
#1  0x0000000120a8ada8 in my_write_core (sig=<optimized out>) at /«PKGBUILDDIR»/mysys/stacktrace.c:457
 
#2  0x000000012049e838 in handle_fatal_signal (sig=<optimized out>) at /«PKGBUILDDIR»/sql/signal_handler.cc:262
 
#3  <signal handler called>
 
#4  0x000000fff6b48e80 in std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*) () from /usr/lib/mips64el-linux-gnuabi64/libstdc++.so.6
 
#5  0x00000001207cf878 in std::__cxx11::list<unsigned char*, std::allocator<unsigned char*> >::_M_insert (__x=<synthetic pointer>, __position=..., this=0x118) at /usr/include/c++/5/bits/stl_list.h:1755
 
#6  std::__cxx11::list<unsigned char*, std::allocator<unsigned char*> >::push_back (__x=<synthetic pointer>, this=0x118) at /usr/include/c++/5/bits/stl_list.h:1089
 
#7  recv_dblwr_t::add (this=0x118, page=0xffeb004000 "\377\060~$") at /«PKGBUILDDIR»/storage/xtradb/log/log0recv.cc:4077
 
#8  0x00000001208df7f8 in buf_dblwr_init_or_load_pages (file=<optimized out>, path=0xfff678b029 <srv_read_only_mode> "", load_corrupt_pages=<optimized out>) at /«PKGBUILDDIR»/storage/xtradb/buf/buf0dblwr.cc:457
 
#9  0x000000fff66e42bc in open_or_create_data_files (sum_of_new_sizes=<synthetic pointer>, max_flushed_lsn=0xffffa89a50, min_flushed_lsn=0xffffa89a48, create_new_db=<synthetic pointer>) at /«PKGBUILDDIR»/storage/innobase/srv/srv0start.cc:1033
 
#10 innobase_start_or_create_for_mysql () at /«PKGBUILDDIR»/storage/innobase/srv/srv0start.cc:2105
 
#11 0x000000fff65f0028 in innobase_init (p=<optimized out>) at /«PKGBUILDDIR»/storage/innobase/handler/ha_innodb.cc:3400
 
#12 0x00000001204a17bc in ha_initialize_handlerton (plugin=0xfff5cb49f0) at /«PKGBUILDDIR»/sql/handler.cc:501
 
#13 0x00000001202fcafc in plugin_initialize (tmp_root=0xffffa91c40, tmp_root@entry=0x0, plugin=plugin@entry=0xfff5cb49f0, argc=0x120fdf680 <remaining_argc>, argc@entry=0x6, argv=0xfff5c7cc58, argv@entry=0xfd8, options_only=<optimized out>) at /«PKGBUILDDIR»/sql/sql_plugin.cc:1379
 
#14 0x00000001202fd7b0 in plugin_init (argc=0x6, argv=0xfd8, flags=<optimized out>) at /«PKGBUILDDIR»/sql/sql_plugin.cc:1659
 
#15 0x00000001202347ec in init_server_components () at /«PKGBUILDDIR»/sql/mysqld.cc:4850
 
#16 0x000000012023b83c in mysqld_main (argc=255, argv=0xff00000000) at /«PKGBUILDDIR»/sql/mysqld.cc:5445
 
#17 0x000000012021cf18 in main (argc=<optimized out>, argv=<optimized out>) at /«PKGBUILDDIR»/sql/main.cc:25
 
 
 
 - deleting it, already saved 5
 
Too many tests(10) failed! Terminating...
 
 
 
Only  484  of 4938 completed.
 
--------------------------------------------------------------------------
 
The servers were restarted 20 times
 
Spent 53.109 of 589 seconds executing testcases
 
 
 
Too many failed: Failed 5/35 tests, 85.71% were successful.
 
 
 
Failing test(s): rpl.rpl_invoked_features rpl.rpl_partition_innodb rpl.rpl_innodb_bug30888 rpl.rpl_mdev6020 rpl.rpl_mdev6386

Is this helpful? Can you deduce from the above enough to make a patch that fixes it or at least almost fixes it and prints out more relevant debug information?

Comment by Otto Kekäläinen [ 2016-01-26 ]

bar It would be ideal if you send PR just like you did in https://github.com/ottok/mariadb-10.0/pull/34/files and then I could upload this to Debian experimental to see the results for this and any other fixes done in the master branch by the time of upload.

Comment by Jan Lindström (Inactive) [ 2016-02-03 ]

It crashes on some STL function when loading doublewrite buffers, strange is that this works on other platforms, I would need access to mips machine to continue but this could be bug on used C++ libraries. What is the version of gcc and stl ?

Comment by Otto Kekäläinen [ 2016-02-03 ]

Thanks for analyzing the stack trace. We don't have any mips machines, sorry. I don't know what you mean by 'stl' but you can check the Debian sid (=Debian unstable) software versions at https://packages.debian.org/search?keywords=stl&searchon=names&suite=unstable&section=all

Comment by Elena Stepanova [ 2016-08-21 ]

svoj, bar,

This cannot be solved from a test side (in scope of MDEV-10604). I could disable 10 tests that fail first, but it makes no sense because obviously there will be hundreds of others.

Comment by Otto Kekäläinen [ 2016-08-22 ]

I think cvicentiu already fixed these?

The 10.0 branch does not contain any commits with MDEV-8691 but mips and mipsel builds in latest release work, see https://buildd.debian.org/status/package.php?p=mariadb-10.0
The mips64el fails, but for other reasons.

Comment by Elena Stepanova [ 2016-08-24 ]

That's good.
cvicentiu, can you clarify what you've done to solve it, and maybe this issue can be closed?

Comment by Vicențiu Ciorbaru [ 2016-08-24 ]

elenst I did not do anything specifically related to those tests. I can see that the builds don't fail. Perhaps this was something introduced in upstream fixes. I recall seeing the federated test failures in buildbot as well.

I suggest you link this issue to MDEV-10417, not necessarily as reporting the same problem but I feel it could be related. We should close it once MDEV-10417 is fixed or at least we know the cause of the failures.

Comment by Otto Kekäläinen [ 2016-10-01 ]

All of these have tentatively been fixed in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=838914 - we just need to wait for confirmation and then upstream those patches. Rmember to use 'git --author' to correctly credit the original author.

Comment by Otto Kekäläinen [ 2016-11-25 ]

Looking at https://buildd.debian.org/status/fetch.php?pkg=mariadb-10.0&arch=mips&ver=10.0.28-2&stamp=1478918498 I can see that some of these still fail and some now pass:

federated.federated_partition 'X,xtradb' w2 [ pass ]    111
 
federated.federated_partition 'X,innodb_plugin' w1 [ pass ]    162
 
federated.federated_innodb               [ disabled ]  MDEV-10617 - Wrong checksum, timeouts
 
federated.federated_server               [ skipped ]  Test needs --big-test
 
federated.federated_transactions         [ disabled ]  MDEV-10617 - Wrong checksum, timeouts
 
federated.federatedx                     [ disabled ]  MDEV-10617 - Wrong checksum, timeouts
 
multi_source.gtid                        [ disabled ]  MDEV-10620 - Timeout in wait condition
 
multi_source.multisource 'innodb_plugin' w2 [ pass ]   3118
 
multi_source.multisource 'xtradb'        w1 [ pass ]   2010
 
rpl.rpl_auto_increment 'innodb_plugin,mix' w2 [ pass ]   2474
 
rpl.rpl_auto_increment 'mix,xtradb'      w1 [ pass ]   2232
 
rpl.rpl_auto_increment 'innodb_plugin,row' w2 [ pass ]   2672
 
rpl.rpl_auto_increment 'row,xtradb'      w1 [ pass ]   2545
 
rpl.rpl_auto_increment 'innodb_plugin,stmt' w2 [ pass ]   2243
 
rpl.rpl_auto_increment 'stmt,xtradb'     w1 [ pass ]   2480
 
rpl.rpl_invoked_features 'innodb_plugin,mix' w2 [ pass ]   2136
 
rpl.rpl_invoked_features 'mix,xtradb'    w1 [ pass ]   1950
 
rpl.rpl_partition_innodb 'innodb_plugin,mix' w2 [ pass ]   1607

Please cvicentiu take the relevant patches from https://anonscm.debian.org/cgit/pkg-mysql/mariadb-10.0.git/tree/debian/patches to upstream and close this issue. The remaining failures can be tracked in MDEV-10617 and MDEV-10620.

Comment by Daniel Black [ 2022-12-15 ]

working now - https://buildd.debian.org/status/logs.php?pkg=mariadb-10.6&arch=mips64el

Generated at Thu Feb 08 07:29:03 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.