Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
10.5, 10.6
-
None
-
AIX
Description
EGuesnet can I get a hand with this please. Its constantly failing and with standard kind of debugging I've got a corrupted stack trace.
As its an assertion its not just a test failure.
https://buildbot.mariadb.org/#/builders/121/builds/1585/steps/6/logs/stdio
Also reproducable on the CMAKE_BUILD_TYPE=Debug
$ mysql-test/mtr --manual-gdb innodb.innodb_scrub
|
Logging: /home/mdborg/mariadb-server/mysql-test/mysql-test-run.pl --manual-gdb innodb.innodb_scrub
|
vardir: /home/mdborg/build-mariadb-server/mysql-test/var
|
Checking leftover processes...
|
Removing old var directory...
|
Creating var directory '/home/mdborg/build-mariadb-server/mysql-test/var'...
|
Checking supported features...
|
2021-07-08 2:06:40 0 [Warning] WSREP: Failed to guess base node address. Set it explicitly via wsrep_node_address.
|
2021-07-08 2:06:40 0 [Warning] WSREP: Failed to guess base node address. Set it explicitly via wsrep_node_address.
|
2021-07-08 2:06:40 0 [Warning] WSREP: Guessing address for incoming client connections failed. Try setting wsrep_node_incoming_address explicitly.
|
2021-07-08 2:06:40 0 [Note] WSREP: Node addr:
|
MariaDB Version 10.5.12-MariaDB
|
- SSL connections supported
|
- binaries built with wsrep patch
|
Collecting tests...
|
Installing system database...
|
|
==============================================================================
|
|
TEST RESULT TIME (ms) or COMMENT
|
--------------------------------------------------------------------------
|
|
worker[1] Using MTR_BUILD_THREAD 800, with reserved ports 26000..26019
|
|
To start gdb for mysqld.1, type in another window:
|
gdb -x /home/mdborg/build-mariadb-server/mysql-test/var/tmp/gdbinit.mysqld.1 /home/mdborg/build-mariadb-server/sql/mariadbd
|
|
gdb -x /home/mdborg/build-mariadb-server/mysql-test/var/tmp/gdbinit.mysqld.1 /home/mdborg/build-mariadb-server/sql/mariadbd
|
GNU gdb (GDB) 8.1.1
|
Copyright (C) 2018 Free Software Foundation, Inc.
|
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
|
This is free software: you are free to change and redistribute it.
|
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
|
and "show warranty" for details.
|
This GDB was configured as "powerpc64-ibm-aix6.1.0.0".
|
Type "show configuration" for configuration details.
|
For bug reporting instructions, please see:
|
<http://www.gnu.org/software/gdb/bugs/>.
|
Find the GDB manual and other documentation resources online at:
|
<http://www.gnu.org/software/gdb/documentation/>.
|
For help, type "help".
|
Type "apropos word" to search for commands related to "word"...
|
Reading symbols from /home/mdborg/build-mariadb-server/sql/mariadbd...done.
|
(gdb) run
|
Starting program: /home/mdborg/build-mariadb-server/sql/mariadbd --defaults-group-suffix=.1 --defaults-file=/home/mdborg/build-mariadb-server/mysql-test/var/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --innodb-immediate-scrub-data-uncompressed=ON --core-file --loose-debug-sync-timeout=300 --loose-gdb < /dev/null
|
BFD: BFD (GNU Binutils) 2.29.51 assertion fail format.c:453
|
warning: "/usr/lib/libdl.a": member "shr_64.o" missing.
|
warning: Could not load shared library symbols for /usr/lib/libdl.a(shr_64.o).
|
Do you need "set solib-search-path" or "set sysroot"?
|
[New Thread 1]
|
2021-07-08 2:07:15 0 [Note] /home/mdborg/build-mariadb-server/sql/mariadbd (mysqld 10.5.12-MariaDB-log) starting as process 13172792 ...
|
[New Thread 258]
|
[New Thread 515]
|
[New Thread 772]
|
[New Thread 1030]
|
[New Thread 1287]
|
[New Thread 1544]
|
[New Thread 1801]
|
[New Thread 2058]
|
[New Thread 2315]
|
[New Thread 2572]
|
[New Thread 2829]
|
[New Thread 3086]
|
[New Thread 3343]
|
[New Thread 3600]
|
[New Thread 3857]
|
[New Thread 4114]
|
[New Thread 4371]
|
[New Thread 4628]
|
[New Thread 4885]
|
[New Thread 5142]
|
[New Thread 5399]
|
[New Thread 5656]
|
[New Thread 5913]
|
[New Thread 6170]
|
[New Thread 6427]
|
[New Thread 6684]
|
[New Thread 6941]
|
[New Thread 7198]
|
[New Thread 7455]
|
[New Thread 7712]
|
[New Thread 7969]
|
|
Thread 2 received signal SIGABRT, Aborted.
|
[Switching to Thread 1]
|
0x0000000000000000 in ?? ()
|
(gdb) bt
|
#0 0x0000000000000000 in ?? ()
|
(gdb) bt full
|
#0 0x0000000000000000 in ?? ()
|
No symbol table info available.
|
notes on other aix errors:
- main.long_unique / main.query_cache memory rlimit issue that I'll correct shortly
- plugins.server_audit / innodb.temporary_table / type_inet.type_inet6 / type_test.type_test_double - probably something simple however a non-functional debugging environment makes it hard to track down.
Attachments
Issue Links
- causes
-
MDEV-28473 field_ref_zero is not initialized in xtrabackup_prepare_func()
-
- Closed
-
This fail occurs since version 10.5.9 according my backlogs.
I can reproduce it. I have the following backtrace from core:
#0 0x0900000000536f94 in pthread_kill () from /usr/lib/libpthreads.a(shr_xpg5_64.o)
#1 0x09000000054cdaf4 in my_write_core () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#2 0x09000000054cd3a0 in handle_fatal_signal () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#3 <signal handler called>
#4 0x0900000000536f94 in pthread_kill () from /usr/lib/libpthreads.a(shr_xpg5_64.o)
#5 0x09000000005367e8 in _p_raise () from /usr/lib/libpthreads.a(shr_xpg5_64.o)
#6 0x0900000000039c6c in raise () from /usr/lib/libc.a(shr_64.o)
#7 0x0900000000056668 in abort () from /usr/lib/libc.a(shr_64.o)
#8 0x090000000493786c in ut_dbg_assertion_failed(char const*, char const*, unsigned int) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#9 0x0900000004b31d68 in os_aio(IORequest const&, void*, unsigned long, unsigned long) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#10 0x0900000004a52b8c in fil_space_t::io(IORequest const&, unsigned long, unsigned long, void*, buf_page_t*) ()
from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#11 0x0900000004a2aa90 in buf_flush_freed_pages(fil_space_t*) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#12 0x0900000004a2b69c in buf_flush_space(unsigned int) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#13 0x0900000004a2be70 in buf_flush_LRU_list_batch(unsigned long, flush_counters_t*) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#14 0x0900000004a2c3f8 in buf_do_LRU_batch(unsigned long) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#15 0x0900000004a2d480 in buf_flush_lists(unsigned long, unsigned long) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#16 0x09000000049e7b60 in buf_LRU_get_free_block(bool) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#17 0x0900000004974864 in buf_page_init_for_read(unsigned long, page_id_t, unsigned long, bool) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#18 0x0900000004975324 in buf_read_page_low(dberr_t*, fil_space_t*, bool, unsigned long, page_id_t, unsigned long, bool) ()
from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#19 0x0900000004975ed4 in buf_read_page(page_id_t, unsigned long) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#20 0x09000000049fab04 in buf_page_get_low(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, char const*, unsigned int, mtr_t*, dberr_t*, bool) ()
from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#21 0x09000000049fc33c in buf_page_get_gen(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, char const*, unsigned int, mtr_t*, dberr_t*, bool) ()
from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#22 0x0900000004d070f0 in trx_rsegf_get(fil_space_t*, unsigned int, mtr_t*) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#23 0x0900000004d092a4 in trx_purge_truncate_rseg_history(trx_rseg_t&, purge_sys_t::iterator const&) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#24 0x0900000004d09be4 in trx_purge_truncate_history() () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#25 0x0900000004d0d018 in trx_purge(unsigned long, bool) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#26 0x0900000004997e30 in srv_do_purge(unsigned long*) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#27 0x0900000004998684 in purge_coordinator_callback_low() () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#28 0x090000000499885c in purge_coordinator_callback(void*) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#29 0x0900000004b43854 in tpool::task_group::execute(tpool::task*) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#30 0x0900000004b3420c in tpool::task::execute() () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#31 0x0900000004b35da0 in tpool::thread_pool_generic::worker_main(tpool::worker_data*) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#32 0x0900000004b40bf4 in void std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>(std::__invoke_memfun_deref, void (tpool::thread_pool_generic::*&&)(tpool::worker_data*), tpool::thread_pool_generic*&&, tpool::worker_data*&&) () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#33 0x0900000004b409f0 in std::__invoke_result<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>::type std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>(void (tpool::thread_pool_generic::*&&)(tpool::worker_data*), tpool::thread_pool_generic*&&, tpool::worker_data*&&)
() from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#34 0x0900000004b3fae4 in decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)(), (_S_declval<2ul>)())) std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) ()
from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#35 0x0900000004b3f9b0 in std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator()() ()
from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#36 0x0900000004b3f8d8 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run() () from /opt/freeware/src/packages/BUILD/mariadb-10.5.9/debug/sql/libserver.a(libserver.so)
#37 0x0900000006befddc in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() () from /opt/freeware/lib/pthread/ppc64/libstdc++.a(libstdc++.so.6)
#38 0x090000000051ee14 in _pthread_body () from /usr/lib/libpthreads.a(shr_xpg5_64.o)
#39 0x0000000000000000 in ?? ()
2 threads. No backtrace for the second, only
aix-thread: ptrace (52, 70713541) returned -1 (errno = 3 No such process)
If I correctly understand the error, it fails at storage/innobase/os/os0file.cc:4141
ut_a(reinterpret_cast<size_t>(cb->m_buffer) % OS_FILE_LOG_BLOCK_SIZE== 0);
with OS_FILE_LOG_BLOCK_SIZE = 512U.
The test mysql-test/suite/innodb/t/innodb_scrub.test repeats 500 times an insert, then commits and does some stuff.
I do not understand the reason of the error as I see in log debug/mysql-test/var/log/innodb.innodb_scrub-innodb/mysqld.1/mysqld.log:
4 Query INSERT INTO t1(f2,f3) VALUES ('repairman', repeat('unicycle', 1000))
500 times. Last command is
4 Query SET GLOBAL innodb_max_purge_lag_wait= 0
that seems OK.
It is the first time I investigate this error, and I do not understand where it is from.