[MDEV-34338] mtr - innodb.insert_into_empty hang - Jira

XML

Word

Printable

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Duplicate
Affects Version/s: 11.6(EOL)
Fix Version/s: 11.4.2, 10.6.19, 10.11.9, 11.1.6, 11.2.5, 11.5.2, 11.6.1
Component/s: Storage Engine - InnoDB, Tests
Labels:
None

Description

https://buildbot.mariadb.net/buildbot/builders/winx64-debug/builds/45129/steps/test/logs/stdio shows a possible hang with only 2 active threads, the page cleaner and the bulk-inserting thread. Similar example on Linux https://buildbot.mariadb.net/buildbot/builders/kvm-deb-buster-amd64/builds/16045/steps/mtr/logs/stdio

Perhaps, it is not a real hang, because page cleaner is doing something, but test times out anyway, where on retry it succeeds in 10 sec.

Seems to happen more often with 4K page size.

.  0  Id: 1ff0.270c Suspend: 0 Teb: 00000022`29ff2000 Unfrozen

      Priority: 0  Priority class: 32

ntdll!NtWaitForMultipleObjects

KERNELBASE!WaitForMultipleObjectsEx

KERNELBASE!WaitForMultipleObjects

server!handle_connections_win(void) [D:\winx64-debug\build\src\sql\handle_connections_win.cc @ 633]

server!mysqld_main(int argc = 0n178, char ** argv = 0x0000019a`7c0fa410) [D:\winx64-debug\build\src\sql\mysqld.cc @ 6108]

server!mysqld_win_main(int argc = 0n46, char ** argv = 0x0000019a`7c0c92c0) [D:\winx64-debug\build\src\sql\winmain.cc @ 278]

mariadbd!main(int argc = 0n46, char ** argv = 0x0000019a`7c0c92c0) [D:\winx64-debug\build\src\sql\main.cc @ 36]

(Inline Function) --------`-------- mariadbd!invoke_main(void) [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 78]

mariadbd!__scrt_common_main_seh(void) [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288]

kernel32!BaseThreadInitThunk

ntdll!RtlUserThreadStart

.  1  Id: 1ff0.6cc Suspend: 0 Teb: 00000022`29ff8000 Unfrozen

      Priority: 0  Priority class: 32

ntdll!NtWaitForAlertByThreadId

ntdll!RtlSleepConditionVariableCS

KERNELBASE!SleepConditionVariableCS

server!pthread_cond_timedwait(struct _RTL_CONDITION_VARIABLE * cond = 0x00007ff8`5a1f6550, struct _RTL_CRITICAL_SECTION * mutex = 0x00007ff8`5a1f6520, struct timespec * abstime = 0x00000022`2a2ffc08) [D:\winx64-debug\build\src\mysys\my_wincond.c @ 82]

server!psi_cond_timedwait(struct st_mysql_cond * that = 0x00007ff8`5a1f6550, struct st_mysql_mutex * mutex = 0x00007ff8`5a1f6520, struct timespec * abstime = 0x00000022`2a2ffc08, char * file = 0x00007ff8`59e943c8 "--- memory read error at address 0x00007ff8`59e943c8 ---", unsigned int line = 0x147) [D:\winx64-debug\build\src\mysys\my_thr_init.c @ 594]

server!inline_mysql_cond_timedwait(struct st_mysql_cond * that = 0x00007ff8`5a1f6550, struct st_mysql_mutex * mutex = 0x00007ff8`5a1f6520, struct timespec * abstime = 0x00000022`2a2ffc08, char * src_file = 0x00007ff8`59e943c8 "--- memory read error at address 0x00007ff8`59e943c8 ---", unsigned int src_line = 0x147) [D:\winx64-debug\build\src\include\mysql\psi\mysql_thread.h @ 1086]

server!timer_handler(void * arg = 0x00000000`00000000) [D:\winx64-debug\build\src\mysys\thr_timer.c @ 328]

server!pfs_spawn_thread(void * arg = 0x0000019a`019eec78) [D:\winx64-debug\build\src\storage\perfschema\pfs.cc @ 2200]

server!pthread_start(void * p = 0x0000019a`7c0d5b20) [D:\winx64-debug\build\src\mysys\my_winthread.c @ 61]

ucrtbase!thread_start<unsigned int

kernel32!BaseThreadInitThunk

ntdll!RtlUserThreadStart

.  2  Id: 1ff0.e80 Suspend: 0 Teb: 00000022`29ffa000 Unfrozen

      Priority: 0  Priority class: 32

ntdll!NtWaitForAlertByThreadId

ntdll!RtlSleepConditionVariableCS

KERNELBASE!SleepConditionVariableCS

server!pthread_cond_timedwait(struct _RTL_CONDITION_VARIABLE * cond = 0x00007ff8`5aba75d0, struct _RTL_CRITICAL_SECTION * mutex = 0x00007ff8`5aba75a0, struct timespec * abstime = 0x00000022`2a3ff958) [D:\winx64-debug\build\src\mysys\my_wincond.c @ 82]

server!psi_cond_timedwait(struct st_mysql_cond * that = 0x00007ff8`5aba75d0, struct st_mysql_mutex * mutex = 0x00007ff8`5aba75a0, struct timespec * abstime = 0x00000022`2a3ff958, char * file = 0x00007ff8`5a150c38 "--- memory read error at address 0x00007ff8`5a150c38 ---", unsigned int line = 0x74) [D:\winx64-debug\build\src\mysys\my_thr_init.c @ 594]

server!inline_mysql_cond_timedwait(struct st_mysql_cond * that = 0x00007ff8`5aba75d0, struct st_mysql_mutex * mutex = 0x00007ff8`5aba75a0, struct timespec * abstime = 0x00000022`2a3ff958, char * src_file = 0x00007ff8`5a150c38 "--- memory read error at address 0x00007ff8`5a150c38 ---", unsigned int src_line = 0x74) [D:\winx64-debug\build\src\include\mysql\psi\mysql_thread.h @ 1086]

server!my_service_thread_sleep(struct st_ma_service_thread_control * control = 0x00007ff8`5a11aae8, unsigned int64 sleep_time = 0x00000006`fc23ac00) [D:\winx64-debug\build\src\storage\maria\ma_servicethread.c @ 119]

server!ma_checkpoint_background(void * arg = 0x00000000`0000001e) [D:\winx64-debug\build\src\storage\maria\ma_checkpoint.c @ 725]

server!pfs_spawn_thread(void * arg = 0x0000019a`03ecb5d8) [D:\winx64-debug\build\src\storage\perfschema\pfs.cc @ 2200]

server!pthread_start(void * p = 0x0000019a`7c0d5be0) [D:\winx64-debug\build\src\mysys\my_winthread.c @ 61]

ucrtbase!thread_start<unsigned int

kernel32!BaseThreadInitThunk

ntdll!RtlUserThreadStart

.  3  Id: 1ff0.192c Suspend: 0 Teb: 00000022`29ffc000 Unfrozen

      Priority: 0  Priority class: 32

ntdll!ZwWaitForWorkViaWorkerFactory

ntdll!RtlInitializeResource

kernel32!BaseThreadInitThunk

ntdll!RtlUserThreadStart

.  4  Id: 1ff0.1ea8 Suspend: 0 Teb: 00000022`29ffe000 Unfrozen

      Priority: 0  Priority class: 32

server!PFS_buffer_scalable_container<PFS_rwlock,1024,1024,PFS_buffer_default_array<PFS_rwlock>,PFS_buffer_default_allocator<PFS_rwlock> >::sanitize(struct PFS_rwlock * unsafe = 0x0000019a`011f0c40) [D:\winx64-debug\build\src\storage\perfschema\pfs_buffer_container.h @ 935]

server!sanitize_rwlock(struct PFS_rwlock * unsafe = 0x0000019a`011f0c40) [D:\winx64-debug\build\src\storage\perfschema\pfs_instr.cc @ 615]

server!pfs_unlock_rwlock_v1(struct PSI_rwlock * rwlock = 0x0000019a`011f0c40) [D:\winx64-debug\build\src\storage\perfschema\pfs.cc @ 3687]

server!srw_lock_impl<0>::wr_unlock(void) [D:\winx64-debug\build\src\storage\innobase\include\srw_lock.h @ 532]

server!buf_flush_sync_for_checkpoint(unsigned int64 lsn = 0x7fe9e2) [D:\winx64-debug\build\src\storage\innobase\buf\buf0flu.cc @ 2146]

server!buf_flush_page_cleaner(void) [D:\winx64-debug\build\src\storage\innobase\buf\buf0flu.cc @ 2386]

server!std::invoke<void (<function> ** _Obj = 0x0000019a`01211020) [C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\include\type_traits @ 1730]

server!std::thread::_Invoke<std::tuple<void (void * _RawVals = 0x0000019a`01211020) [C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\include\thread @ 61]

ucrtbase!thread_start<unsigned int

kernel32!BaseThreadInitThunk

ntdll!RtlUserThreadStart

.  5  Id: 1ff0.2898 Suspend: 0 Teb: 00000022`29e04000 Unfrozen

      Priority: 0  Priority class: 32

ntdll!NtWaitForAlertByThreadId

ntdll!RtlSleepConditionVariableCS

KERNELBASE!SleepConditionVariableCS

server!pthread_cond_timedwait(struct _RTL_CONDITION_VARIABLE * cond = 0x00007ff8`5a1a4568, struct _RTL_CRITICAL_SECTION * mutex = 0x00007ff8`5a1a4538, struct timespec * abstime = 0x00000000`00000000) [D:\winx64-debug\build\src\mysys\my_wincond.c @ 82]

server!pthread_cond_wait(struct _RTL_CONDITION_VARIABLE * cond = 0x00007ff8`5a1a4568, struct _RTL_CRITICAL_SECTION * mutex = 0x00007ff8`5a1a4538) [D:\winx64-debug\build\src\mysys\my_wincond.c @ 91]

server!psi_cond_wait(struct st_mysql_cond * that = 0x00007ff8`5a1a4568, struct st_mysql_mutex * mutex = 0x00007ff8`5a1a4538, char * file = 0x00007ff8`585bf178 "D:\winx64-debug\build\src\sql\sql_manager.cc", unsigned int line = 0x6d) [D:\winx64-debug\build\src\mysys\my_thr_init.c @ 581]

server!inline_mysql_cond_wait(struct st_mysql_cond * that = 0x00007ff8`5a1a4568, struct st_mysql_mutex * mutex = 0x00007ff8`5a1a4538, char * src_file = 0x00007ff8`585bf178 "D:\winx64-debug\build\src\sql\sql_manager.cc", unsigned int src_line = 0x6d) [D:\winx64-debug\build\src\include\mysql\psi\mysql_thread.h @ 1070]

server!handle_manager(void * arg = 0x00000000`00000000) [D:\winx64-debug\build\src\sql\sql_manager.cc @ 109]

server!pfs_spawn_thread(void * arg = 0x0000019a`040d5b78) [D:\winx64-debug\build\src\storage\perfschema\pfs.cc @ 2200]

server!pthread_start(void * p = 0x0000019a`040011d0) [D:\winx64-debug\build\src\mysys\my_winthread.c @ 61]

ucrtbase!thread_start<unsigned int

kernel32!BaseThreadInitThunk

ntdll!RtlUserThreadStart

.  6  Id: 1ff0.b20 Suspend: 0 Teb: 00000022`29e0a000 Unfrozen

      Priority: 0  Priority class: 32

ntdll!NtWaitForAlertByThreadId

ntdll!RtlSleepConditionVariableCS

KERNELBASE!SleepConditionVariableCS

server!pthread_cond_timedwait(struct _RTL_CONDITION_VARIABLE * cond = 0x00007ff8`5ac3d8e0, struct _RTL_CRITICAL_SECTION * mutex = 0x00007ff8`5ac39580, struct timespec * abstime = 0x00000022`2aa4b288) [D:\winx64-debug\build\src\mysys\my_wincond.c @ 82]

server!buf_LRU_get_free_block(buf_LRU_get get = have_no_mutex (0n0)) [D:\winx64-debug\build\src\storage\innobase\buf\buf0lru.cc @ 459]

server!fsp_page_create(struct fil_space_t * space = 0x0000019a`040260b8, unsigned int offset = 0x729, struct mtr_t * mtr = 0x00000022`2aa4b670) [D:\winx64-debug\build\src\storage\innobase\fsp\fsp0fsp.cc @ 1055]

server!fseg_alloc_free_page_low(struct fil_space_t * space = 0x0000019a`040260b8, unsigned char * seg_inode = 0x0000019a`05a3e272 "--- memory read error at address 0x0000019a`05a3e272 ---", struct buf_block_t * iblock = 0x0000019a`05988b10, unsigned int hint = 0x729, unsigned char direction = 0x71 'q', bool has_done_reservation = true, struct mtr_t * mtr = 0x00000022`2aa4b670, struct mtr_t * init_mtr = 0x00000022`2aa4b670, dberr_t * err = 0x00000022`2aa4baa4) [D:\winx64-debug\build\src\storage\innobase\fsp\fsp0fsp.cc @ 2250]

server!fseg_alloc_free_page_general(unsigned char * seg_header = 0x0000019a`05a3f04a "--- memory read error at address 0x0000019a`05a3f04a ---", unsigned int hint = 0x729, unsigned char direction = 0x71 'q', bool has_done_reservation = true, struct mtr_t * mtr = 0x00000022`2aa4b670, struct mtr_t * init_mtr = 0x00000022`2aa4b670, dberr_t * err = 0x00000022`2aa4baa4) [D:\winx64-debug\build\src\storage\innobase\fsp\fsp0fsp.cc @ 2303]

server!btr_page_alloc(struct dict_index_t * index = 0x0000019a`045fd388, unsigned int hint_page_no = 0x729, unsigned char file_direction = 0x71 'q', unsigned int64 level = 0, struct mtr_t * mtr = 0x00000022`2aa4b670, struct mtr_t * init_mtr = 0x00000022`2aa4b670, dberr_t * err = 0x00000022`2aa4baa4) [D:\winx64-debug\build\src\storage\innobase\btr\btr0btr.cc @ 588]

server!btr_store_big_rec_extern_fields(struct btr_pcur_t * pcur = 0x00000022`2aa4bd20, unsigned short * offsets = 0x0000019a`045e0268, struct big_rec_t * big_rec_vec = 0x0000019a`088df0d8, struct mtr_t * btr_mtr = 0x0000019a`045dfd88, blob_op op = BTR_STORE_INSERT_BULK (0n3)) [D:\winx64-debug\build\src\storage\innobase\btr\btr0cur.cc @ 5863]

server!PageBulk::storeExt(struct big_rec_t * big_rec = 0x0000019a`088df0d8, unsigned short * offsets = 0x0000019a`045e0268) [D:\winx64-debug\build\src\storage\innobase\btr\btr0bulk.cc @ 819]

server!BtrBulk::insert(struct dtuple_t * tuple = 0x0000019a`04611b08, unsigned int64 level = 0) [D:\winx64-debug\build\src\storage\innobase\btr\btr0bulk.cc @ 1113]

server!BtrBulk::insert(struct dtuple_t * tuple = 0x0000019a`04611b08) [D:\winx64-debug\build\src\storage\innobase\include\btr0bulk.h @ 302]

server!row_merge_insert_index_tuples(struct dict_index_t * index = 0x0000019a`045fd388, struct dict_table_t * old_table = 0x0000019a`043b8088, struct pfs_os_file_t * fd = 0x00000022`2aa4c5e8, unsigned char * block = 0x00000000`00000000 "", struct row_merge_buf_t * row_buf = 0x00000022`2aa4c508, class BtrBulk * btr_bulk = 0x00000022`2aa4c568, unsigned int64 table_total_rows = 0, double pct_progress = 0, double pct_cost = 0, unsigned char * crypt_block = 0x00000000`00000000 "", unsigned int64 space = 0x2e, class ut_stage_alter_t * stage = 0x00000000`00000000, struct merge_file_t * blob_file = 0x0000019a`0438a0d0) [D:\winx64-debug\build\src\storage\innobase\row\row0merge.cc @ 3854]

server!row_merge_bulk_t::write_to_index(unsigned int64 index_no = 0, struct trx_t * trx = 0x0000019a`05171b40) [D:\winx64-debug\build\src\storage\innobase\row\row0merge.cc @ 5298]

server!row_merge_bulk_t::write_to_table(struct dict_table_t * table = 0x0000019a`043b8088, struct trx_t * trx = 0x0000019a`05171b40) [D:\winx64-debug\build\src\storage\innobase\row\row0merge.cc @ 5335]

server!trx_mod_table_time_t::write_bulk(struct dict_table_t * table = 0x0000019a`043b8088, struct trx_t * trx = 0x0000019a`05171b40) [D:\winx64-debug\build\src\storage\innobase\row\row0merge.cc @ 5348]

server!innodb_prepare_commit_versioned(class THD * thd = 0x0000019a`04361ff8, unsigned int64 * trx_id = 0x00000022`2aa4ca28) [D:\winx64-debug\build\src\storage\innobase\handler\ha_innodb.cc @ 3695]

server!ha_commit_trans(class THD * thd = 0x0000019a`04361ff8, bool all = false) [D:\winx64-debug\build\src\sql\handler.cc @ 1837]

server!trans_commit_stmt(class THD * thd = 0x0000019a`04361ff8) [D:\winx64-debug\build\src\sql\transaction.cc @ 497]

server!mysql_execute_command(class THD * thd = 0x0000019a`04361ff8, bool is_called_from_prepared_stmt = false) [D:\winx64-debug\build\src\sql\sql_parse.cc @ 5927]

server!mysql_parse(class THD * thd = 0x0000019a`04361ff8, char * rawbuf = 0x0000019a`04350520 "--- memory read error at address 0x0000019a`04350520 ---", unsigned int length = 0x29, class Parser_state * parser_state = 0x00000022`2aa4eb60) [D:\winx64-debug\build\src\sql\sql_parse.cc @ 7868]

server!dispatch_command(enum_server_command command = COM_QUERY (0n3), class THD * thd = 0x0000019a`04361ff8, char * packet = 0x0000019a`04573bf9 "--- memory read error at address 0x0000019a`04573bf9 ---", unsigned int packet_length = 0x29, bool blocking = false) [D:\winx64-debug\build\src\sql\sql_parse.cc @ 1894]

server!do_command(class THD * thd = 0x0000019a`04361ff8, bool blocking = false) [D:\winx64-debug\build\src\sql\sql_parse.cc @ 1405]

server!threadpool_process_request(class THD * thd = 0x0000019a`04361ff8) [D:\winx64-debug\build\src\sql\threadpool_common.cc @ 435]

server!tp_callback(struct TP_connection * c = 0x0000019a`041fa610) [D:\winx64-debug\build\src\sql\threadpool_common.cc @ 249]

server!tp_callback(struct _TP_CALLBACK_INSTANCE * instance = 0x00000022`2aa4fba8, void * context = 0x0000019a`041fa610) [D:\winx64-debug\build\src\sql\threadpool_win.cc @ 279]

server!io_completion_callback(struct _TP_CALLBACK_INSTANCE * instance = 0x00000022`2aa4fba8, void * context = 0x0000019a`041fa610, void * overlapped = 0x0000019a`041fa680, unsigned long io_result = 0, unsigned int64 nbytes = 0x2e, struct _TP_IO * io = 0x0000019a`042d0a80) [D:\winx64-debug\build\src\sql\threadpool_win.cc @ 299]

kernel32!QueryPerformanceCounter

ntdll!TpReleaseWork

ntdll!RtlInitializeResource

kernel32!BaseThreadInitThunk

ntdll!RtlUserThreadStart

Total threads: 8

Duplicate callstacks: 1 (windbg thread #s follow):

Attachments

Issue Links

duplicates

MDEV-34166 Server could hang with BP < 80M under stress

Closed

Activity

People

Assignee:: Marko Mäkelä

Reporter:: Vladislav Vaintroub

Votes:: 0 Vote for this issue

Watchers:: 1 Start watching this issue

Dates

Created:: 2024-06-07 18:17

Updated:: 2024-06-08 14:50

Resolved:: 2024-06-08 14:50

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.