[MDEV-12899] innodb.innodb_prefix_index_restart_server failed in buildbot with crash in server shutdown Created: 2017-05-24  Updated: 2017-08-17  Resolved: 2017-08-17

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB, Tests
Affects Version/s: 10.1
Fix Version/s: 10.1.24, 10.0.32, 10.2.7

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Marko Mäkelä
Resolution: Duplicate Votes: 0
Labels: None

Issue Links:
Duplicate
duplicates MDEV-12052 Shutdown crash presumably due to mast... Closed

 Description   

http://buildbot.askmonty.org/buildbot/builders/win32-debug/builds/3843/steps/test/logs/stdio

innodb.innodb_prefix_index_restart_server 'xtradb' w2 [ fail ]  Found warnings/errors in server log file!
        Test ended at 2017-05-12 12:24:13
line
abort() has been called170512 12:23:46 [ERROR] mysqld got exception 0x80000003 ;
 
Attempting backtrace. You can use the following information to find out

Response                         Time (ms)     Location
OK                                             C:\Windows\System32
OK                                             D:\win32-debug\build\sql\Debug
OK                                             .
 
Response                         Time (ms)     Location
OK                                             C:\Windows\System32
OK                               15            D:\win32-debug\build\sql\Debug
OK                                             .
Deferred                                       cache*C:\cdb_symbols
Symbol search path is: C:\Windows\System32;D:\win32-debug\build\sql\Debug;.;cache*C:\cdb_symbols
Executable search path is: C:\Windows\System32;D:\win32-debug\build\sql\Debug;.
Windows 8.1 Version 9600 MP (4 procs) Free x86 compatible
Product: Server, suite: TerminalServer DataCenter SingleUserTS
Built by: 6.3.9600.17031 (winblue_gdr.140221-1952)
Machine Name:
Debug session time: Fri May 12 12:23:47.000 2017 (UTC + 0:00)
System Uptime: not available
Process Uptime: 0 days 0:00:43.000
.............................
This dump file has a breakpoint exception stored in it.
The stored exception information can be accessed via .ecxr.
eax=00000000 ebx=0aaf1a78 ecx=00000000 edx=00000000 esi=0aaf1a30 edi=0aaf1a40
eip=7733d04c esp=09b7e954 ebp=09b7e960 iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202
ntdll!NtGetContextThread:
         ret     8
0:009> cdb: Reading initial command '!sym prompts off; !analyze -v; .ecxr; !for_each_frame dv /t;!uniqstack -p;q'
quiet mode - symbol prompts off
 
Unable to load image C:\Windows\System32\KERNELBASE.dll, Win32 error 0n2
Unable to load image C:\Windows\System32\user32.dll, Win32 error 0n2
 
DUMP_CLASS: 2
 
DUMP_QUALIFIER: 400
 
CONTEXT:  (.ecxr)
eax=00000000 ebx=00000000 ecx=01b96250 edx=01b96250 esi=09b7fdd4 edi=09b7fe44
eip=01b96253 esp=09b7fd34 ebp=09b7fd34 iopl=0         nv up ei pl nz ac po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000212
mysqld!my_sigabrt_handler:
             int     3
Resetting default scope
 
FAULTING_IP: 
mysqld!my_sigabrt_handler+3 [d:\win32-debug\build\src\mysys\my_thr_init.c @ 488]
             int     3
 
EXCEPTION_RECORD:  (.exr -1)
ExceptionAddress: 01b96253 (mysqld!my_sigabrt_handler)
   ExceptionCode: 80000003 (Break instruction exception)
  ExceptionFlags: 00000000
NumberParameters: 1
   Parameter[0]: 00000000
 
DEFAULT_BUCKET_ID:  STATUS_BREAKPOINT
 
PROCESS_NAME:  mysqld.exe
 
ERROR_CODE: (NTSTATUS) 0x80000003 - {EXCEPTION}  Breakpoint  A breakpoint has been reached.
 
EXCEPTION_CODE: (HRESULT) 0x80000003 (2147483651) - One or more arguments are invalid
 
EXCEPTION_CODE_STR:  80000003
 
EXCEPTION_PARAMETER1:  00000000
 
WATSON_BKT_PROCSTAMP:  5915a077
 
WATSON_BKT_PROCVER:  10.1.24.0
 
WATSON_BKT_MODULE:  mysqld.exe
 
WATSON_BKT_MODSTAMP:  5915a077
 
WATSON_BKT_MODOFFSET:  9e6253
 
WATSON_BKT_MODVER:  10.1.24.0
 
BUILD_VERSION_STRING:  6.3.9600.17031 (winblue_gdr.140221-1952)
 
MODLIST_WITH_TSCHKSUM_HASH:  dfd24fdcf5716563dd5bbd88aa586e81cbc60528
 
MODLIST_SHA1_HASH:  81207203dd633cd768ae1922baf22c46a9235801
 
DUMP_FLAGS:  0
 
DUMP_TYPE:  2
 
APP:  mysqld.exe
 
ANALYSIS_SESSION_HOST:  MARIADB-02
 
ANALYSIS_SESSION_TIME:  05-12-2017 12:24:17.0463
 
ANALYSIS_VERSION: 10.0.10586.567 amd64fre
 
THREAD_ATTRIBUTES: 
PROBLEM_CLASSES: 
 
 
 
 
    Tid    [0x0]
    Frame  [0x00]
    String [STATUS_BREAKPOINT]
    Data Bucketing
 
 
BUGCHECK_STR:  STATUS_BREAKPOINT
 
LAST_CONTROL_TRANSFER:  from 01d462bb to 01b96253
 
STACK_TEXT:  
mysqld!my_sigabrt_handler
mysqld!raise
mysqld!abort
mysqld!buf_flush_page_cleaner_thread
kernel32!BaseThreadInitThunk
ntdll!__RtlUserThreadStart
ntdll!_RtlUserThreadStart
 
 
THREAD_SHA1_HASH_MOD_FUNC:  045355b9027418e5864d8b85ba57b969b5c24459
 
THREAD_SHA1_HASH_MOD_FUNC_OFFSET:  e7a76265a4236b7edd0e1ac9bf040f626e34e2a5
 
THREAD_SHA1_HASH_MOD:  f39d91b852cd70c7b54e3e2254f6333713335972
 
FOLLOWUP_IP: 
mysqld!my_sigabrt_handler+3 [d:\win32-debug\build\src\mysys\my_thr_init.c @ 488]
             int     3
 
FAULT_INSTR_CODE:  ccc35dcc
 
FAULTING_SOURCE_LINE:  d:\win32-debug\build\src\mysys\my_thr_init.c
 
FAULTING_SOURCE_FILE:  d:\win32-debug\build\src\mysys\my_thr_init.c
 
FAULTING_SOURCE_LINE_NUMBER:  488
 
FAULTING_SOURCE_CODE:  
   484: 
   485: #if (_MSC_VER >= 1400)
   486: static void my_sigabrt_handler(int sig)
   487: {
>  488:   __debugbreak();
   489: }
   490: #endif /*_MSC_VER >=1400 */
   491: 
   492: static void install_sigabrt_handler(void)
   493: {
 
 
SYMBOL_STACK_INDEX:  0
 
SYMBOL_NAME:  mysqld!my_sigabrt_handler+3
 
FOLLOWUP_NAME:  MachineOwner
 
MODULE_NAME: mysqld
 
IMAGE_NAME:  mysqld.exe
 
DEBUG_FLR_IMAGE_TIMESTAMP:  5915a077
 
STACK_COMMAND:  .ecxr ; kb
 
BUCKET_ID:  STATUS_BREAKPOINT_mysqld!my_sigabrt_handler+3
 
PRIMARY_PROBLEM_CLASS:  STATUS_BREAKPOINT_mysqld!my_sigabrt_handler+3
 
BUCKET_ID_OFFSET:  3
 
BUCKET_ID_MODULE_STR:  mysqld
 
BUCKET_ID_MODTIMEDATESTAMP:  5915a077
 
BUCKET_ID_MODCHECKSUM:  0
 
BUCKET_ID_MODVER_STR:  10.1.24.0
 
BUCKET_ID_PREFIX_STR:  STATUS_BREAKPOINT_
 
FAILURE_PROBLEM_CLASS:  STATUS_BREAKPOINT
 
FAILURE_EXCEPTION_CODE:  80000003
 
FAILURE_IMAGE_NAME:  mysqld.exe
 
FAILURE_FUNCTION_NAME:  my_sigabrt_handler
 
BUCKET_ID_FUNCTION_STR:  my_sigabrt_handler
 
FAILURE_SYMBOL_NAME:  mysqld.exe!my_sigabrt_handler
 
FAILURE_BUCKET_ID:  STATUS_BREAKPOINT_80000003_mysqld.exe!my_sigabrt_handler
 
WATSON_STAGEONE_URL:  http://watson.microsoft.com/StageOne/mysqld.exe/10.1.24.0/5915a077/mysqld.exe/10.1.24.0/5915a077/80000003/009e6253.htm?Retriage=1
 
TARGET_TIME:  2017-05-12T12:23:47.000Z
 
OSBUILD:  9600
 
OSSERVICEPACK:  17415
 
SERVICEPACK_NUMBER: 0
 
OS_REVISION: 0
 
SUITE_MASK:  400
 
PRODUCT_TYPE:  3
 
OSPLATFORM_TYPE:  x86
 
OSNAME:  Windows 8.1
 
OSEDITION:  Windows 8.1 Server TerminalServer DataCenter SingleUserTS
 
OS_LOCALE:  
 
USER_LCID:  0
 
OSBUILD_TIMESTAMP:  2014-10-29 01:58:22
 
BUILDDATESTAMP_STR:  140221-1952
 
BUILDLAB_STR:  winblue_gdr
 
BUILDOSVER_STR:  6.3.9600.17031
 
ANALYSIS_SESSION_ELAPSED_TIME: 1155
 
ANALYSIS_SOURCE:  UM
 
FAILURE_ID_HASH_STRING:  um:status_breakpoint_80000003_mysqld.exe!my_sigabrt_handler
 
FAILURE_ID_HASH:  {f6294d5d-cc99-d322-9f92-e2c8a944cc10}
 
Followup:     MachineOwner
---------
 
eax=00000000 ebx=00000000 ecx=01b96250 edx=01b96250 esi=09b7fdd4 edi=09b7fe44
eip=01b96253 esp=09b7fd34 ebp=09b7fd34 iopl=0         nv up ei pl nz ac po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000212
mysqld!my_sigabrt_handler:
             int     3
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
mysqld!my_sigabrt_handler [d:\win32-debug\build\src\mysys\my_thr_init.c @ 488]
int sig = 0n22
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
mysqld!raise [d:\th\minkernel\crts\ucrt\src\appcrt\misc\signal.cpp @ 516]
int signum = 0n22
int old_fpecode = 0n0
struct _EXCEPTION_POINTERS * old_pxcptinfoptrs = 0x00000000
bool return0 = false
bool action_is_global = true
<function> * action = 0x01b96250
struct __acrt_ptd * ptd = 0x00000000
<function> ** action_pointer = 0x036179a4
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
mysqld!abort [d:\th\minkernel\crts\ucrt\src\appcrt\startup\abort.cpp @ 64]
<function> * sigabrt_action = 0x01b96250
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
mysqld!buf_flush_page_cleaner_thread [d:\win32-debug\build\src\storage\xtradb\buf\buf0flu.cc @ 2881]
struct buf_pool_t * buf_pool = 0x072ef320
unsigned long i = 0
void * arg = 0x00000000
unsigned long last_activity = 0x14e
bool success = true
unsigned long n_flushed = 0
unsigned long last_activity_time = 0xfc9d64a1
unsigned long next_loop_time = 0xfc9d6889
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
kernel32!BaseThreadInitThunk
Unable to enumerate locals, HRESULT 0x80004005
Private symbols (symbols.pri) are required for locals.
Type ".hh dbgerr005" for details.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
ntdll!__RtlUserThreadStart
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
ntdll!_RtlUserThreadStart
Unable to enumerate locals, HRESULT 0x80004005
Private symbols (symbols.pri) are required for locals.
Type ".hh dbgerr005" for details.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
mysqld!my_sigabrt_handler [d:\win32-debug\build\src\mysys\my_thr_init.c @ 488]
Processing 20 threads, please wait
 
.  0  Id: 16a0.1144 Suspend: 0 Teb: ffbce000 Unfrozen
      Priority: 0  Priority class: 32
  
ntdll!NtWaitForAlertByThreadId
KERNELBASE!SleepConditionVariableCS
mysqld!pthread_cond_timedwait(union pthread_cond_t * cond = 0x02db2884, struct _RTL_CRITICAL_SECTION * mutex = 0x02db2600, struct timespec * abstime = 0x00000000) [d:\win32-debug\build\src\mysys\my_wincond.c @ 298]
mysqld!pthread_cond_wait(union pthread_cond_t * cond = 0x02db2884, struct _RTL_CRITICAL_SECTION * mutex = 0x02db2600) [d:\win32-debug\build\src\mysys\my_wincond.c @ 309]
mysqld!inline_mysql_cond_wait(struct st_mysql_cond * that = 0x02db2884, struct st_mysql_mutex * mutex = 0x02db2600, char * src_file = 0x01e4faf4 "D:\win32-debug\build\src\sql\mysqld.cc", unsigned int src_line = 0x1736) [d:\win32-debug\build\src\include\mysql\psi\mysql_thread.h @ 1154]
mysqld!win_main(int argc = 0n144, char ** argv = 0x00a16448) [d:\win32-debug\build\src\sql\mysqld.cc @ 5942]
mysqld!mysql_service(void * p = 0x00000000) [d:\win32-debug\build\src\sql\mysqld.cc @ 5976]
mysqld!mysqld_main(int argc = 0n19, char ** argv = 0x00a06178) [d:\win32-debug\build\src\sql\mysqld.cc @ 6170]
mysqld!main(int argc = 0n19, char ** argv = 0x00a06178) [d:\win32-debug\build\src\sql\main.cc @ 25]
mysqld!invoke_main(void) [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 64]
mysqld!__scrt_common_main_seh(void) [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 253]
mysqld!__scrt_common_main(void) [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 296]
mysqld!mainCRTStartup(void) [f:\dd\vctools\crt\vcstartup\src\startup\exe_main.cpp @ 17]
kernel32!BaseThreadInitThunk
ntdll!__RtlUserThreadStart
ntdll!_RtlUserThreadStart
 
.  1  Id: 16a0.16f8 Suspend: 0 Teb: ffbc8000 Unfrozen
      Priority: 0  Priority class: 32
  
ntdll!NtWaitForAlertByThreadId
KERNELBASE!SleepConditionVariableCS
mysqld!pthread_cond_timedwait(union pthread_cond_t * cond = 0x0360c620, struct _RTL_CRITICAL_SECTION * mutex = 0x0360c604, struct timespec * abstime = 0x07aff89c) [d:\win32-debug\build\src\mysys\my_wincond.c @ 298]
mysqld!inline_mysql_cond_timedwait(struct st_mysql_cond * that = 0x0360c620, struct st_mysql_mutex * mutex = 0x0360c604, struct timespec * abstime = 0x07aff89c, char * src_file = 0x02be8ec8 "--- memory read error at address 0x02be8ec8 ---", unsigned int src_line = 0x124) [d:\win32-debug\build\src\include\mysql\psi\mysql_thread.h @ 1191]
mysqld!timer_handler(void * arg = 0x00000000) [d:\win32-debug\build\src\mysys\thr_timer.c @ 293]
mysqld!pfs_spawn_thread(void * arg = 0x009f5648) [d:\win32-debug\build\src\storage\perfschema\pfs.cc @ 1860]
mysqld!pthread_start(void * p = 0x00a039f8) [d:\win32-debug\build\src\mysys\my_winthread.c @ 60]
mysqld!invoke_thread_procedure(<function> * procedure = 0x01bd7610, void * context = 0x00a039f8) [d:\th\minkernel\crts\ucrt\src\appcrt\startup\thread.cpp @ 92]
mysqld!thread_start<unsigned int (void * parameter = 0x00a0beb8) [d:\th\minkernel\crts\ucrt\src\appcrt\startup\thread.cpp @ 115]
kernel32!BaseThreadInitThunk
ntdll!__RtlUserThreadStart
ntdll!_RtlUserThreadStart
 
.  2  Id: 16a0.168c Suspend: 0 Teb: ffbc5000 Unfrozen
      Priority: 0  Priority class: 32
  
ntdll!NtRemoveIoCompletion
mysqld!os_aio_windows_handle(unsigned long segment = 0, unsigned long pos = 0, struct fil_node_t ** message1 = 0x086afa00, void ** message2 = 0x086af9f4, unsigned long * type = 0x086af9e8, unsigned long * space_id = 0x086af9dc) [d:\win32-debug\build\src\storage\xtradb\os\os0file.cc @ 5171]
mysqld!fil_aio_wait(unsigned long segment = 0) [d:\win32-debug\build\src\storage\xtradb\fil\fil0fil.cc @ 6550]
mysqld!io_handler_thread(void * arg = 0x035fcf38) [d:\win32-debug\build\src\storage\xtradb\srv\srv0start.cc @ 546]
kernel32!BaseThreadInitThunk
ntdll!__RtlUserThreadStart
ntdll!_RtlUserThreadStart
 
.  8  Id: 16a0.ab4 Suspend: 0 Teb: ffa87000 Unfrozen
      Priority: 0  Priority class: 32
  
ntdll!NtWaitForAlertByThreadId
KERNELBASE!SleepConditionVariableCS
mysqld!os_cond_wait(struct _RTL_CONDITION_VARIABLE * cond = 0x00a12528, struct os_fast_mutex_t * fast_mutex = 0x00a12504) [d:\win32-debug\build\src\storage\xtradb\os\os0sync.cc @ 194]
mysqld!os_event_wait_low(struct os_event * event = 0x00a12500, int64 reset_sig_count = 0n1) [d:\win32-debug\build\src\storage\xtradb\os\os0sync.cc @ 477]
mysqld!srv_resume_thread(struct srv_slot_t * slot = 0x00a13af0, int64 sig_count = 0n0, bool wait = true, unsigned long timeout_usec = 0) [d:\win32-debug\build\src\storage\xtradb\srv\srv0srv.cc @ 1087]
mysqld!srv_master_thread(void * arg = 0x00000000) [d:\win32-debug\build\src\storage\xtradb\srv\srv0srv.cc @ 3241]
kernel32!BaseThreadInitThunk
ntdll!__RtlUserThreadStart
ntdll!_RtlUserThreadStart
 
.  9  Id: 16a0.12fc Suspend: 0 Teb: ffa81000 Unfrozen
      Priority: 0  Priority class: 32
  
ntdll!NtGetContextThread
WARNING: Frame IP not in any known module. Following frames may be wrong.
0x9b7ea18
ntdll!_except_handler4
 
. 10  Id: 16a0.728 Suspend: 0 Teb: ffa6c000 Unfrozen
      Priority: 0  Priority class: 32
  
ntdll!NtWaitForWorkViaWorkerFactory
kernel32!BaseThreadInitThunk
ntdll!__RtlUserThreadStart
ntdll!_RtlUserThreadStart
 
. 11  Id: 16a0.c18 Suspend: 0 Teb: ffa69000 Unfrozen
      Priority: 0  Priority class: 32
  
ntdll!NtDelayExecution
KERNELBASE!Sleep
mysqld!os_thread_sleep(unsigned long tm = 0x186a0) [d:\win32-debug\build\src\storage\xtradb\os\os0thread.cc @ 300]
mysqld!logs_empty_and_mark_files_at_shutdown(void) [d:\win32-debug\build\src\storage\xtradb\log\log0log.cc @ 3673]
mysqld!innobase_shutdown_for_mysql(void) [d:\win32-debug\build\src\storage\xtradb\srv\srv0start.cc @ 3139]
mysqld!innobase_end(struct handlerton * hton = 0x07227068, ha_panic_function type = HA_PANIC_CLOSE (0n0)) [d:\win32-debug\build\src\storage\xtradb\handler\ha_innodb.cc @ 4541]
mysqld!ha_finalize_handlerton(struct st_plugin_int * plugin = 0x071fd270) [d:\win32-debug\build\src\sql\handler.cc @ 452]
mysqld!plugin_deinitialize(struct st_plugin_int * plugin = 0x071fd270, bool ref_check = true) [d:\win32-debug\build\src\sql\sql_plugin.cc @ 1201]
mysqld!reap_plugins(void) [d:\win32-debug\build\src\sql\sql_plugin.cc @ 1279]
mysqld!plugin_shutdown(void) [d:\win32-debug\build\src\sql\sql_plugin.cc @ 1933]
mysqld!clean_up(bool print_message = true) [d:\win32-debug\build\src\sql\mysqld.cc @ 2120]
mysqld!unireg_end(void) [d:\win32-debug\build\src\sql\mysqld.cc @ 2000]
mysqld!kill_server(int sig_ptr = 0n15) [d:\win32-debug\build\src\sql\mysqld.cc @ 1931]
mysqld!handle_shutdown(void * arg = 0x00000000) [d:\win32-debug\build\src\sql\mysqld.cc @ 3575]
mysqld!pfs_spawn_thread(void * arg = 0x0748c830) [d:\win32-debug\build\src\storage\perfschema\pfs.cc @ 1860]
mysqld!pthread_start(void * p = 0x073c1620) [d:\win32-debug\build\src\mysys\my_winthread.c @ 60]
mysqld!invoke_thread_procedure(<function> * procedure = 0x01bd7610, void * context = 0x073c1620) [d:\th\minkernel\crts\ucrt\src\appcrt\startup\thread.cpp @ 92]
mysqld!thread_start<unsigned int (void * parameter = 0x073baa30) [d:\th\minkernel\crts\ucrt\src\appcrt\startup\thread.cpp @ 115]
kernel32!BaseThreadInitThunk
ntdll!__RtlUserThreadStart
ntdll!_RtlUserThreadStart
 
Total threads: 20
Duplicate callstacks: 13 (windbg thread #s follow):
3, 4, 5, 6, 7, 12, 13, 14, 15, 16, 17, 18, 19
quit:



 Comments   
Comment by Marko Mäkelä [ 2017-08-17 ]

This part of the trace makes me think that this could be a duplicate of MDEV-12052:

mysqld!abort
mysqld!buf_flush_page_cleaner_thread

Generated at Thu Feb 08 08:01:21 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.