[MXS-279] Log manager deadlock Created: 2015-07-20  Updated: 2016-03-03  Resolved: 2016-03-03

Status: Closed
Project: MariaDB MaxScale
Component/s: Core
Affects Version/s: 1.2.0
Fix Version/s: None

Type: Bug Priority: Trivial
Reporter: markus makela Assignee: markus makela
Resolution: Won't Fix Votes: 0
Labels: None


 Description   

The log manager can end up in a deadlock situation when a SIGABRT is received on the log_manager thread.

Here is an example stack trace from gdb:

#0  0x00007fea8d392054 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007fea8d38d388 in _L_lock_854 () from /lib64/libpthread.so.0
#2  0x00007fea8d38d257 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x0000000000b1fe9d in simple_mutex_lock (sm=0x20839d8, block=true) at /home/ec2-user/workspace/utils/skygw_utils.cc:1405
#4  0x00007fea8da11e8f in blockbuf_get_writepos (p_bb=0x7ffffd4b9280, id=LOGFILE_ERROR, str_len=85, flush=true) at /home/ec2-user/workspace/log_manager/log_manager.cc:982
#5  0x00007fea8da115bc in logmanager_write_log(logfile_id_t, bool, bool, bool, bool, size_t, const char *, typedef __va_list_tag __va_list_tag *) (id=LOGFILE_ERROR, flush=true, 
    use_valist=true, spread_down=true, rotate=false, str_len=63, str=0xb22ad8 "Fatal: MaxScale received fatal signal %d. Attempting backtrace.", valist=0x7ffffd4b92e0)
    at /home/ec2-user/workspace/log_manager/log_manager.cc:761
#6  0x00007fea8da1372c in skygw_log_write_flush (id=LOGFILE_ERROR, str=0xb22ad8 "Fatal: MaxScale received fatal signal %d. Attempting backtrace.")
    at /home/ec2-user/workspace/log_manager/log_manager.cc:1410
#7  0x0000000000524d3f in sigfatal_handler (i=6) at /home/ec2-user/workspace/server/core/gateway.c:352
#8  <signal handler called>
#9  0x00007fea8bc478a5 in raise () from /lib64/libc.so.6
#10 0x00007fea8bc49085 in abort () from /lib64/libc.so.6
#11 0x00007fea8bc857b7 in __libc_message () from /lib64/libc.so.6
#12 0x00007fea8bc8b0e6 in malloc_printerr () from /lib64/libc.so.6
#13 0x00007fea8bc8eb64 in _int_malloc () from /lib64/libc.so.6
#14 0x00007fea8bc8f5a6 in calloc () from /lib64/libc.so.6
#15 0x00007fea8da12d2f in blockbuf_init (id=LOGFILE_ERROR) at /home/ec2-user/workspace/log_manager/log_manager.cc:1215
#16 0x00007fea8da12872 in blockbuf_get_writepos (p_bb=0x7ffffd4ba8f0, id=LOGFILE_ERROR, str_len=82, flush=false) at /home/ec2-user/workspace/log_manager/log_manager.cc:1131
#17 0x00007fea8da115bc in logmanager_write_log(logfile_id_t, bool, bool, bool, bool, size_t, const char *, typedef __va_list_tag __va_list_tag *) (id=LOGFILE_ERROR, flush=false, 
    use_valist=true, spread_down=true, rotate=false, str_len=60, str=0x7fea8521a008 "%s: No router options supplied for binlogrouter", valist=0x7ffffd4ba950)
    at /home/ec2-user/workspace/log_manager/log_manager.cc:761
#18 0x00007fea8da13a15 in skygw_log_write (id=LOGFILE_ERROR, str=0x7fea8521a008 "%s: No router options supplied for binlogrouter")
    at /home/ec2-user/workspace/log_manager/log_manager.cc:1473
#19 0x00007fea8520ade0 in createInstance (service=0x21647a0, options=0x0) at /home/ec2-user/workspace/server/modules/routing/binlog/blr.c:371
#20 0x0000000000536021 in serviceStart (service=0x21647a0) at /home/ec2-user/workspace/server/core/service.c:452
#21 0x00000000005362c5 in serviceStartAll () at /home/ec2-user/workspace/server/core/service.c:534
#22 0x000000000052782c in main (argc=2, argv=0x7ffffd4bf438) at /home/ec2-user/workspace/server/core/gateway.c:1759



 Comments   
Comment by markus makela [ 2015-08-06 ]

Lowering priority due to the fact that only fatal system errors can cause this situation and MaxScale currently doesn't handle those kinds of situations very well.

Comment by markus makela [ 2015-09-21 ]

Added missing NULL checks.

Comment by markus makela [ 2015-11-20 ]

Core reason for this was not fixed.

Comment by Johan Wikman [ 2016-03-03 ]

Closing this because to fix this properly, the signal handling needs an overhaul.

MXS-599 has been created for that.

Generated at Thu Feb 08 03:58:07 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.