Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
2.2.21, 2.3.7
-
None
Description
Stopping MaxScale will stop all the running workers(including worker#0), and the exit of worker#0 interrupt the processing of microhttpd API query.
In the resource_handle_request function, microhttpd thread will post a task to worker#0 and wait for the semaphore. But when worker#0 shutdowns without finish processing that semaphore, microhttpd thread will be blocked.
And then the stopping of MaxScale continues, calls MHD_stop_daemon afterwhile, that will call thread_join on microhttpd threads, then dramatically hung by the blocked thread.
You may reproduce the bug in this way:
- First, call any Rest-API in an infinite loop (using Python):
while True: |
requests.get('http://127.0.0.1:30000/v1/services', auth=('admin', 'mariadb')) |
- Second, using maxadmin interface to call 'shutdown maxscale'.
In most cases, the MaxScale process will be hung and stop responding to any request. (It's really easy to reproduce for me)
And here is a sample stack info of a blocked micorhttpd thread:
do_futex_wait.constprop 0x00007ffff7bccafb
__new_sem_wait_slow.constprop.0 0x00007ffff7bccb8f
sem_wait@@GLIBC_2.2.5 0x00007ffff7bccc2b
maxbase::Semaphore::wait semaphore.hh:115
maxbase::Worker::call(std::function<void ()>, maxbase::Worker::execute_mode_t) worker.cc:516
resource_handle_request resource.cc:1337
Client::process admin.cc:126
handle_client admin.cc:266
call_connection_handler connection.c:1834
MHD_connection_handle_idle connection.c:2909
call_handlers daemon.c:1154
MHD_epoll daemon.c:4386
MHD_select_thread daemon.c:4544
start_thread 0x00007ffff7bc6e25
clone 0x00007ffff1e09bad