Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
5.5.28a-galera
-
None
-
Cent OS 5.8 x64
Description
/etc/init.d/mysql stop hangs forever.
Version: '5.5.28a-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 53306 MariaDB Server, wsrep_23.7rc1.rXXXX
130205 6:27:44 [Note] WSREP: Member 1 (devdb03) synced with group.
130205 6:27:44 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 277387)
130205 6:27:44 [Note] WSREP: Synchronized with group, ready for connections
130205 6:27:44 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
130205 6:29:22 [Note] /usr/sbin/mysqld: Normal shutdown
130205 6:29:22 [Note] WSREP: Stop replication
130205 6:29:22 [Note] WSREP: Closing send monitor...
130205 6:29:22 [Note] WSREP: Closed send monitor.
130205 6:29:22 [Note] WSREP: gcomm: terminating thread
130205 6:29:22 [Note] WSREP: gcomm: joining thread
130205 6:29:22 [Note] WSREP: gcomm: closing backend
130205 6:29:23 [Note] WSREP: view(view_id(NON_PRIM,8e7f4cb4-6f5c-11e2-0800-4990a8ab3479,4) memb {
b247ae48-6f5c-11e2-0800-53c97b8485c2,
} joined {
} left {
} partitioned {
8e7f4cb4-6f5c-11e2-0800-4990a8ab3479,
})
130205 6:29:23 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
130205 6:29:23 [Note] WSREP: view((empty))
130205 6:29:23 [Note] WSREP: gcomm: closed
130205 6:29:23 [Note] WSREP: Flow-control interval: [16, 16]
130205 6:29:23 [Note] WSREP: Received NON-PRIMARY.
130205 6:29:23 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 277388)
130205 6:29:23 [Note] WSREP: Received self-leave message.
130205 6:29:23 [Note] WSREP: Flow-control interval: [0, 0]
130205 6:29:23 [Note] WSREP: Received SELF-LEAVE. Closing connection.
130205 6:29:23 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 277388)
130205 6:29:23 [Note] WSREP: RECV thread exiting 0: Success
130205 6:29:23 [Note] WSREP: New cluster view: global state: 34b486ae-6f0c-11e2-0800-5cdefb8ea919:277388, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 2
130205 6:29:23 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
130205 6:29:23 [Note] WSREP: New cluster view: global state: 34b486ae-6f0c-11e2-0800-5cdefb8ea919:277388, view# -1: non-Primary, number of nodes: 0, my index: -1, protocol version 2
130205 6:29:23 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
130205 6:29:23 [Note] WSREP: applier thread exiting (code:0)
130205 6:29:23 [Note] WSREP: recv_thread() joined.
130205 6:29:23 [Note] WSREP: Closing slave action queue.
130205 6:29:23 [Note] WSREP: applier thread exiting (code:5)
Attachments
Issue Links
- links to
Activity
Thank you, Aleksey.
It is totally reproducible with thread pool indeed.
Reproducible on current maria-5.5-galera tree as well (revno 3378).
To reproduce, it's enough to start server with
--wsrep_cluster_address=gcomm:// --wsrep_provider=<path_to_the_library>/libgalera_smm.so --thread-handling=pool-of-threads
wait till it starts, then try to shut it down.
Seppo,
Wlad can consult from the thread pool side, please contact him if needed.
Committed a simple fix, which enables graceful shutdown of wsrep replicator when thread pool scheduler is being used.
We have conflicting work in upstream, which deals with dynamic slave thread management. This fix is therefore quite temporary and will be re-factored latest by when wsrep API #24 will be supported.
Fix was committed in revision: http://bazaar.launchpad.net/~maria-captains/maria/maria-5.5-galera/revision/3379
I think I figured it out. The problem is in the "thread_handling=pool-of-threads" setting. If I remove it then the shutdown goes just fine. With the "thread_handling=pool-of-threads", the unlink_thd() is not called (probably because one_thread_per_connection_scheduler() is not called.
So yes, it was MariaDB specific at the end