[MDEV-7126] replication slave - deadlock in terminate_slave_thread with stop slave and show variables of replication filters and show global status Created: 2014-11-18  Updated: 2015-07-23  Resolved: 2015-04-27

Status: Closed
Project: MariaDB Server
Component/s: Replication
Affects Version/s: 5.5.37, 10.0.17
Fix Version/s: 5.5.43

Type: Bug Priority: Critical
Reporter: Daniel Black Assignee: Sergei Golubchik
Resolution: Fixed Votes: 1
Labels: verified
Environment:

rhel

Issue Links:
Relates
relates to MDEV-8527 show slave status, show global status... Closed
relates to MDEV-7667 aborting stop slave doesn't recover Closed
relates to MDEV-8039 parallel replication slave - indefina... Closed

 Description   

On a new 5.5.40 server the replication didn't even start. Databases where created. No replication could start. Identically configured server at same time was created and is replicating ok.

SHOW STATUS of any portion of it fails to return. Show global variables like (any replication filter variable) fails to return.

configuration:

replicate_ignore_table as set to a list of tables. No other replication filters where set.

results from queries:

MariaDB [(none)]> show master status;
 
+------------------------------+----------+--------------+------------------+
 
| File                         | Position | Binlog_Do_DB | Binlog_Ignore_DB |
 
+------------------------------+----------+--------------+------------------+
 
| tulwaurp009-mysql-bin.000001 |      575 |              |                  |
 
+------------------------------+----------+--------------+------------------+
 
1 row in set (0.00 sec)
 
 
 
MariaDB [(none)]> show binlog events in 'tulwaurp009-mysql-bin.000001'
 
    -> ;
 
+------------------------------+-----+-------------+-----------+-------------+---------------------------------------------------------------------------+
 
| Log_name                     | Pos | Event_type  | Server_id | End_log_pos | Info                                                                      |
 
+------------------------------+-----+-------------+-----------+-------------+---------------------------------------------------------------------------+
 
| tulwaurp009-mysql-bin.000001 |   4 | Format_desc |        15 |         245 | Server ver: 5.5.40-MariaDB-log, Binlog ver: 4                             |
 
| tulwaurp009-mysql-bin.000001 | 245 | Query       |        15 |         369 | use `test`; create table  test ( x int unsigned not null, y varchar(300)) |
 
| tulwaurp009-mysql-bin.000001 | 369 | Query       |        15 |         437 | BEGIN                                                                     |
 
| tulwaurp009-mysql-bin.000001 | 437 | Query       |        15 |         548 | use `test`; insert into test values (3,'banana'), (4,'pear')              |
 
| tulwaurp009-mysql-bin.000001 | 548 | Xid         |        15 |         575 | COMMIT /* xid=66401467 */                                                 |
 
+------------------------------+-----+-------------+-----------+-------------+---------------------------------------------------------------------------+

Above create table and insert was made after the replication was started as a test.

SHOW PROCESSLIST

+-------+-------------+---------------------+--------------------+---------+---------+----------------------------------+--------------------------------+----------+
 
| Id    | User        | Host                | db                 | Command | Time    | State                            | Info                           | Progress |
 
+-------+-------------+---------------------+--------------------+---------+---------+----------------------------------+--------------------------------+----------+
 
|     1 | system user |                     | NULL               | Connect | 2171672 | Waiting for master to send event | NULL                           |    0.000 |
 
|     2 | system user |                     | NULL               | Connect | 2044491 | closing tables                   | NULL                           |    0.000 |
 
|  6122 | mysql       | localhost           | NULL               | Query   | 2044491 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6123 | mysql       | localhost           | NULL               | Killed  | 2044491 | Killing slave                    | STOP SLAVE                     |    0.000 |
 
|  6124 | mysql       | 172.16.66.21:49108  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6125 | mysql       | 172.16.66.21:49117  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6126 | mysql       | localhost           | NULL               | Killed  | 2044431 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6127 | mysql       | 172.16.66.21:49124  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6128 | mysql       | localhost           | NULL               | Query   | 2044371 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6129 | mysql       | localhost           | NULL               | Query   | 2044311 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6130 | mysql       | 172.16.66.21:49142  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6131 | mysql       | 172.16.66.21:49189  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6132 | mysql       | localhost           | NULL               | Query   | 2044251 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6133 | mysql       | localhost           | NULL               | Query   | 2044191 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6134 | mysql       | 172.16.66.21:49304  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6135 | mysql       | 172.16.66.21:49311  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6136 | mysql       | localhost           | NULL               | Query   | 2044131 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6137 | mysql       | 172.16.66.21:49318  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6138 | mysql       | localhost           | NULL               | Query   | 2044071 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6139 | mysql       | 172.16.66.21:49328  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6140 | mysql       | localhost           | NULL               | Query   | 2044011 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6141 | mysql       | 172.16.66.21:49385  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6142 | mysql       | localhost           | NULL               | Query   | 2043951 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6143 | mysql       | localhost           | NULL               | Query   | 2043891 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6144 | mysql       | 172.16.66.21:49392  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6145 | mysql       | 172.16.66.21:49401  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6146 | mysql       | localhost           | NULL               | Query   | 2043831 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6147 | mysql       | localhost           | NULL               | Query   | 2043771 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6148 | mysql       | 172.16.66.21:49408  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6149 | mysql       | 172.16.66.21:49418  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6150 | mysql       | localhost           | NULL               | Query   | 2043711 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6151 | mysql       | 172.16.66.21:49473  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6152 | mysql       | localhost           | NULL               | Query   | 2043651 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6153 | mysql       | 172.16.66.21:49480  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6154 | mysql       | localhost           | NULL               | Query   | 2043591 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6155 | mysql       | localhost           | NULL               | Query   | 2043531 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6156 | mysql       | 172.16.66.21:49487  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6157 | mysql       | 172.16.66.21:49602  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6158 | mysql       | localhost           | NULL               | Query   | 2043471 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6159 | mysql       | localhost           | NULL               | Query   | 2043411 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6160 | mysql       | 172.16.66.21:49621  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6161 | mysql       | 172.16.66.21:49669  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6162 | mysql       | localhost           | NULL               | Query   | 2043351 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6163 | mysql       | 172.16.66.21:49676  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6164 | mysql       | localhost           | NULL               | Query   | 2043291 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6165 | mysql       | localhost           | NULL               | Query   | 2043231 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6166 | mysql       | 172.16.66.21:49685  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6167 | mysql       | 172.16.66.21:49692  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6168 | mysql       | localhost           | NULL               | Query   | 2043171 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6169 | mysql       | 172.16.66.21:49707  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6170 | mysql       | localhost           | NULL               | Query   | 2043111 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6171 | mysql       | 172.16.66.21:49757  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6172 | mysql       | localhost           | NULL               | Query   | 2043051 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6173 | mysql       | 172.16.66.21:49764  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6174 | mysql       | localhost           | NULL               | Query   | 2042991 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6175 | mysql       | 172.16.66.21:49771  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6176 | mysql       | localhost           | NULL               | Query   | 2042931 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6177 | mysql       | 172.16.66.21:49778  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6178 | mysql       | localhost           | NULL               | Query   | 2042871 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6179 | mysql       | 172.16.66.21:49793  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6180 | mysql       | localhost           | NULL               | Query   | 2042811 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6181 | mysql       | 172.16.66.21:49845  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6182 | mysql       | localhost           | NULL               | Query   | 2042751 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6183 | mysql       | localhost           | NULL               | Query   | 2042691 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6184 | mysql       | 172.16.66.21:49854  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6185 | mysql       | 172.16.66.21:49969  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6186 | mysql       | localhost           | NULL               | Query   | 2042631 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6187 | mysql       | 172.16.66.21:49976  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6188 | mysql       | localhost           | NULL               | Query   | 2042571 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6189 | mysql       | localhost           | NULL               | Query   | 2042511 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6190 | mysql       | 172.16.66.21:49993  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6191 | mysql       | 172.16.66.21:50042  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6192 | mysql       | localhost           | NULL               | Query   | 2042451 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6193 | mysql       | localhost           | NULL               | Query   | 2042391 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6194 | mysql       | 172.16.66.21:50049  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6195 | mysql       | localhost           | NULL               | Query   | 2042331 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6196 | mysql       | 172.16.66.21:50060  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6197 | mysql       | localhost           | NULL               | Query   | 2042271 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6198 | mysql       | 172.16.66.21:50171  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6199 | mysql       | localhost           | NULL               | Query   | 2042211 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6200 | mysql       | 172.16.66.21:50190  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6201 | mysql       | localhost           | NULL               | Query   | 2042151 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6202 | mysql       | 172.16.66.21:50238  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6203 | mysql       | localhost           | NULL               | Query   | 2042091 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6204 | mysql       | 172.16.66.21:50245  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6205 | mysql       | localhost           | NULL               | Query   | 2042031 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6206 | mysql       | 172.16.66.21:50254  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6207 | mysql       | localhost           | NULL               | Query   | 2041971 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6208 | mysql       | 172.16.66.21:50261  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6209 | mysql       | localhost           | NULL               | Query   | 2041911 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6210 | mysql       | 172.16.66.21:50273  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6211 | mysql       | localhost           | NULL               | Query   | 2041851 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6212 | mysql       | 172.16.66.21:50338  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6213 | mysql       | localhost           | NULL               | Query   | 2041791 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6214 | mysql       | 172.16.66.21:50441  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6215 | mysql       | localhost           | NULL               | Query   | 2041731 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6216 | mysql       | 172.16.66.21:50448  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6217 | mysql       | localhost           | NULL               | Query   | 2041671 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6218 | mysql       | 172.16.66.21:50455  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6219 | mysql       | localhost           | NULL               | Query   | 2041611 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6220 | mysql       | 172.16.66.21:50471  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6221 | mysql       | localhost           | NULL               | Query   | 2041551 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6222 | mysql       | 172.16.66.21:50522  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6223 | mysql       | localhost           | NULL               | Query   | 2041491 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6224 | mysql       | 172.16.66.21:50529  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6225 | mysql       | localhost           | NULL               | Query   | 2041431 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6226 | mysql       | 172.16.66.21:50538  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6227 | mysql       | localhost           | NULL               | Query   | 2041371 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6228 | mysql       | 172.16.66.21:50545  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6229 | mysql       | localhost           | NULL               | Query   | 2041311 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6230 | mysql       | 172.16.66.21:50558  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6231 | mysql       | localhost           | NULL               | Query   | 2041251 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6232 | mysql       | 172.16.66.21:50610  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6233 | mysql       | localhost           | NULL               | Query   | 2041191 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6234 | mysql       | 172.16.66.21:50617  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6235 | mysql       | localhost           | NULL               | Query   | 2041131 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6236 | mysql       | 172.16.66.21:50630  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6237 | mysql       | localhost           | NULL               | Query   | 2041071 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6238 | mysql       | 172.16.66.21:50739  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6239 | mysql       | localhost           | NULL               | Query   | 2041011 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6240 | mysql       | 172.16.66.21:50761  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6241 | mysql       | localhost           | NULL               | Query   | 2040951 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6242 | mysql       | 172.16.66.21:50804  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6243 | mysql       | localhost           | NULL               | Query   | 2040891 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6244 | mysql       | localhost           | NULL               | Query   | 2040891 | Filling schema table             | show status                    |    0.000 |
 
|  6245 | mysql       | 172.16.66.21:50811  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6246 | mysql       | localhost           | NULL               | Query   | 2040831 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6247 | mysql       | 172.16.66.21:50823  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6248 | mysql       | localhost           | NULL               | Query   | 2040771 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6249 | mysql       | 172.16.66.21:50830  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6250 | mysql       | localhost           | NULL               | Query   | 2040711 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6251 | mysql       | 172.16.66.21:50845  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6252 | mysql       | localhost           | NULL               | Query   | 2040651 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6253 | mysql       | 172.16.66.21:50913  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6254 | mysql       | 172.16.66.21:51010  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6255 | mysql       | localhost           | NULL               | Query   | 2040591 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6256 | mysql       | localhost           | NULL               | Query   | 2040531 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6257 | mysql       | 172.16.66.21:51017  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6258 | mysql       | localhost           | NULL               | Query   | 2040471 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6259 | mysql       | 172.16.66.21:51024  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6260 | mysql       | 172.16.66.21:51039  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6261 | mysql       | localhost           | NULL               | Query   | 2040411 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6262 | mysql       | localhost           | NULL               | Query   | 2040351 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6263 | mysql       | 172.16.66.21:51089  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6264 | mysql       | 172.16.66.21:51096  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6265 | mysql       | localhost           | NULL               | Query   | 2040291 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6266 | mysql       | localhost           | NULL               | Query   | 2040231 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6267 | mysql       | 172.16.66.21:51105  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
|  6268 | mysql       | localhost           | NULL               | Query   | 2040171 | Filling schema table             | show /*!50002 GLOBAL */ status |    0.000 |
 
|  6269 | mysql       | 172.16.66.21:51112  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
| 14881 | mysql       | 172.16.66.21:39304  | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
| 14882 | percona     | 172.17.100.40:49399 | NULL               | Killed  | 1402484 | NULL                             | SHOW SLAVE STATUS              |    0.000 |
 
| 14883 | root        | localhost           | NULL               | Killed  |  465087 | NULL                             | show slave status              |    0.000 |
 
| 14884 | root        | localhost           | NULL               | Killed  |  465043 | NULL                             | NULL                           |    0.000 |
 
| 14885 | root        | localhost           | NULL               | Killed  |  465037 | NULL                             | NULL                           |    0.000 |
 
| 14886 | root        | localhost           | NULL               | Killed  |  465028 | NULL                             | NULL                           |    0.000 |
 
| 14887 | root        | localhost           | NULL               | Killed  |  464998 | NULL                             | show slave status              |    0.000 |
 
| 14888 | root        | localhost           | NULL               | Killed  |  464976 | NULL                             | NULL                           |    0.000 |
 
| 14889 | root        | localhost           | NULL               | Killed  |  464971 | NULL                             | NULL                           |    0.000 |
 
| 14890 | root        | localhost           | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
| 14891 | root        | localhost           | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
| 14892 | root        | localhost           | NULL               | Killed  |    NULL | login                            | NULL                           |    0.000 |
 
| 14893 | root        | localhost           | NULL               | Killed  |  357241 | NULL                             | show slave status              |    0.000 |
 
| 14894 | root        | 172.16.66.21:47981  | NULL               | Killed  |    3529 | NULL                             | show slave status              |    0.000 |
 
| 14895 | root        | 172.16.66.21:48146  | NULL               | Killed  |    3370 | NULL                             | NULL                           |    0.000 |
 
| 14896 | root        | 172.16.66.21:48147  | NULL               | Killed  |    3368 | NULL                             | NULL                           |    0.000 |
 
| 14897 | root        | 172.16.66.21:48148  | NULL               | Killed  |    3082 | Filling schema table             | show global variables          |    0.000 |
 
| 14898 | root        | 172.16.66.21:48207  | NULL               | Killed  |    3080 | NULL                             | NULL                           |    0.000 |
 
| 14899 | root        | 172.16.66.21:48208  | NULL               | Killed  |    3079 | NULL                             | NULL                           |    0.000 |
 
| 14900 | root        | 172.16.66.21:48209  | NULL               | Killed  |    3075 | Filling schema table             | show variables                 |    0.000 |
 
| 14901 | root        | 172.16.66.21:48319  | NULL               | Killed  |    3029 | NULL                             | NULL                           |    0.000 |
 
| 14902 | root        | 172.16.66.21:48320  | NULL               | Killed  |    3028 | NULL                             | NULL                           |    0.000 |
 
| 14903 | root        | 172.16.66.21:48321  | NULL               | Killed  |    2948 | Filling schema table             | show status                    |    0.000 |
 
| 14904 | root        | 172.16.66.21:48324  | NULL               | Killed  |    2877 | NULL                             | NULL                           |    0.000 |
 
| 14905 | root        | 172.16.66.21:48327  | NULL               | Killed  |    2872 | NULL                             | NULL                           |    0.000 |
 
| 14906 | root        | 172.16.66.21:48329  | information_schema | Killed  |    2423 | NULL                             | NULL                           |    0.000 |
 
| 14907 | root        | 172.16.66.21:48441  | NULL               | Killed  |    1746 | NULL                             | show slave status              |    0.000 |
 
| 14908 | root        | 172.16.66.21:48667  | NULL               | Killed  |    1745 | NULL                             | NULL                           |    0.000 |
 
| 14909 | root        | 172.16.66.21:48669  | NULL               | Killed  |    1727 | NULL                             | NULL                           |    0.000 |
 
| 14910 | root        | 172.16.66.21:48670  | NULL               | Killed  |    1715 | NULL                             | NULL                           |    0.000 |
 
| 14911 | root        | 172.16.66.21:48671  | mysql              | Killed  |     199 | NULL                             | NULL                           |    0.000 |
 
| 14912 | root        | 172.16.66.21:49289  | NULL               | Killed  |     174 | NULL                             | NULL                           |    0.000 |
 
| 14913 | root        | 172.16.66.21:49291  | NULL               | Query   |       0 | NULL                             | show processlist               |    0.000 |

(several 1000 killed login and duplicate processes in same status removed)

show databases worked.

I am able to query databases including mysql database.

All of these queries returned results:

MariaDB [information_schema]> select * from KEY_CACHES;
 
+----------------+----------+----------------+-----------+------------+-------------+---------------+--------------+---------------+-------+----------------+--------+
 
| KEY_CACHE_NAME | SEGMENTS | SEGMENT_NUMBER | FULL_SIZE | BLOCK_SIZE | USED_BLOCKS | UNUSED_BLOCKS | DIRTY_BLOCKS | READ_REQUESTS | READS | WRITE_REQUESTS | WRITES |
 
+----------------+----------+----------------+-----------+------------+-------------+---------------+--------------+---------------+-------+----------------+--------+
 
| default        |     NULL |           NULL | 209715200 |       1024 |         177 |        166470 |            0 |          7253 |   171 |           1745 |   1734 |
 
+----------------+----------+----------------+-----------+------------+-------------+---------------+--------------+---------------+-------+----------------+--------+
 
1 row in set (0.00 sec)
 
 
 
MariaDB [information_schema]> select * from VIEWS;
 
Empty set (0.03 sec)
 
 
 
 
 
MariaDB [information_schema]> select @@max_connections;
 
+-------------------+
 
| @@max_connections |
 
+-------------------+
 
|              3200 |
 
+-------------------+
 

MariaDB [information_schema]> show variables like '%conn%';
 
Current database: information_schema
 
 
 
+--------------------------+-----------------+
 
| Variable_name            | Value           |
 
+--------------------------+-----------------+
 
| character_set_connection | utf8            |
 
| collation_connection     | utf8_general_ci |
 
| connect_timeout          | 10              |
 
| extra_max_connections    | 1               |
 
| init_connect             |                 |
 
| max_connect_errors       | 10              |
 
| max_connections          | 3200            |
 
| max_user_connections     | 0               |
 
+--------------------------+-----------------+
 
8 rows in set (0.00 sec)
 
 
 
MariaDB [information_schema]> show variables like '%binlog%';
 
| binlog_annotate_row_events              | ON                   |
 
| binlog_cache_size                       | 32768                |
 
| binlog_checksum                         | NONE                 |
 
| binlog_direct_non_transactional_updates | OFF                  |
 
| binlog_format                           | MIXED                |
 
| binlog_optimize_thread_scheduling       | ON                   |
 
| binlog_stmt_cache_size                  | 32768                |
 
| innodb_locks_unsafe_for_binlog          | OFF                  |
 
| max_binlog_cache_size                   | 18446744073709547520 |
 
| max_binlog_size                         | 1073741824           |
 
| max_binlog_stmt_cache_size              | 18446744073709547520 |
 
| sync_binlog                             | 0                    |
 
 
 
 
 
MariaDB [(none)]> show global variables like 'Aria%';
 
...
 
MariaDB [(none)]> show global variables like 'auto%';
 
...
 
 
 
MariaDB [(none)]> show global variables like 'back%';
 
+---------------+-------+
 
| Variable_name | Value |
 
+---------------+-------+
 
| back_log      | 50    |
 
+---------------+-------+
 
 
 
MariaDB [(none)]> show global variables like 'character%';
 
...
 
MariaDB [(none)]> show global variables like 'col%';
 
.
 
 
 
MariaDB [(none)]> show global variables like 'con%';
 
...
 
 
 
MariaDB [(none)]> show global variables like 'da%';
 
 
 
 
 
MariaDB [(none)]> show global variables like 'de%';
 
 
 
show global variables like 'd%';
 
..
 
show global variables like 'e%';
 
.....
 
show global variables like 'q%';
 
 
 
show global variables like 'ra%';
 
 
 
+------------------------+-------+
 
| Variable_name          | Value |
 
+------------------------+-------+
 
| range_alloc_block_size | 4096  |
 
 
 
show global variables like 'read%';
 
 
 
MariaDB [(none)]> show global variables like 'relay%';
 
+-----------------------+-----------------------+
 
| Variable_name         | Value                 |
 
+-----------------------+-----------------------+
 
| relay_log             | tulwaurp009-relay-bin |
 
| relay_log_index       |                       |
 
| relay_log_info_file   | relay-log.info        |
 
| relay_log_purge       | ON                    |
 
| relay_log_recovery    | OFF                   |
 
| relay_log_space_limit | 0                     |
 
+-----------------------+-----------------------+
 
 
 
MariaDB [(none)]> show global variables like 'replicate_a%';
 
 
 
+-------------------------------+-------+
 
| Variable_name                 | Value |
 
+-------------------------------+-------+
 
| replicate_annotate_row_events | ON    |
 
+-------------------------------+-------+
 
1 row in set (0.00 sec)
 
 
 
show global variables like 'report%';
 
| Variable_name   | Value       |
 
+-----------------+-------------+
 
| report_host     | tulwaurp009 |
 
| report_password |             |
 
| report_port     | 3306        |
 
| report_user     |             |
 
 
 
show global variables like 'ro%';
 
show global variables like 's%';
 
show global variables like 't%';
 
....
 
show global variables like 'z%';

no response:

  • show variables like '%table%';
  • show global variables like 'r%';
  • show global variables like 're%';
  • show global variables like 'repl%';
  • show global variables like 'replicate_d%';
  • show global variables like 'replicate_do_db';
  • show global variables like 'replicate_do_table;
  • show global variables like 'replicate_i%';
  • show global status like 'Aria%';
  • show global status like 'Aria_pagecache%';
  • show global status like 'Aria_trans%';
  • show global status like 'Com_%';
  • show global status like 'Bytes_%';
  • show global status like 'Busy_%';
  • show global status like 'x%';
  • show status
  • show global status
  • show variables
  • show global variables
  • select * from information_schema.session_status;
  • select * from information_schema.session_status limit 3;

 Comments   
Comment by Elena Stepanova [ 2014-11-18 ]

Hi Daniel,

Technically replication did start, we can see both slave threads in the processlist.
But apparently the SQL thread hang on "closing tables".
Possibly it happened on the very first event, I suppose that's what you meant by "replication didn't even start".

Is this instance still up hanging? If so, can you get a stack trace from the running process?

If the instance was shutdown, then – is it reproducible? Does it hang again when you start it again?

Did you check the obvious, like the disk space or IO errors? Closing tables has to do with writing to the disk.

Could you please also attach the error log from the server startup and up to the time it definitely hang?

Comment by Daniel Black [ 2014-11-26 ]

> Is this instance still up hanging?

yes.

> If so, can you get a stack trace from the running process?

I couldn't

>If the instance was shutdown, then – is it reproducible? Does it hang again when you start it again?

no.

>Did you check the obvious, like the disk space or IO errors?

Space there and no io errors.

> Could you please also attach the error log from the server startup and up to the time it definitely hang?

141024 13:07:15 mysqld_safe Starting mysqld daemon with databases from /u01/mariadb
 
141024 13:07:15 InnoDB: The InnoDB memory heap is disabled
 
141024 13:07:15 InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
141024 13:07:15 InnoDB: Compressed tables use zlib 1.2.3
 
141024 13:07:15 InnoDB: Using Linux native AIO
 
141024 13:07:15 InnoDB: Initializing buffer pool, size = 24.0G
 
141024 13:07:17 InnoDB: Completed initialization of buffer pool
 
141024 13:07:17 InnoDB: highest supported file format is Barracuda.
 
141024 13:07:19 InnoDB: Restoring buffer pool pages from ib_lru_dump
 
141024 13:07:19  InnoDB: Waiting for the background threads to start
 
141024 13:07:20 Percona XtraDB (http://www.percona.com) 5.5.40-MariaDB-36.1 started; log sequence number 3338597042050
 
141024 13:07:20 [Note] Plugin 'FEEDBACK' is disabled.
 
141024 13:07:20 [Note] Server socket created on IP: '0.0.0.0'.
 
141024 13:07:20 [Note] Slave SQL thread initialized, starting replication in log 'tulwaurp005-mysql-bin.002166' at position 11843844, relay log './tulwaurp007-relay-bin.005152' positio
 
n: 11728905
 
141024 13:07:20 [Note] Event Scheduler: Loaded 0 events
 
141024 13:07:20 [Note] /usr/sbin/mysqld: ready for connections.
 
Version: '5.5.40-MariaDB-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Server
 
141024 13:07:20 [Note] Slave I/O thread: connected to master 'repl@tulwaurp005:3306',replication started in log 'tulwaurp005-mysql-bin.002166' at position 11843844
 
141024 13:08:16 InnoDB: Completed reading buffer pool pages (requested: 1568414, read: 1561319)
 
141026  0:27:01 [Note] Error reading relay log event: slave SQL thread was killed
 
141126 10:51:20 [Note] Slave I/O thread killed while reading event
 
141126 10:51:20 [Note] Slave I/O thread exiting, read up to log 'tulwaurp005-mysql-bin.002329', position 762362495
 
141126 10:51:20 [Note] /usr/sbin/mysqld: Normal shutdown
 
 
 
141126 10:51:20 [Note] Event Scheduler: Purging the queue. 0 events

Happy to write this off as MDEV-7026

Comment by Elena Stepanova [ 2014-11-26 ]

Okay then, I think at this point it's reasonable to blame MDEV-7026, especially since we don't have much else here. Lets wait and see whether a) it happens again (if it does, maybe you can get a stack trace?), and b) if it goes away after we release 5.5.41 and you upgrade to it.
I will close it for now as a duplicate of MDEV-7026, but please feel free to comment if you have any new information, and we'll re-open if necessary.

Comment by Elena Stepanova [ 2014-11-26 ]

Hopefully fixed along with MDEV-7026

Comment by Daniel Black [ 2015-01-30 ]

This also re-occured on 5.5.39 instance (same machine).

Id	User	Host	db	Command	Time	State	Info	Progress
 
6699	system user		NULL	Connect	3488030	Queueing master event to the relay log	NULL	0.000
 
6700	system user		NULL	Connect	2883230	closing tables	NULL	0.000
 
26870	mysql	localhost	NULL	Query	2883230	Filling schema table	show /*!50002 GLOBAL */ status	0.000
 
26871	mysql	localhost	NULL	Query	2883230	Killing slave	STOP SLAVE	0.000
 
26872	mysql	localhost	NULL	Query	2883170	Filling schema table	show /*!50002 GLOBAL */ status	0.000
 
26873	mysql	localhost	NULL	Query	2883110	Filling schema table	show /*!50002 GLOBAL */ status	0.000

The show global status at the same second as the stop slave was consistent.

2883230 seconds from the STOP SLAVE in process list corresponds to 33 days go on Dec 28.

This machine was out of disk space however this occurred on 28 January as the SQL IO thread was still running and the relay logs filled the disk up.

I can't see any fixed to race conditions in replication in the 5.5.40 or 5.5.41 change logs.

I will try to reproduce with a 5.5.41 instance.

Comment by Arjen Lentz [ 2015-01-30 ]

apparently not yet fixed, as per Daniel Black's latest reports.

Comment by Daniel Black [ 2015-01-30 ]

This is a snapshot from the log file of system stats: tulwaurp009_GenInfo_20141228_0127.log.bz2 at 0127.

mysql    55574 55569  0 00:27 ?        00:00:00 /bin/sh -c $HOME/bin/mysql_stat.sh
 
mysql    55576 55570  0 00:27 ?        00:00:00 /bin/sh -c $HOME/bin/mysql_backup.sh
 
mysql    55586 55574  0 00:27 ?        00:00:00 mysqladmin extended-status
 
mysql    55588 55576  0 00:27 ?        00:00:00 mysqladmin stop-slave
 
mysql    55589 55570  0 00:27 ?        00:00:00 /usr/sbin/sendmail -FCronDaemon -i -odi -oem -oi -t -f root
 
mysql    55591 55589  0 00:27 ?        00:00:00 /usr/sbin/postdrop -r
 
root     55593  3528  0 00:28 ?        00:00:00 CROND
 
mysql    55595 55593  0 00:28 ?        00:00:00 /bin/sh -c $HOME/bin/mysql_stat.sh
 
mysql    55600 55595  0 00:28 ?        00:00:00 mysqladmin extended-status
 
root     55605  3528  0 00:29 ?        00:00:00 CROND
 
mysql    55607 55605  0 00:29 ?        00:00:00 /bin/sh -c $HOME/bin/mysql_stat.sh
 
mysql    55612 55607  0 00:29 ?        00:00:00 mysqladmin extended-status
 
root     55616  3528  0 00:30 ?        00:00:00 CROND
 
mysql    55620 55616  0 00:30 ?        00:00:00 /bin/sh -c $HOME/bin/mysql_stat.sh
 
mysql    55625 55620  0 00:30 ?        00:00:00 mysqladmin extended-status

As mysqladmin extended-status/stop-slave don't take an hour to run it points to a deadlock there. Also the minute /mysql_stat.sh scripts that runs the following:

mysqladmin extended-status > $logfile
 
 
 
#
 
# Check that the slave is running
 
if test `grep 'Slave_running' $logfile | grep -c 'ON'` -eq 0
 
then
 
  #
 
  # get some diagnostics
 
  mysql --batch <<EOT >> $logfile 2>&1
 
select sysdate()\\G
 
show innodb status\\G
 
show processlist\\G
 
show slave status\\G
 
EOT
 
...

Comment by Elena Stepanova [ 2015-02-08 ]

Was there anything else in the process list apart from the quoted 6 lines and apparently countless other SHOW GLOBAL STATUS if it kept being issued every 60 seconds – any other SQL queries?

There were a couple deadlocks fixed in 5.5.40, but both involve several types of statements executed at once:
MDEV-6774 (involves SHOW STATUS as well)
MDEV-6749 (less chance it's relevant, but still)

Comment by Daniel Black [ 2015-02-10 ]

> any other SQL queries?
many days later there is the following at the end of the show global status.

27116	mysql	localhost	NULL	Query	546830	Filling schema table	show status	0.000
 
27117	mysql	localhost	NULL	Query	464030	NULL	STOP SLAVE	0.000
 
27118	mysql	localhost	NULL	Query	460429	Filling schema table	show status	0.000
 
27119	mysql	localhost	NULL	Query	374030	Filling schema table	show status	0.000
 
27120	mysql	localhost	NULL	Query	287629	Filling schema table	show status	0.000
 
27121	mysql	localhost	NULL	Query	201230	Filling schema table	show status	0.000
 
27122	mysql	localhost	NULL	Query	114830	Filling schema table	show status	0.000
 
27123	mysql	localhost	NULL	Query	28430	Filling schema table	show status	0.000
 
27124	root	localhost	NULL	Killed	NULL	login	NULL	0.000
 
27125	munin	localhost	mysql	Killed	428	NULL	NULL	0.000
 
27126	munin	localhost	mysql	Query	428	Filling schema table	SHOW GLOBAL STATUS	0.000

> a couple deadlocks fixed in 5.5.40

The original bug report before reopening was on 5.5.40. Have reasked client for permission to test .41/.42

Comment by Daniel Black [ 2015-02-10 ]

5.5.41

 sudo cat /var/lib/mysql/mysqld.log
 
150210 15:50:32 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
 
150210 15:50:34 mysqld_safe Starting mysqld daemon with databases from /u01/mariadb
 
150210 15:50:34 InnoDB: The InnoDB memory heap is disabled
 
150210 15:50:34 InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
150210 15:50:34 InnoDB: Compressed tables use zlib 1.2.3
 
150210 15:50:34 InnoDB: Using Linux native AIO
 
150210 15:50:34 InnoDB: Initializing buffer pool, size = 24.0G
 
150210 15:50:35 InnoDB: Completed initialization of buffer pool
 
150210 15:50:35 InnoDB: highest supported file format is Barracuda.
 
150210 15:50:35 InnoDB: Restoring buffer pool pages from ib_lru_dump
 
150210 15:50:35  InnoDB: Waiting for the background threads to start
 
150210 15:50:36 Percona XtraDB (http://www.percona.com) 5.5.40-MariaDB-36.1 started; log sequence number 4168206481196
 
150210 15:50:37 [Note] Plugin 'FEEDBACK' is disabled.
 
150210 15:50:37 [Note] Server socket created on IP: '0.0.0.0'.
 
150210 15:50:37 [Note] Slave SQL thread initialized, starting replication in log 'tulwaurp005-mysql-bin.002754' at position 635777838, relay log './tulwaurp009-relay-bin.000164' position: 615300928
 
150210 15:50:37 [Note] Event Scheduler: Loaded 0 events
 
150210 15:50:37 [Note] /usr/sbin/mysqld: ready for connections.
 
Version: '5.5.41-MariaDB-log'  socket: '/u01/mariadb/mysql.sock'  port: 3306  MariaDB Server
 
150210 15:50:37 [Note] Slave I/O thread: connected to master 'repl@tulwaurp005:3306',replication started in log 'tulwaurp005-mysql-bin.002754' at position 635777838
 
150210 15:51:21 InnoDB: Completed reading buffer pool pages (requested: 1464750, read: 1461467)
 
150210 15:54:02 [Note] Error reading relay log event: slave SQL thread was killed
 
150210 15:54:02 [ERROR] Error reading packet from server: Lost connection to MySQL server during query ( server_errno=2013)
 
150210 15:54:02 [Note] Slave I/O thread killed while reading event
 
150210 15:54:02 [Note] Slave I/O thread exiting, read up to log 'tulwaurp005-mysql-bin.002754', position 645162051
 

crontab -l
 
# MDEV-6774 test
 
* * * * * mysql -e 'show global status' > /dev/null 2>&1
 
* * * * * mysql -e 'stop slave; start slave;'  > /dev/null 2>&1

immediately got to the state:

| Id | User        | Host      | db    | Command | Time | State                           | Info               | Progress |
 
+----+-------------+-----------+-------+---------+------+---------------------------------+--------------------+----------+
 
|  1 | system user |           | NULL  | Connect |  509 | Waiting for slave mutex on exit | NULL               |    0.000 |
 
|  8 | openquery   | localhost | NULL  | Query   |  304 | Flushing relay-log info file.   | stop slave         |    0.000 |
 
|  9 | openquery   | localhost | NULL  | Query   |  304 | Filling schema table            | show global status |    0.000 |
 
| 10 | openquery   | localhost | NULL  | Killed  |  296 | NULL                            | stop slave         |    0.000 |
 
| 11 | openquery   | localhost | NULL  | Killed  |  171 | NULL                            | show slave status  |    0.000 |
 
| 12 | openquery   | localhost | NULL  | Query   |  245 | Filling schema table            | show global status |    0.000 |
 
| 13 | openquery   | localhost | NULL  | Query   |  245 | NULL                            | stop slave         |    0.000 |
 
| 14 | munin       | localhost | mysql | Query   |  243 | Filling schema table            | SHOW GLOBAL STATUS |    0.000 |
 
| 15 | openquery   | localhost | NULL  | Query   |  185 | Filling schema table            | show global status |    0.000 |
 
| 16 | openquery   | localhost | NULL  | Query   |  185 | NULL                            | stop slave         |    0.000 |
 
| 17 | munin       | localhost | mysql | Query   |  180 | Filling schema table            | SHOW GLOBAL STATUS |    0.000 |
 
| 18 | openquery   | localhost | NULL  | Killed  |  155 | NULL                            | NULL               |    0.000 |
 
| 19 | openquery   | localhost | NULL  | Killed  |  153 | NULL                            | NULL               |    0.000 |
 
| 20 | openquery   | localhost | NULL  | Killed  |  152 | NULL                            | show slave status  |    0.000 |
 
| 21 | openquery   | localhost | NULL  | Killed  |  149 | NULL                            | NULL               |    0.000 |
 
| 22 | openquery   | localhost | NULL  | Killed  |  147 | NULL                            | NULL               |    0.000 |
 
| 23 | openquery   | localhost | NULL  | Killed  |  106 | NULL                            | show slave status  |    0.000 |
 
| 24 | openquery   | localhost | NULL  | Query   |  125 | NULL                            | stop slave         |    0.000 |
 
| 25 | openquery   | localhost | NULL  | Query   |  125 | Filling schema table            | show global status |    0.000 |
 
| 26 | munin       | localhost | mysql | Query   |  118 | Filling schema table            | SHOW GLOBAL STATUS |    0.000 |
 
| 27 | openquery   | localhost | NULL  | Killed  |  110 | NULL                            | NULL               |    0.000 |
 
| 28 | openquery   | localhost | NULL  | Killed  |  109 | NULL                            | NULL               |    0.000 |
 
| 29 | openquery   | localhost | NULL  | Killed  |  104 | NULL                            | NULL               |    0.000 |
 
| 30 | openquery   | localhost | NULL  | Killed  |  103 | NULL                            | NULL               |    0.000 |
 
| 31 | openquery   | localhost | NULL  | Killed  |   90 | NULL                            | show slave status  |    0.000 |
 
| 32 | openquery   | localhost | NULL  | Killed  |   88 | NULL                            | NULL               |    0.000 |
 
| 33 | openquery   | localhost | NULL  | Killed  |   87 | NULL                            | NULL               |    0.000 |
 
| 34 | openquery   | localhost | NULL  | Query   |   65 | NULL                            | stop slave         |    0.000 |
 
| 35 | openquery   | localhost | NULL  | Query   |   65 | Filling schema table            | show global status |    0.000 |
 
| 36 | munin       | localhost | mysql | Query   |   56 | Filling schema table            | SHOW GLOBAL STATUS |    0.000 |
 
| 37 | openquery   | localhost | NULL  | Query   |    5 | Filling schema table            | show global status |    0.000 |
 
| 38 | openquery   | localhost | NULL  | Query   |    5 | NULL                            | stop slave         |    0.000 |
 
| 39 | openquery   | localhost | NULL  | Query   |    0 | NULL                            | show processlist   |    0.000 |
 

(gdb) bt full
 
#0  0x0000003f8d0df293 in poll () from /lib64/libc.so.6
 
No symbol table info available.
 
#1  0x0000000000523653 in handle_connections_sockets() ()
 
No symbol table info available.
 
#2  0x00000000005273a2 in mysqld_main(int, char**) ()
 
No symbol table info available.
 
#3  0x0000003f8d01ecdd in __libc_start_main () from /lib64/libc.so.6
 
No symbol table info available.
 
#4  0x000000000051d7e9 in _start ()
 
No symbol table info available.
 
(gdb) quit

Comment by Elena Stepanova [ 2015-02-11 ]

I can reproduce it on MariaDB 5.5, but not on 10.0 (and not on MySQL 5.5/5.6).

Test case:

--source include/master-slave.inc
 
 
 
--connection slave
 
--echo # If everything is okay, the test will end in several seconds; maybe a minute. 
--echo # If the problem shows up, it will hang until testcase timeout is exceeded.
 
--exec $MYSQL_SLAP --socket=$SLAVE_MYSOCK -q "START SLAVE; STOP SLAVE; SHOW GLOBAL STATUS" -c 2 --number-of-queries=100 --create-schema=test
 
--echo # All done.

The problem appeared in 5.5.37 with the following revision:

revno: 4110
 
revision-id: monty@mariadb.org-20140314142923-wvho2wb4i8s7wb1v
 
parent: svoj@mariadb.org-20140213074049-wo2l3qdtgi0s2mjd
 
committer: Michael Widenius <monty@mariadb.org>
 
branch nick: maria-5.5
 
timestamp: Fri 2014-03-14 16:29:23 +0200
 
message:
 
  MDEV-5829: STOP SLAVE resets global status variables
 
  
  Reason for the bug was an optimization for higher connect speed where we moved when global status was updated,
 
  but forgot to update states when slave thread dies.
 
  Fixed by adding thd->add_status_to_global() before deleting slave thread's thd.

Comment by Sergei Golubchik [ 2015-04-23 ]

danblack, I'm only fixing 5.5 bug here. 10.0 deadlock is different and needs a separate bug report. Could you move it there, please?

Comment by Daniel Black [ 2015-04-23 ]

Could you move it there, please?

ack - MDEV-8039

Comment by Daniel Black [ 2015-04-23 ]

re-adding over-culled comment:

MariaDB [(none)]> show processlist;
 
+----+-------------+-----------+------+---------+------+----------------------------------+--------------------+----------+
 
| Id | User        | Host      | db   | Command | Time | State                            | Info               | Progress |
 
+----+-------------+-----------+------+---------+------+----------------------------------+--------------------+----------+
 
|  3 | openquery   | localhost | NULL | Query   |    0 | NULL                             | show processlist   |    0.000 |
 
| 38 | system user |           | NULL | Connect |   35 | Waiting for master to send event | NULL               |    0.000 |
 
| 39 | system user |           | NULL | Connect |   31 | closing tables                   | NULL               |    0.000 |
 
| 40 | openquery   | localhost | NULL | Query   |   31 | Filling schema table             | show global status |    0.000 |
 
| 41 | openquery   | localhost | NULL | Query   |   31 | Killing slave                    | stop slave         |    0.000 |
 
+----+-------------+-----------+------+---------+------+----------------------------------+--------------------+----------+
 
5 rows in set (0.00 sec)
 

(gdb) thread apply all bt
 
 
 
Thread 48 (Thread 0x7f2cc83ff700 (LWP 51655)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 47 (Thread 0x7f2642bff700 (LWP 51656)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 46 (Thread 0x7f26421fe700 (LWP 51657)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 45 (Thread 0x7f26417fd700 (LWP 51658)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 44 (Thread 0x7f2640dfc700 (LWP 51659)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 43 (Thread 0x7f26403fb700 (LWP 51660)):
 
---Type <return> to continue, or q <return> to quit---
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 42 (Thread 0x7f263f9fa700 (LWP 51661)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 41 (Thread 0x7f263eff9700 (LWP 51662)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 40 (Thread 0x7f263e5f8700 (LWP 51663)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 39 (Thread 0x7f263dbf7700 (LWP 51664)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 38 (Thread 0x7f263d1f6700 (LWP 51665)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
---Type <return> to continue, or q <return> to quit---
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 37 (Thread 0x7f263c7f5700 (LWP 51666)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 36 (Thread 0x7f263bdf4700 (LWP 51667)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 35 (Thread 0x7f263b3f3700 (LWP 51668)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 34 (Thread 0x7f263a9f2700 (LWP 51669)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 33 (Thread 0x7f2639ff1700 (LWP 51670)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
---Type <return> to continue, or q <return> to quit---
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 32 (Thread 0x7f26395f0700 (LWP 51671)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 31 (Thread 0x7f2638bef700 (LWP 51672)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 30 (Thread 0x7f26381ee700 (LWP 51673)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 29 (Thread 0x7f26377ed700 (LWP 51674)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 28 (Thread 0x7f2636dec700 (LWP 51675)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
---Type <return> to continue, or q <return> to quit---
 
 
 
Thread 27 (Thread 0x7f26363eb700 (LWP 51676)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 26 (Thread 0x7f26359ea700 (LWP 51677)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 25 (Thread 0x7f2634fe9700 (LWP 51678)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 24 (Thread 0x7f26345e8700 (LWP 51679)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 23 (Thread 0x7f2633be7700 (LWP 51680)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 22 (Thread 0x7f26331e6700 (LWP 51681)):
 
---Type <return> to continue, or q <return> to quit---
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 21 (Thread 0x7f26327e5700 (LWP 51682)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 20 (Thread 0x7f2631de4700 (LWP 51683)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 19 (Thread 0x7f26313e3700 (LWP 51684)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 18 (Thread 0x7f26309e2700 (LWP 51685)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 17 (Thread 0x7f262ffe1700 (LWP 51686)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
---Type <return> to continue, or q <return> to quit---
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 16 (Thread 0x7f262f5e0700 (LWP 51687)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 15 (Thread 0x7f262ebdf700 (LWP 51688)):
 
#0  0x0000000000950d14 in __io_getevents_0_4 ()
 
#1  0x0000000000927564 in os_aio_linux_handle ()
 
#2  0x00000000008da2d5 in fil_aio_wait ()
 
#3  0x00000000008480b8 in io_handler_thread ()
 
#4  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#5  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 14 (Thread 0x7f260a5fe700 (LWP 51690)):
 
#0  0x0000003f8d40b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
 
#1  0x000000000092959b in os_event_wait_time_low ()
 
#2  0x0000000000844a70 in srv_lock_timeout_thread ()
 
#3  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#4  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 13 (Thread 0x7f260afff700 (LWP 51691)):
 
#0  0x0000003f8d40b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
 
#1  0x000000000092959b in os_event_wait_time_low ()
 
#2  0x0000000000844fc4 in srv_error_monitor_thread ()
 
#3  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#4  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 12 (Thread 0x7f2609bfd700 (LWP 51692)):
 
#0  0x0000003f8d40b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
 
#1  0x000000000092959b in os_event_wait_time_low ()
 
#2  0x0000000000844783 in srv_monitor_thread ()
 
#3  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#4  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
---Type <return> to continue, or q <return> to quit---
 
Thread 11 (Thread 0x7f26091fc700 (LWP 51693)):
 
#0  0x0000003f8d40b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
 
#1  0x000000000092959b in os_event_wait_time_low ()
 
#2  0x0000000000845373 in srv_LRU_dump_restore_thread ()
 
#3  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#4  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 10 (Thread 0x7f26087fb700 (LWP 51694)):
 
#0  0x0000003f8d40b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
 
#1  0x0000000000929491 in os_event_wait_low ()
 
#2  0x0000000000845eee in srv_master_thread ()
 
#3  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#4  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 9 (Thread 0x7f2607dfa700 (LWP 51695)):
 
#0  0x0000003f8d40b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
 
#1  0x000000000092959b in os_event_wait_time_low ()
 
#2  0x0000000000847e91 in srv_purge_thread ()
 
#3  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#4  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 8 (Thread 0x7f26073f9700 (LWP 51696)):
 
#0  0x0000003f8d40b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
 
#1  0x00000000009a4587 in my_service_thread_sleep ()
 
#2  0x000000000099ce02 in ma_checkpoint_background ()
 
#3  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#4  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 7 (Thread 0x7f2cc8d3d700 (LWP 51697)):
 
#0  0x0000003f8d40f2a5 in sigwait () from /lib64/libpthread.so.0
 
#1  0x000000000051fdff in signal_hand ()
 
#2  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#3  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 6 (Thread 0x7f2cc8c62700 (LWP 51721)):
 
#0  0x0000003f8d40e54d in read () from /lib64/libpthread.so.0
 
#1  0x0000000000aa4a98 in vio_read ()
 
#2  0x00000000005281d1 in my_real_read(st_net*, unsigned long*) ()
 
#3  0x0000000000529099 in my_net_read ()
 
#4  0x00000000005a4df2 in do_command(THD*) ()
 
#5  0x000000000065512b in do_handle_one_connection(THD*) ()
 
#6  0x00000000006551ac in handle_one_connection ()
 
---Type <return> to continue, or q <return> to quit---
 
#7  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#8  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 5 (Thread 0x7f2cc79fe700 (LWP 51738)):
 
#0  0x0000003f8d40b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
 
#1  0x0000000000535532 in terminate_slave_thread(THD*, st_mysql_mutex*, st_mysql_cond*, unsigned int volatile*, bool) ()
 
#2  0x000000000053d0ac in terminate_slave_threads ()
 
#3  0x00000000005b691d in stop_slave(THD*, Master_info*, bool) ()
 
#4  0x000000000059eebe in mysql_execute_command(THD*) ()
 
#5  0x00000000005a2854 in mysql_parse ()
 
#6  0x00000000005a4d08 in dispatch_command(enum_server_command, THD*, char*, unsigned int) ()
 
#7  0x000000000065512b in do_handle_one_connection(THD*) ()
 
#8  0x00000000006551ac in handle_one_connection ()
 
#9  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#10 0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 4 (Thread 0x7f2cc79b5700 (LWP 51739)):
 
#0  0x0000003f8d40e054 in __lll_lock_wait () from /lib64/libpthread.so.0
 
#1  0x0000003f8d4093be in _L_lock_995 () from /lib64/libpthread.so.0
 
#2  0x0000003f8d409326 in pthread_mutex_lock () from /lib64/libpthread.so.0
 
#3  0x000000000051f6b8 in inline_mysql_mutex_lock.constprop.126 ()
 
#4  0x000000000052024c in show_heartbeat_period(THD*, st_mysql_show_var*, char*) ()
 
#5  0x00000000005f38a4 in show_status_array(THD*, char const*, st_mysql_show_var*, enum_var_type, system_status_var*, char const*, TABLE*, bool, Item*) ()
 
#6  0x00000000005fe618 in fill_status(THD*, TABLE_LIST*, Item*) ()
 
#7  0x0000000000602d67 in get_schema_tables_result(JOIN*, enum_schema_table_state) ()
 
#8  0x00000000005e9c8d in JOIN::exec() ()
 
#9  0x00000000005e5525 in mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) ()
 
#10 0x00000000005ec113 in handle_select(THD*, LEX*, select_result*, unsigned long) ()
 
#11 0x00000000005970a6 in execute_sqlcom_select(THD*, TABLE_LIST*) ()
 
#12 0x000000000059e87f in mysql_execute_command(THD*) ()
 
#13 0x00000000005a2854 in mysql_parse ()
 
#14 0x00000000005a4d08 in dispatch_command(enum_server_command, THD*, char*, unsigned int) ()
 
#15 0x000000000065512b in do_handle_one_connection(THD*) ()
 
#16 0x00000000006551ac in handle_one_connection ()
 
#17 0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#18 0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 3 (Thread 0x7f2cc8cf4700 (LWP 51804)):
 
#0  0x0000003f8d40e54d in read () from /lib64/libpthread.so.0
 
#1  0x0000000000aa4a98 in vio_read ()
 
#2  0x0000000000aa4b32 in vio_read_buff ()
 
---Type <return> to continue, or q <return> to quit---
 
#3  0x00000000005281d1 in my_real_read(st_net*, unsigned long*) ()
 
#4  0x0000000000529035 in my_net_read ()
 
#5  0x00000000006a9365 in cli_safe_read ()
 
#6  0x000000000053db43 in handle_slave_io ()
 
#7  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#8  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 2 (Thread 0x7f2cc8cab700 (LWP 51805)):
 
#0  0x0000003f8d40e054 in __lll_lock_wait () from /lib64/libpthread.so.0
 
#1  0x0000003f8d4093be in _L_lock_995 () from /lib64/libpthread.so.0
 
#2  0x0000003f8d409326 in pthread_mutex_lock () from /lib64/libpthread.so.0
 
#3  0x0000000000534c50 in inline_mysql_mutex_lock ()
 
#4  0x000000000053ee10 in THD::add_status_to_global() ()
 
#5  0x000000000053c34b in handle_slave_sql ()
 
#6  0x0000003f8d407851 in start_thread () from /lib64/libpthread.so.0
 
#7  0x0000003f8d0e894d in clone () from /lib64/libc.so.6
 
 
 
Thread 1 (Thread 0x7f2cc8da2800 (LWP 51653)):
 
#0  0x0000003f8d0df293 in poll () from /lib64/libc.so.6
 
#1  0x0000000000523653 in handle_connections_sockets() ()
 
#2  0x00000000005273a2 in mysqld_main(int, char**) ()
 
#3  0x0000003f8d01ecdd in __libc_start_main () from /lib64/libc.so.6
 
#4  0x000000000051d7e9 in _start ()

)

Generated at Thu Feb 08 07:17:13 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.