[MDEV-31763] parallel slave replication not enabled when thread pool is on Created: 2023-07-22  Updated: 2023-08-27  Resolved: 2023-08-27

Status: Closed
Project: MariaDB Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Fan Lyu Assignee: Kristian Nielsen
Resolution: Incomplete Votes: 0
Labels: None


 Description   

I have a master with 128 threads and a slave with 32 parallel slave workers.
Firstly I used dafault variable of threadpool, thread_handling = one-thread-per-connection
TPS of master about 3800, slave about 2500 to 3000.
when I show processlist, 32 slave workers are running, though waiting for transaction.

then I want to improve the slave replication speed, then set thread_handling= pool-of-threads.(to reduce the cost of creating and destroying threads)
however when I show processlist, only one slave worker is running, other 31 slave workers are in status "waiting for work from sql thread".
The TPS of slave fall from 2500 to 70.

I was wondering if there is some variable fault that I ignored?
my parameters in slave:
innodb_flush_log_at_trx_commit = 2
sync_binlog = 0
thread_handling = pool-of-threads

I also have thread_handling = pool-of-threads in my master but the concurrency of master is even better (higher TPS , 4400 after enabling threadpool ,compared with 3800 by one-thread-per-connection)

 Comments   
Comment by Kristian Nielsen [ 2023-07-24 ]

--thread-handling should not affect the parallel replication (the connection thread pool and the parallel replication worker thread pool are separate and not sharing configuration).

If slave workers are in state "waiting for work from sql thread", it means the slave is caught up, there is no further work to be done, that's why only one worker thread is running.

You could try to stop the slave temporarily for a bit to accumulate a good amount of transactions pending from the master, and then check the TPS of the slave when it catches up to see that it's a reasonable high number.

So try to double-check that the --thread-handling=pool-of-threads doesn't affect TPS of slave.

Hope this helps,

- Kristian.

Generated at Thu Feb 08 10:26:17 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.