[MDEV-5289] master server starts slave parallel threads Created: 2013-11-14  Updated: 2015-03-11  Resolved: 2015-03-11

Status: Closed
Project: MariaDB Server
Component/s: Replication
Affects Version/s: 10.0.5
Fix Version/s: 10.0.18

Type: Bug Priority: Major
Reporter: Giuseppe Maxia Assignee: Kristian Nielsen
Resolution: Fixed Votes: 0
Labels: master, parallelslave, replication
Environment:

install master/slave replication



 Description   

When you run this command in a slave
set global slave_parallel_threads=10;
you get a rightful error

ERROR 1198 (HY000): This operation cannot be performed as you have a running slave ''; run STOP SLAVE '' first

However, if you run the same command in a master, the statement is accepted, and the thread started.

master [localhost] {msandbox} ((none)) > select ID, DB, state, time_ms, memory_used from information_schema .PROCESSLIST where USER='system user';
Empty set (0.01 sec)
 
master [localhost] {msandbox} ((none)) > set global slave_parallel_threads=10;
Query OK, 0 rows affected (0.00 sec)
 
master [localhost] {msandbox} ((none)) > select ID, DB, state, time_ms, memory_used from information_schema .PROCESSLIST where USER='system user';
+----+------+----------------------------------+----------+-------------+
| ID | DB   | state                            | time_ms  | memory_used |
+----+------+----------------------------------+----------+-------------+
| 47 | NULL | Waiting for work from SQL thread | 2207.683 |       34704 |
| 46 | NULL | Waiting for work from SQL thread | 2207.686 |       34704 |
| 45 | NULL | Waiting for work from SQL thread | 2207.722 |       34704 |
| 44 | NULL | Waiting for work from SQL thread | 2207.723 |       34704 |
| 43 | NULL | Waiting for work from SQL thread | 2207.754 |       34704 |
| 42 | NULL | Waiting for work from SQL thread | 2207.757 |       34704 |
| 41 | NULL | Waiting for work from SQL thread | 2207.765 |       34704 |
| 40 | NULL | Waiting for work from SQL thread | 2207.801 |       34704 |
| 39 | NULL | Waiting for work from SQL thread | 2207.843 |       34704 |
| 38 | NULL | Waiting for work from SQL thread | 2207.851 |       34704 |
+----+------+----------------------------------+----------+-------------+
10 rows in set (0.01 sec)



 Comments   
Comment by Elena Stepanova [ 2013-11-14 ]

Is it wrong? There is no slave running on the master (I assume there is none, right?), so why would it throw ER_SLAVE_MUST_STOP?
And we can't forbid setting the option before configuring server as a slave, because then it will be impossible to put the value in the config file.

Maybe I misunderstand the nature of the problem, could you please elaborate on it?

Comment by Giuseppe Maxia [ 2013-11-14 ]

It is wrong, because slave threads are started in a server that is NOT configured as a slave. I have no idea what these threads do in the master, but surely I would not want additional threads running in a production server.

The right behavior should be do nothing. Then if I run 'start slave', I should get an error that the server is not configured as a slave, and all is well.
This is what MySQL 5.6 does for its corresponding variable, and it is the right thing, because this kind of protection allows you to configure all your servers in the same way, but only the active slaves get the additional threads.

Comment by Elena Stepanova [ 2013-11-14 ]

I see, yes, it makes sense that the value should be configurable, but the actual thread start should only happen when the server starts serving as a slave.

Comment by Kristian Nielsen [ 2013-11-14 ]

I do not think this is worth it, it just causes needless complexity.

Those are just background threads. They are waiting for work, if no work
arrives (because no SQL threads are running), then they will just sit idle.

It is just like the InnoDB background threads. If you enable InnoDB you get
all the InnoDB threads started, even if you do not ever create any InnoDB
tables; they just will not be doing anything.

Comment by Kristian Nielsen [ 2013-11-14 ]

BTW, the MySQL 5.6 variable is quite different if I understand correctly.

In MariaDB, the threads just form a thread pool, shared among any configured slaves. They do not do anything except sit ready, waiting for work from slave SQL threads.

In the future, one could imagine a more advanced thread pool management, eg. that
could dynamically spawn and shutdown threads depending on the current needs and
load on the replication. But in this first version, a simple fixed-size pool is what we
have.

Comment by Giuseppe Maxia [ 2013-11-14 ]

I agree that MySQL 5.6 behavior is quite different.
What I wanted to stress is that the server should not start any task related to slave role unless it is configured as such.
This, is also how Tungsten Replicator works. You can configure a service to run parallel threads, but the threads are only enabled if the service role changes to 'slave'. When a slave is promoted to master, it shelves its additional threads and resumes working in a single thread.
I would expect the same thing from any server. Tasks that are only valid for a slave should not be enabled by default when the service is running as master.

Comment by Kristian Nielsen [ 2013-11-14 ]

Right, so the pool of threads could magically spring into life upon the first CHANGE MASTER statement, and be teared down when the last slave is removed by RESET SLAVE ALL.

But I do not have bandwidth to work on that kind of stuff...

Comment by Kristian Nielsen [ 2013-11-17 ]

I changed my mind a bit, would be nice to not have the threads started unnecessarily, even if they are not harmful as such. So re-opening.

Comment by Kristian Nielsen [ 2014-06-10 ]

Idea for how to implement this in a way that works with the current code without introducing extra locking overhead:

1. Do not start the worker threads in the pool at server start.

2. In start_slave_threads(), first startup the worker threads, if we did not already.

3. At the end of handle_slave_sql(), if we are shutting down the last SQL driver thread, shut down the pool of worker threads also.

And do the similar thing for the slave background thread also. This has to be started briefly during server startup (to load mysql.gtid_slave_pos), but we can at that point stop it again, and then start and stop it along with the worker threads in the pool.

Comment by Kristian Nielsen [ 2015-03-11 ]

Pushed to 10.0.18:

http://lists.askmonty.org/pipermail/commits/2015-March/007562.html

With this patch, parallel replication worker threads are not spawned until
needed (when an SQL thread is started), and they will be de-spawned if all SQL
threads are stopped.

Thanks, Giuseppe, for reporting this!

Generated at Thu Feb 08 07:03:07 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.