Details
-
Task
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
Description
In some case, the speed of reading binlog from master is high, especially when doing a new replica.
It would bring the high traffic in master.
So We introduce a new variable "read_binlog_speed_limit" to control the binlog reading rate for IO thread to solve the problem.
handle_io_slave:
|
last_add_time = now()
|
tokens = read_binlog_speed_limit //there are some initial tokens in the bucket |
while(true){ |
event = read_event()
|
if (read_binlog_speed_limit > 0) { |
if (tokents > TOKEN_MAX) { |
tokens = TOKEN_MAX
|
last_add_time = now()
|
}
|
do{ |
//put some token |
tokens = tokens + (now()-last_add_time)*read_binlog_speed_limit;
|
//if the token is not enough, sleep some time. |
if(tokens < event.real_network_read_len) |
sleep((event.len-tokens)/speed_limit)
|
}
|
}
|
write_event(event)
|
}
|
It can work when slave_compressed_protocol is on.
But it maybe doesn't work well when the binlog event is very big.
Attachments
Issue Links
- relates to
-
MDEV-11120 sys_vars.sysvars_server_notembedded fails on 32-bit
-
- Closed
-
- links to
Activity
| Field | Original Value | New Value |
|---|---|---|
| Remote Link | This issue links to "GitHub Pull Request #246 (Web Link)" [ 27628 ] |
| Fix Version/s | 10.2.3 [ 22115 ] | |
| Fix Version/s | 10.2 [ 14601 ] | |
| Resolution | Fixed [ 1 ] | |
| Status | Open [ 1 ] | Closed [ 6 ] |
| Link |
This issue relates to |
| Description |
In some case, the speed of reading binlog from master is high, especially when doing a new replica.
It would bring the high traffic in master. So We introduce a new variable "read_binlog_speed_limit" to control the binlog reading rate for IO thread to solve the problem. handle_io_slave: last_add_time = now() tokens = read_binlog_speed_limit //there are some initial tokens in the bucket while(true){ event = read_event() if (read_binlog_speed_limit > 0) { if (tokents > TOKEN_MAX) { tokens = TOKEN_MAX last_add_time = now() } do{ //put some token tokens = tokens + (now()-last_add_time)*read_binlog_speed_limit; //if the token is not enough, sleep some time. if(tokens < event.real_network_read_len) sleep((event.len-tokens)/speed_limit) } } write_event(event) } It can work when slave_compressed_protocol is on. But it maybe doesn't work well when the binlog event is very big. |
In some case, the speed of reading binlog from master is high, especially when doing a new replica.
It would bring the high traffic in master. So We introduce a new variable "read_binlog_speed_limit" to control the binlog reading rate for IO thread to solve the problem. {code:c++} handle_io_slave: last_add_time = now() tokens = read_binlog_speed_limit //there are some initial tokens in the bucket while(true){ event = read_event() if (read_binlog_speed_limit > 0) { if (tokents > TOKEN_MAX) { tokens = TOKEN_MAX last_add_time = now() } do{ //put some token tokens = tokens + (now()-last_add_time)*read_binlog_speed_limit; //if the token is not enough, sleep some time. if(tokens < event.real_network_read_len) sleep((event.len-tokens)/speed_limit) } } write_event(event) } {code} It can work when slave_compressed_protocol is on. But it maybe doesn't work well when the binlog event is very big. |
| Description |
In some case, the speed of reading binlog from master is high, especially when doing a new replica.
It would bring the high traffic in master. So We introduce a new variable "read_binlog_speed_limit" to control the binlog reading rate for IO thread to solve the problem. {code:c++} handle_io_slave: last_add_time = now() tokens = read_binlog_speed_limit //there are some initial tokens in the bucket while(true){ event = read_event() if (read_binlog_speed_limit > 0) { if (tokents > TOKEN_MAX) { tokens = TOKEN_MAX last_add_time = now() } do{ //put some token tokens = tokens + (now()-last_add_time)*read_binlog_speed_limit; //if the token is not enough, sleep some time. if(tokens < event.real_network_read_len) sleep((event.len-tokens)/speed_limit) } } write_event(event) } {code} It can work when slave_compressed_protocol is on. But it maybe doesn't work well when the binlog event is very big. |
In some case, the speed of reading binlog from master is high, especially when doing a new replica.
It would bring the high traffic in master. So We introduce a new variable "read_binlog_speed_limit" to control the binlog reading rate for IO thread to solve the problem. {code:c++} handle_io_slave: last_add_time = now() tokens = read_binlog_speed_limit //there are some initial tokens in the bucket while(true){ event = read_event() if (read_binlog_speed_limit > 0) { if (tokents > TOKEN_MAX) { tokens = TOKEN_MAX last_add_time = now() } do{ //put some token tokens = tokens + (now()-last_add_time)*read_binlog_speed_limit; //if the token is not enough, sleep some time. if(tokens < event.real_network_read_len) sleep((event.len-tokens)/speed_limit) } } write_event(event) } {code} It can work when slave_compressed_protocol is on. But it maybe doesn't work well when the binlog event is very big. |
| Workflow | MariaDB v3 [ 77844 ] | MariaDB v4 [ 132970 ] |
Pushed to 10.2.3, thanks!