[MDEV-23862] High contention on last_value in my_timer_microseconds Created: 2020-10-01  Updated: 2020-10-07

Status: Open
Project: MariaDB Server
Component/s: None
Fix Version/s: None

Type: Task Priority: Minor
Reporter: Georgy Kirichenko Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Description:
There are two functions: my_timer_microseconds and my_timer_milliceconds in mysys/my_rdtsc.cc. Both functions save last seen value in a last_value static variable which is shared across all running threads. This trick was implemented because of an ancient bug report about gettimeofday intermittent errors.

After Linux kernel source investigation I found that the only way in which gettimeofday can fail with tz=NULL passed is to use an invalid pointer as the first argument but this is definitely not our case.

Even if we assume that intermittent errors of gettimeofday are possible then this failures are handled quite inconsistently throughout the MySQL source. In some cases we run an endless loop until it succeeds, in some cases we just ignore failures, and in some cases we return "last seen" value. The latter is the most expensive method, because it involves updating a global variable.

The price to pay is visible on modern Linux where gettimeofday with tz=NULL may never fail, below are sysbench results on X86 and AARCH64 for vanilla and patched version of my_timer_microseconds where assumed that gettimeofday is reliable:

AARCH64:

test threads vanilla patched diff
:-------: --------: ------------: -----------: ------:
OLTP_PS 64 791578.16 807313.49 1.99%
OLTP_PS 128 986087.11 1007570.61 2.18%
OLTP_PS 256 986725.61 1009114.64 2.27%
OLTP_RO 64 30258.31 30336.54 0.26%
OLTP_RO 128 34946.59 35087.35 0.40%
OLTP_RO 256 34994.82 35150.73 0.45%
OLTP_RW 64 20159.63 20201.69 0.21%
OLTP_RW 128 18579.80 18681.46 0.55%
OLTP_RW 256 19140.54 19244.04 0.54%

X86:

test threads vanilla patched diff
:-------: --------: ------------: -----------: ------:
OLTP_PS 64 551636.06 553277.90 0.30%
OLTP_PS 128 574358.40 579725.62 0.93%
OLTP_PS 256 559891.53 565747.08 1.05%
OLTP_RO 64 18662.21 18783.30 0.65%
OLTP_RO 128 18827.19 18948.55 0.64%
OLTP_RO 256 18602.29 18741.15 0.75%
OLTP_RW 64 9343.21 9380.53 0.40%
OLTP_RW 128 9880.84 9927.12 0.47%
OLTP_RW 256 9634.62 9666.60 0.33%

The code in question is used only in Performance Schema. Even if gettimeofday() may fail, using the expensive "last seen value" method is not any different from any other ways to handle errors: it will only result in skewed statistics, which is not that important, given how unlikely this event is.



 Comments   
Comment by Sergei Golubchik [ 2020-10-03 ]

I don't see that my_timer_milliseconds() uses last_value in MariaDB.

Returning 0 when gettimeofday() fails could significantly corrupt the statistis, I'm afraid. If the caller doesn't reject zero values, the sudden huge duration will make averages pretty much meaningless for quite a while.

Because we don't know why gettimeofday() might fail, I'd say that looping is not an option. But we canfallback to some other existing counter, scaled accordingly.

Comment by Georgy Kirichenko [ 2020-10-07 ]

You are right - my_timer_milliseconds does not use gettimeofday, excuse me.

Despite the statistic corruption could be significant this function does not seem to able to fail (according to Linux code investigation).
However, if we would think that the function is still unreliable then we could use clock_gettime instead in the same way as in my_timer_milliseconds.

Generated at Thu Feb 08 09:25:37 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.