[MDEV-3789] LP:470590 - Multiple fsync()s when sync_binlog=N (>0) Created: 2009-11-02  Updated: 2012-10-04  Resolved: 2012-10-04

Status: Closed
Project: MariaDB Server
Component/s: None
Fix Version/s: None

Type: Task Priority: Minor
Reporter: Arjen Lentz (Inactive) Assignee: Kristian Nielsen
Resolution: Not a Bug Votes: 0
Labels: Launchpad

Attachments: XML File LPexportBug470590.xml    

 Description   

Depending on the setup and the choice of speed vs integrity in case of a server crash, configs need sync_binlog=1.
This slows down the server, but actually unnecessarily so. The setting apparently causes separate fsync() calls in various places in the code (binlog, relay log/index, possibly the master.info file), and that's ridiculous.

Can someone (Monty?) please fix this, and make the logic such that we get only a single fsync() per commit?
If this is a logistical problem given the XtraDB/InnoDB plugin, I can cope with one extra fsync() in the main server code in addition to the one in InnoDB... that can be a good intermediate step and would already reduce the overhead and slow-down significantly.

Thanks!

 Comments   
Comment by Sergei Petrunia [ 2009-11-24 ]

Re: Multiple fsync()s when sync_binlog=N (>0)
Arjen,

Could you please point at the offending fsync() calls or provide some evidence that there are multiple syncs being done for one transaction? (i.e. what has led you to suspect that there are more fsync calls done when necessary?)

Comment by Arjen Lentz (Inactive) [ 2009-12-03 ]

Re: [Bug 470590] Re: Multiple fsync()s when sync_binlog=N (>0)
Hi Sergey

On 24/11/2009, at 7:24 PM, Sergey Petrunia wrote:
> Could you please point at the offending fsync() calls or provide some
> evidence that there are multiple syncs being done for one transaction?
> (i.e. what has led you to suspect that there are more fsync calls done
> when necessary?)

Hearsay from people with a clue. Hadn't browsed the code for this
myself.
However, just doing a quick grep/less finds them within a few seconds.
There's mysys/my_sync.c that contains my_sync()

my_sync() gets called from sql/log.cc in several places, inside
functions that do something... for instance when binlog is written, if
sync_binlog is 1 or reaches its configured number, it's fsynced. This,
by definition, implies that it's a separate sync from an InnoDB
transaction or MariaDB action. That's two syncs already, that should
in fact be one.

And the binlog index should really also be part of that, to be
resilient to crashes. It would also be less overhead than it is now,
provided fsync/my_sync is only called once for a commit.

Similarly, the relay log and relay index should be synced at the same
time, and possibly master.info with it.

I hope this provides you with sufficient detail to address the problem.
Thanks

Cheers,
Arjen.

Comment by Hakan Küçükyılmaz (Inactive) [ 2009-12-08 ]

Re: Multiple fsync()s when sync_binlog=N (>0)
We have to investigate the use of fsync(s) and whether it is feasable to reduce the number of fsync() calls.

Comment by Kristian Nielsen [ 2011-09-05 ]

Re: Multiple fsync()s when sync_binlog=N (>0)
The issue is more complex than just saying that things are "ridiculous".

There are several worklogs that describe ideas for improving this. For example:

http://askmonty.org/worklog/Server-RawIdeaBin/?tid=164
http://askmonty.org/worklog/Server-RawIdeaBin/?tid=187
http://askmonty.org/worklog/Server-Sprint/?tid=175

Comment by Rasmus Johansson (Inactive) [ 2011-09-05 ]

Launchpad bug id: 470590

Generated at Thu Feb 08 06:51:13 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.