Details
-
Task
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
Description
The mysqlbinlog client program needs to be updated to support GTID.
Here is a suggested list of things to be done:
- The --start-position and --stop-position options should be able to take
GTID positions; or maybe there should be new --start-gtid and --stop-gtid
options. Like --start-gtid=0-1-100,1-2-200,2-1-1000.
- A GTID position means the point just after that GTID. So starting from
GTID 0-1-100 and stopping at GTID 0-1-200, the first GTID output will
probably be 0-1-101 and the last one 0-1-200. Note that if some domain is
not specified in the position, it means to start from the begining,
respectively stop immediately in that domain.
- Starting and stopping GTID should work both with local files, and with
--read-from-remote-server. For the latter, there are a couple of extra
things that need doing in the master-slave protocol, see
get_master_version_and_clock() in sql/slave.cc.
- At the end of the dump, put these statements, to reduce the risk of those session variables incorrectly spilling into subsequent statements run in the same session:
SET session.server_id = @@global.server_id,
session.gtid_domain_id=@@global.gtid_domain_id;
Probably some more things will come up during the work, but this looks like a
reasonable start.
Attachments
Issue Links
- causes
-
MDEV-29803 Unexpected ERROR: Found out of order GTID after replaying binlog read from remote
-
- Open
-
- is part of
-
-
- Closed
-
- relates to
-
-
- Closed
-
-
-
- Closed
-
-
-
- Open
-
-
MXS-2581 Support DO_DOMAIN_IDS and IGNORE_DOMAIN_IDS for the Binlog Router
-
- Closed
-
Activity
Some suggestions for getting started with this:
I would recommend to start by reading the GTID documentation, and
experimenting with all the relevant features described there, to familiarise
yourself with it. The main thing is to understand the concept of GTID position
and binlog order, and the relation to replication domains:
https://mariadb.com/kb/en/mariadb/global-transaction-id/
There are also some blog posts:
http://kristiannielsen.livejournal.com/16826.html
http://kristiannielsen.livejournal.com/17008.html
http://kristiannielsen.livejournal.com/17238.html
And similarly, familiarise yourself with mysqlbinlog and how it would make
sense for it to be extended to support GTID.
Then make a detailed proposal for how the user-visible interface of
mysqlbinlog should be extended, and send it to the list for comments.
The next step will be to understand the concept of binlog state
(@@gtid_binlog_state), and how it is used to deal correctly with out-of-order
GTID sequence numbers in binlogs. This is mostly an internal implementation
detail, but necessary to be able to correctly identify starting and stopping
GTID positions in all cases.
Hi Kristian,
You have logged this issue/idea to four weeks only. Do you recommend GSoC applicants to propose to take on other ideas as well, or elaborate on this one, considering that they might finish this implementation within the estimated time.
Thanks in advance,
Frederik
Hi Kristian,
I'm a senior year undergraduate student majoring in Computer Science. I
intend to be a part of the MariaDB Developer Community and would love to
start my journey through the Google Summer of Code this summer.
Do you expect any task that I should complete before submitting proposal for this particular project so that I can get a positive response.
Why this project was not allocated last year, even though it is being considered for Gsoc from last 3 years.
Hi,
I am Mike Reiss a MS Computer Science student and I would love to work on this project during GSoC 2017.I have studied the mysqlbinlog.cc file and experimented with the various options available in the my_options array of the tool. The proposed idea is straight forward to implement: We add the --start-gtid and --stop-gtid options to the array of options then parse the GTID passed to these options to get the GTID sequence number for the binlog. One quick question. Is there any relationship between a GTID sequence number and the argument passed to the --start-position option? that way we need to set the start position to the GTID sequence number. I am still doing more research into the source files and I will love to discuss this more with the mentor. I am having difficulties debugging mysqlbinlog on OS X sierra because it does not support GDB any ideas please?
mikereiss - a number of gdb options for OS X Sierra are mentioned here http://stackoverflow.com/questions/39702871/gdb-kind-of-doesnt-work-on-macos-sierra . Its essential that you can work an debugger in your development environment and utilising a mentors time to achieve this isn't a productive use of their time especially if they use a different OS. There are plenty of other people/resources that should be able to help you with this.
Also related: https://lists.launchpad.net/maria-developers/msg09332.html and
https://lists.launchpad.net/maria-developers/msg08186.html
@Daniel Black - I'm sorry I should have crafted my question as what is the recommended debugger used by MariaDB developers on OS X ? Thank you for the tip on gdb. Vicențiu on Maria-Developers recommended that I use lldb and it works great for me. Hope its fine with you I continue using lldb.
{--start.stop}-position may be reused to accept the new type of GTID format string.
This is ready for review.
Commit: f7e582ff9c5ace871c9b98ea8465956c1e94fb96
Buildbot: bb-10.7-MDEV-4989
-
- Updated commit Aug 27 with some documentation, code formatting, and slight code changes to help with
MDEV-20119
- Updated commit Aug 27 with some documentation, code formatting, and slight code changes to help with
I have pushed a new commit which addresses an initial round of reviews from Andrei:
Commit: 02f98a22d2caef7c2953fe02b7f7516cd08c9544
Buildbot: bb-10.7-MDEV-4989
This is actually a really good starter project for someone to get into working with the GTID code.
The tasks requires getting some moderate understanding on how GTID works in general, but the actual code that needs to be worked on is fairly isolated and doesn't need to touch a majority of the GTID and replication stuff.