Details
-
Task
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
Description
We add new event types to support compress the binlog as follow:
QUERY_COMPRESSED_EVENT,
WRITE_ROWS_COMPRESSED_EVENT_V1,
UPDATE_ROWS_COMPRESSED_EVENT_V1,
DELETE_POWS_COMPRESSED_EVENT_V1,
WRITE_ROWS_COMPRESSED_EVENT,
UPDATE_ROWS_COMPRESSED_EVENT,
DELETE_POWS_COMPRESSED_EVENT
Of course, in mariadb 10.2, it's always the V1 version of compressed binlog row event.
We introduce two option for this feature: "log_bin_compress " and "log_bin_compress_min_len", the
former is a switch of whether the binlog should be compressed and the latter is the minimum length of
sql statement(in statement mode) or record(in row mode) that can be compressed.
In master, it can be described by the code:
if binlog_format == statement
if binlog_format == row
{ if log_bin_compress == true and record_len >= log_bin_compress_min_len create a Write_rows_compressed_log_event(when INSERT) else create a Write_log_event(when INSERT); }And in slave, the compressed binlog events would be converted to the uncompressed form in IO thread, such as QUERY_COMPRESSED_EVENT convert to QUERY_EVENT.
So the SQL and worker threads can be stay unchanged. And the sql thread will be executed as fast as executing uncompressed binlog.
Now, we use zlib as compressed algrithm.
And the compressed record is
Compressed Record(3 parts)
Record Header: 1 Byte
0 Bit: Always 1, mean compressed;
1-3 Bit: Reversed, compressed algorithm。0 means zlib, It's always 0 by now.
4-7 Bit: Bytes of "Record Original Length"
Record Original Length: 1-4 Bytes
Compressed Buf: the real compressed part.
The feture can reduce 42% ~ 70% binlog capacity in our production environment.
Attachments
Issue Links
- links to