Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
When insert script contains more than one rows, the cdc adapter inserts only one row in one transaction and waits for the next insert event to insert the rest of the rows in the next transaction.
Prerequisites:
Setup the environment as follows
MariaDB Server ---> MaxScale ---> ColumnStore
In testing environment the MariaDB Server + MaxScale + CDC Adapter and all needed components are installed on the same machine.
Steps to reproduce:
1. Create table in MariaDB Server
CREATE TABLE cdc_eli.bst (
|
i int(11),
|
x char(20)
|
)
|
2. Insert 1 row in it
insert into bst values (1,'a1');
|
3. Run CDC streaming adapter
mxs_adapter -c /etc/Columnstore.xml -u cdcuser -p cdcpassword -h localhost -P 4001 cdc_eli bst
|
4. The cdc adapter displays message
Table not found, create with:
CREATE TABLE cdc_eli.bst (domain int, event_number int, event_type varchar(50), i int(11), sequence int, server_id int, timestamp int, x char(20)) ENGINE=ColumnStore;
|
|
5. Create the required table in ColumnStore
6. Run cdc adapter again
7. The adapter will process the data and insert them in the ColumnStore table.
1 rows and 1 transactions inserted in 0.814648 seconds. GTID = 0-1-48
|
8. Insert more records through the MariaDB Server
insert into bst values (2,'a'),(3,'a3');
|
Message from adapter is:
1 rows and 1 transactions inserted in 884.539 seconds. GTID = 0-1-53
|
In ColumnStore table only one row is inserted. The second row is missing.
9.Insert one row
insert into bst values (4,'a4');
|
Message from adapter is:
2 rows and 1 transactions inserted in 44.3701 seconds. GTID = 0-1-54
|
In ColumnStore table the second record from the previous transaction is inserted together with the first record in the next transaction.
Note: No meter how many rows are inserted at the same time in MariaDB Server table the cdc adapter inserts only one row in the first transaction.
Very similar behavior is observer if -r option is used.
For example if -r is set to be 2 then the inserted records in the first transaction must be exact 2. If they are more than 2 the rest will be inserted in the next bulk transactions.
Expected: The number of rows inserted together in the MariaDB Server to be inserted in ColumnStore in the same transaction.
The next event can come after long time and until then the databases will not have the same data.