[MCOL-1468] CDC adapter: mxs_adapter doesn't measure transaction insert time as expected Created: 2018-06-13  Updated: 2023-10-26  Resolved: 2018-08-07

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: 1.1.5
Fix Version/s: 1.1.6

Type: Bug Priority: Major
Reporter: Elena Kotsinova (Inactive) Assignee: markus makela
Resolution: Fixed Votes: 0
Labels: None


 Description   

CDC adapter calculates time between two transactions and displays this time as time for inserting of data from one transaction. It is expected to measure the time for insert operation only.

Prerequisites:
Setup the environment as follows
MariaDB Server ---> MaxScale ---> ColumnStore
In testing environment the MariaDB Server + MaxScale + CDC Adapter and all needed components are installed on the same machine.

Steps to reproduce:
1. Create table in MariaDB Server

 CREATE TABLE cdc_eli.bst (
 i int(11),
 x char(20)
)

2. Insert 1 row in it

insert into bst values (1,'a1');

3. Run CDC streaming adapter

mxs_adapter -c /etc/Columnstore.xml -u cdcuser -p cdcpassword -h localhost -P 4001 cdc_eli bst

4. The cdc adapter displays message
Table not found, create with:

   CREATE TABLE cdc_eli.bst (domain int, event_number int, event_type varchar(50), i int(11), sequence int, server_id int, timestamp int, x char(20)) ENGINE=ColumnStore;

5. Create the required table in ColumnStore
6. Run cdc adapter again
7. The adapter will process the data and insert them in the ColumnStore table.

1 rows and 1 transactions inserted in 0.814648 seconds. GTID = 0-1-48

8. Wait for a couple of minutes and insert one more record through the MariaDB Server

insert into bst values (2,'a');

The measured time is displayed as:

1 rows and 1 transactions inserted in 731.168 seconds. GTID = 0-1-49

9. Wait more - for example around 1 hour and insert one more row

insert into bst values (3,'a3');

The measured time is displayed as:

1 rows and 1 transactions inserted in 3044.4 seconds. GTID = 0-1-50

Expected:
Display time for inserting if the records. Waiting time must not be taken in considerations.



 Comments   
Comment by markus makela [ 2018-06-14 ]

The output is the value returned by mcsapi::ColumnStoreSummary::getExecutionTime(). LinuxJedi does this value count the total bulk insert operation time? If so, either the wording of the output should be changed or another mechanism to track execution times needs to be added.

Comment by Andrew Hutchings (Inactive) [ 2018-06-14 ]

markus makela the API's timer starts when the bulk insert object is created and stops at the end of commit or rollback.

Comment by markus makela [ 2018-08-07 ]

Updated the wording in 42909280c7b29c4de416b81c26b93bd82c7749e2 to correctly describe that the time is not for transaction insertion but the span over which the data was loaded.

Generated at Thu Feb 08 02:28:59 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.