Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-1469

CDC adapter: mxs_adapter doesn't insert only one row if there are more rows inserted at the same time

    XMLWordPrintable

    Details

      Description

      When insert script contains more than one rows, the cdc adapter inserts only one row in one transaction and waits for the next insert event to insert the rest of the rows in the next transaction.

      Prerequisites:
      Setup the environment as follows
      MariaDB Server ---> MaxScale ---> ColumnStore
      In testing environment the MariaDB Server + MaxScale + CDC Adapter and all needed components are installed on the same machine.

      Steps to reproduce:
      1. Create table in MariaDB Server

       CREATE TABLE cdc_eli.bst (
       i int(11),
       x char(20)
      )
      

      2. Insert 1 row in it

      insert into bst values (1,'a1');
      

      3. Run CDC streaming adapter

      mxs_adapter -c /etc/Columnstore.xml -u cdcuser -p cdcpassword -h localhost -P 4001 cdc_eli bst
      

      4. The cdc adapter displays message
      Table not found, create with:

         CREATE TABLE cdc_eli.bst (domain int, event_number int, event_type varchar(50), i int(11), sequence int, server_id int, timestamp int, x char(20)) ENGINE=ColumnStore;
      
      

      5. Create the required table in ColumnStore
      6. Run cdc adapter again
      7. The adapter will process the data and insert them in the ColumnStore table.

      1 rows and 1 transactions inserted in 0.814648 seconds. GTID = 0-1-48
      

      8. Insert more records through the MariaDB Server

      insert into bst values (2,'a'),(3,'a3');
      

      Message from adapter is:

      1 rows and 1 transactions inserted in 884.539 seconds. GTID = 0-1-53
      

      In ColumnStore table only one row is inserted. The second row is missing.

      9.Insert one row

      insert into bst values (4,'a4');
      

      Message from adapter is:

      2 rows and 1 transactions inserted in 44.3701 seconds. GTID = 0-1-54
      

      In ColumnStore table the second record from the previous transaction is inserted together with the first record in the next transaction.
      Note: No meter how many rows are inserted at the same time in MariaDB Server table the cdc adapter inserts only one row in the first transaction.
      Very similar behavior is observer if -r option is used.
      For example if -r is set to be 2 then the inserted records in the first transaction must be exact 2. If they are more than 2 the rest will be inserted in the next bulk transactions.

      Expected: The number of rows inserted together in the MariaDB Server to be inserted in ColumnStore in the same transaction.
      The next event can come after long time and until then the databases will not have the same data.

        Attachments

          Activity

            People

            Assignee:
            markus makela markus makela
            Reporter:
            elena.kotsinova Elena Kotsinova
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Git Integration