Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-2047

multi data streaming with mxs_adapter is not streaming data if table was initially empty

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.2.2
    • Fix Version/s: Icebox
    • Component/s: mcsapi, mxs-cdc-adapter
    • Labels:
      None
    • Environment:
      MariaDB-TX ; maxscale, mcs connector, mxs_adapter ; MariaDB-Columnstore

      Description

      multi data streaming with mxs_adapter is not streaming data if table was initially empty

      1. mxs_adapter is started with -f option with file describing the database and 2 table names to be streamed from MariaDB-TX to MariaDB-Columnstore.
      both tables was create manually on MCS server .
      table testA.t03 was loaded initially with data , testA.t04 was initially empty on MariaDB-TX server.
      the data from MariaDB-TX.testA.t03 is streamed successful to MariaDB-Columnstore.testA.t03

      2. Insert data to table testA.t03 on MariaDB-TX server
      Data is streamed to testA.t03 on MariaDB-Columnstore.

      3. Insert data to table testA.t04 on MariaDB-TX server
      Data is not streamed to MariaDB-Columnstore.

      [root@localhost ~]#  mxs_adapter -c /etc/Columnstore.xml -u cdcuser -p cdcpassword -h 127.0.0.1  -P 4001 -r 2 -d -n  -f  stream-table-list
      2018-12-21 15:46:32 [main] Started thread 0x85f2a0
      2018-12-21 15:46:32 [main] Started thread 0x85f0c0
      2018-12-21 15:46:32 [main] Started 2 threads
      2018-12-21 15:46:32 [testA.t04] Requesting data for table: testA.t04
      2018-12-21 15:46:32 [testA.t03] Requesting data for table: testA.t03
      2018-12-21 15:46:32 [testA.t04] MaxScale connection could not be created: MaxScale responded with an error: ERR NO-FILE File 'testA.t04.000001.avro' not found.
       
      2018-12-21 15:46:41 [testA.t03] a: 1
      2018-12-21 15:46:41 [testA.t03] b: 1
      2018-12-21 15:46:41 [testA.t03] 1 rows, 0 transactions inserted over 0.161213 seconds. GTID = 0-1-366:1
      2018-12-21 15:46:41 [testA.t03] a: 2
      2018-12-21 15:46:41 [testA.t03] b: 1
      2018-12-21 15:46:41 [testA.t03] a: 3
      2018-12-21 15:46:41 [testA.t03] b: 1
      2018-12-21 15:46:41 [testA.t03] GTID 0-1-368
      2018-12-21 15:46:41 [testA.t03] 2 rows, 1 transactions inserted over 0.145406 seconds. GTID = 0-1-368:1
      2018-12-21 15:46:42 [testA.t03] a: 4
      2018-12-21 15:46:42 [testA.t03] b: 1
      2018-12-21 15:46:42 [testA.t03] a: 5
      2018-12-21 15:46:42 [testA.t03] b: 1
      2018-12-21 15:46:42 [testA.t03] GTID 0-1-370
      2018-12-21 15:46:42 [testA.t03] 2 rows, 1 transactions inserted over 0.153414 seconds. GTID = 0-1-370:1
      2018-12-21 15:46:43 [testA.t03] a: 6
      2018-12-21 15:46:43 [testA.t03] b: 1
      2018-12-21 15:46:43 [testA.t03] a: 7
      2018-12-21 15:46:43 [testA.t03] b: 1
      2018-12-21 15:46:43 [testA.t03] GTID 0-1-372
      2018-12-21 15:46:43 [testA.t03] 2 rows, 1 transactions inserted over 0.156 seconds. GTID = 0-1-372:1
      2018-12-21 15:46:44 [testA.t03] a: 8
      2018-12-21 15:46:44 [testA.t03] b: 1
      2018-12-21 15:46:44 [testA.t03] a: 9
      2018-12-21 15:46:44 [testA.t03] b: 1
      2018-12-21 15:46:44 [testA.t03] GTID 0-1-374
      2018-12-21 15:46:44 [testA.t03] 2 rows, 1 transactions inserted over 0.147712 seconds. GTID = 0-1-374:1
      2018-12-21 15:46:45 [testA.t03] a: 10
      2018-12-21 15:46:45 [testA.t03] b: 1
      2018-12-21 15:46:55 [testA.t03] Flushing batch
      2018-12-21 15:46:55 [testA.t03] 1 rows, 0 transactions inserted over 10.1583 seconds. GTID = 0-1-375:1
      2018-12-21 15:47:05 [testA.t03] Read timeout
      2018-12-21 15:47:15 [testA.t03] Read timeout
       
       
       
      2018-12-21 15:47:25 [testA.t03] Read timeout
      2018-12-21 15:47:36 [testA.t03] Read timeout
      2018-12-21 15:47:44 [testA.t03] a: 11
      2018-12-21 15:47:44 [testA.t03] b: 1
      2018-12-21 15:47:45 [testA.t03] 1 rows, 0 transactions inserted over 0.162896 seconds. GTID = 0-1-376:1
      2018-12-21 15:47:55 [testA.t03] Read timeout with empty batch, closing it
      2018-12-21 15:48:01 [testA.t03] a: 12
      2018-12-21 15:48:01 [testA.t03] b: 1
      2018-12-21 15:48:02 [testA.t03] 1 rows, 0 transactions inserted over 0.165726 seconds. GTID = 0-1-377:1
      2018-12-21 15:48:02 [testA.t03] a: 13
      2018-12-21 15:48:02 [testA.t03] b: 1
      2018-12-21 15:48:02 [testA.t03] a: 14
      2018-12-21 15:48:02 [testA.t03] b: 1
      2018-12-21 15:48:02 [testA.t03] GTID 0-1-379
      2018-12-21 15:48:03 [testA.t03] 2 rows, 1 transactions inserted over 0.161785 seconds. GTID = 0-1-379:1
      2018-12-21 15:48:03 [testA.t03] a: 15
      2018-12-21 15:48:03 [testA.t03] b: 1
      2018-12-21 15:48:03 [testA.t03] a: 16
      2018-12-21 15:48:03 [testA.t03] b: 1
      2018-12-21 15:48:03 [testA.t03] GTID 0-1-381
      2018-12-21 15:48:04 [testA.t03] 2 rows, 1 transactions inserted over 0.151464 seconds. GTID = 0-1-381:1
      
      

      MCS Columnstore :
      MariaDB [testA]> select * from  t03 ;
      +------+------+
      | a    | b    |
      +------+------+
      |    1 |    1 |      <<<<< Initial load  when mxs_adapter is started
      |    2 |    1 |
      |    3 |    1 |
      |    4 |    1 |
      |    5 |    1 |
      |    6 |    1 |
      |    7 |    1 |
      |    8 |    1 |
      |    9 |    1 |
      |   10 |    1 |     <<<<< Initial load
      |   11 |    1 |     <<<<< Subsequential load  after mxs_adapter was  started
      |   12 |    1 |
      |   13 |    1 |
      |   14 |    1 |
      |   15 |    1 |
      |   16 |    1 |     <<<<< Subsequential load 
      +------+------+
      16 rows in set (0.062 sec)
       
       
      Subsequent data on  t04 is not streamed  :
      MariaDB [testA]> select * from t04 ;
      Empty set (0.079 sec)
       
       
       
      On   MariaDB-TX  server :
       
      MariaDB [(none)]> select * from  testA.t03 ;
      +------+------+
      | a    | b    |
      +------+------+
      |    1 |    1 |
      |    2 |    1 |
      |    3 |    1 |
      |    4 |    1 |
      |    5 |    1 |
      |    6 |    1 |
      |    7 |    1 |
      |    8 |    1 |
      |    9 |    1 |
      |   10 |    1 |
      |   11 |    1 |
      |   12 |    1 |
      |   13 |    1 |
      |   14 |    1 |
      |   15 |    1 |
      |   16 |    1 |
      +------+------+
      16 rows in set (0.000 sec)
       
      MariaDB [(none)]> select * from  testA.t04 ;
      +------+------+
      | b    | c    |
      +------+------+
      |   12 |    1 |
      |   13 |    1 |
      |   14 |    1 |
      |   15 |    1 |
      |   16 |    1 |
      |   12 |    1 |
      |   13 |    1 |
      |   14 |    1 |
      |   15 |    1 |
      |   16 |    1 |
      +------+------+
      10 rows in set (0.000 sec)
       
      
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              winstone Zdravelina Sokolovska (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:

                  Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.