[#MCOL-2047] multi data streaming with mxs_adapter is not streaming data if table was initially empty

multi data streaming with mxs_adapter is not streaming data if table was initially empty

1. mxs_adapter is started with -f option with file describing the database and 2 table names to be streamed from MariaDB-TX to MariaDB-Columnstore.
both tables was create manually on MCS server .
table testA.t03 was loaded initially with data , testA.t04 was initially empty on MariaDB-TX server.
the data from MariaDB-TX.testA.t03 is streamed successful to MariaDB-Columnstore.testA.t03

2. Insert data to table testA.t03 on MariaDB-TX server
Data is streamed to testA.t03 on MariaDB-Columnstore.

3. Insert data to table testA.t04 on MariaDB-TX server
Data is not streamed to MariaDB-Columnstore.

[root@localhost ~]#  mxs_adapter -c /etc/Columnstore.xml -u cdcuser -p cdcpassword -h 127.0.0.1  -P 4001 -r 2 -d -n  -f  stream-table-list

2018-12-21 15:46:32 [main] Started thread 0x85f2a0

2018-12-21 15:46:32 [main] Started thread 0x85f0c0

2018-12-21 15:46:32 [main] Started 2 threads

2018-12-21 15:46:32 [testA.t04] Requesting data for table: testA.t04

2018-12-21 15:46:32 [testA.t03] Requesting data for table: testA.t03

2018-12-21 15:46:32 [testA.t04] MaxScale connection could not be created: MaxScale responded with an error: ERR NO-FILE File 'testA.t04.000001.avro' not found.

2018-12-21 15:46:41 [testA.t03] a: 1

2018-12-21 15:46:41 [testA.t03] b: 1

2018-12-21 15:46:41 [testA.t03] 1 rows, 0 transactions inserted over 0.161213 seconds. GTID = 0-1-366:1

2018-12-21 15:46:41 [testA.t03] a: 2

2018-12-21 15:46:41 [testA.t03] b: 1

2018-12-21 15:46:41 [testA.t03] a: 3

2018-12-21 15:46:41 [testA.t03] b: 1

2018-12-21 15:46:41 [testA.t03] GTID 0-1-368

2018-12-21 15:46:41 [testA.t03] 2 rows, 1 transactions inserted over 0.145406 seconds. GTID = 0-1-368:1

2018-12-21 15:46:42 [testA.t03] a: 4

2018-12-21 15:46:42 [testA.t03] b: 1

2018-12-21 15:46:42 [testA.t03] a: 5

2018-12-21 15:46:42 [testA.t03] b: 1

2018-12-21 15:46:42 [testA.t03] GTID 0-1-370

2018-12-21 15:46:42 [testA.t03] 2 rows, 1 transactions inserted over 0.153414 seconds. GTID = 0-1-370:1

2018-12-21 15:46:43 [testA.t03] a: 6

2018-12-21 15:46:43 [testA.t03] b: 1

2018-12-21 15:46:43 [testA.t03] a: 7

2018-12-21 15:46:43 [testA.t03] b: 1

2018-12-21 15:46:43 [testA.t03] GTID 0-1-372

2018-12-21 15:46:43 [testA.t03] 2 rows, 1 transactions inserted over 0.156 seconds. GTID = 0-1-372:1

2018-12-21 15:46:44 [testA.t03] a: 8

2018-12-21 15:46:44 [testA.t03] b: 1

2018-12-21 15:46:44 [testA.t03] a: 9

2018-12-21 15:46:44 [testA.t03] b: 1

2018-12-21 15:46:44 [testA.t03] GTID 0-1-374

2018-12-21 15:46:44 [testA.t03] 2 rows, 1 transactions inserted over 0.147712 seconds. GTID = 0-1-374:1

2018-12-21 15:46:45 [testA.t03] a: 10

2018-12-21 15:46:45 [testA.t03] b: 1

2018-12-21 15:46:55 [testA.t03] Flushing batch

2018-12-21 15:46:55 [testA.t03] 1 rows, 0 transactions inserted over 10.1583 seconds. GTID = 0-1-375:1

2018-12-21 15:47:05 [testA.t03] Read timeout

2018-12-21 15:47:15 [testA.t03] Read timeout

2018-12-21 15:47:25 [testA.t03] Read timeout

2018-12-21 15:47:36 [testA.t03] Read timeout

2018-12-21 15:47:44 [testA.t03] a: 11

2018-12-21 15:47:44 [testA.t03] b: 1

2018-12-21 15:47:45 [testA.t03] 1 rows, 0 transactions inserted over 0.162896 seconds. GTID = 0-1-376:1

2018-12-21 15:47:55 [testA.t03] Read timeout with empty batch, closing it

2018-12-21 15:48:01 [testA.t03] a: 12

2018-12-21 15:48:01 [testA.t03] b: 1

2018-12-21 15:48:02 [testA.t03] 1 rows, 0 transactions inserted over 0.165726 seconds. GTID = 0-1-377:1

2018-12-21 15:48:02 [testA.t03] a: 13

2018-12-21 15:48:02 [testA.t03] b: 1

2018-12-21 15:48:02 [testA.t03] a: 14

2018-12-21 15:48:02 [testA.t03] b: 1

2018-12-21 15:48:02 [testA.t03] GTID 0-1-379

2018-12-21 15:48:03 [testA.t03] 2 rows, 1 transactions inserted over 0.161785 seconds. GTID = 0-1-379:1

2018-12-21 15:48:03 [testA.t03] a: 15

2018-12-21 15:48:03 [testA.t03] b: 1

2018-12-21 15:48:03 [testA.t03] a: 16

2018-12-21 15:48:03 [testA.t03] b: 1

2018-12-21 15:48:03 [testA.t03] GTID 0-1-381

2018-12-21 15:48:04 [testA.t03] 2 rows, 1 transactions inserted over 0.151464 seconds. GTID = 0-1-381:1

MCS Columnstore :

MariaDB [testA]> select * from  t03 ;

+------+------+

| a    | b    |

+------+------+

|    1 |    1 |      <<<<< Initial load  when mxs_adapter is started

|    2 |    1 |

|    3 |    1 |

|    4 |    1 |

|    5 |    1 |

|    6 |    1 |

|    7 |    1 |

|    8 |    1 |

|    9 |    1 |

|   10 |    1 |     <<<<< Initial load

|   11 |    1 |     <<<<< Subsequential load  after mxs_adapter was  started

|   12 |    1 |

|   13 |    1 |

|   14 |    1 |

|   15 |    1 |

|   16 |    1 |     <<<<< Subsequential load

+------+------+

16 rows in set (0.062 sec)

Subsequent data on  t04 is not streamed  :

MariaDB [testA]> select * from t04 ;

Empty set (0.079 sec)

On   MariaDB-TX  server :

MariaDB [(none)]> select * from  testA.t03 ;

+------+------+

| a    | b    |

+------+------+

|    1 |    1 |

|    2 |    1 |

|    3 |    1 |

|    4 |    1 |

|    5 |    1 |

|    6 |    1 |

|    7 |    1 |

|    8 |    1 |

|    9 |    1 |

|   10 |    1 |

|   11 |    1 |

|   12 |    1 |

|   13 |    1 |

|   14 |    1 |

|   15 |    1 |

|   16 |    1 |

+------+------+

16 rows in set (0.000 sec)

MariaDB [(none)]> select * from  testA.t04 ;

+------+------+

| b    | c    |

+------+------+

|   12 |    1 |

|   13 |    1 |

|   14 |    1 |

|   15 |    1 |

|   16 |    1 |

|   12 |    1 |

|   13 |    1 |

|   14 |    1 |

|   15 |    1 |

|   16 |    1 |

+------+------+

10 rows in set (0.000 sec)

[MCOL-2047] multi data streaming with mxs_adapter is not streaming data if table was initially empty Created: 2018-12-21 Updated: 2023-10-26 Resolved: 2022-11-05
Status:	Closed
Project:	MariaDB ColumnStore
Component/s:	?
Affects Version/s:	1.2.2
Fix Version/s:	Icebox

[MCOL-2047] multi data streaming with mxs_adapter is not streaming data if table was initially empty Created: 2018-12-21 Updated: 2023-10-26 Resolved: 2022-11-05