[MDEV-8882] CONNECT Engine - Unable to Merge Mutliple SQLite - Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Fixed
Affects Version/s: 10.0.19
Fix Version/s: 10.0.22, 10.1.9
Component/s: Storage Engine - Connect
Labels:
None
Environment:
Centos 7

Description

We're trying to merge two sqlite databases that are similar to those outlined below, and see the following when we try to query the resulting table in MariaDB:

/usr/local/data/SQLiteDB_1.dat < create table users ( id varchar(10), name varchar(24) );

-----------------

| id  |  name

| f3  | Alice

| 6g | Bob

/usr/local/data/SQLiteDB_2.dat < create table users( id varchar(10), name varchar(24) );

------------------

| id   |  name

| s1  | Alison

| 9y  | Robert

MariaDB > create table people ( id varchar(10), name varchar(24) ) ENGINE=CONNECT TABLE_TYPE=ODBC multiple=1 tabname='users' CONNECTION='DSN=SQLite3 Datasource;DBQ=/usr/local/data/SQLiteDB_*.dat;';

MariaDB > select * from people:

----------------

| id  | name

| f3   | Alice

| 6g  | Bob

| f3   | Alice

| 6g  | Bob

Attachments

Activity

Ascending order - Click to sort in descending order

Brandon Darbyson created issue - 2015-10-02 18:33

Brandon Darbyson made changes - 2015-10-02 18:35

Field	Original Value	New Value
Description	We're trying to merge two sqlite databases that are similar to those outlined below, and see the following when we try to query the resulting table in MariaDB: /usr/local/data/SQLiteDB_1.dat < create table users ( id varchar(10), name varchar(24) ); ----------------- \| id \| name ----------------- \| f3 \| Alice \| 6g \| Bob ----------------- /usr/local/data/SQLiteDB_2.dat < create table users( id varchar(10), name varchar(24) ); ------------------ \| id \| name ------------------ \| s1 \| Alison \| 9y \| Robert ------------------ MariaDB > create table people ( id varchar(10), name varchar(24) ) ENGINE=CONNECT TABLE_TYPE=ODBC multiple=1 tabname='users' CONNECTION='DSN=SQLite3 Datasource;DBQ=/usr/local/data/SQLiteDB_.dat;'; MariaDB > select from people: ---------------- \| id \| name ---------------- \| f3 \| Alice \| 6g \| Bob \| f3 \| Alice \| 6g \| Bob ----------------	We're trying to merge two sqlite databases that are similar to those outlined below, and see the following when we try to query the resulting table in MariaDB: /usr/local/data/SQLiteDB_1.dat < create table users ( id varchar(10), name varchar(24) ); ----------------- \| id \| name \| f3 \| Alice \| 6g \| Bob ----------------- /usr/local/data/SQLiteDB_2.dat < create table users( id varchar(10), name varchar(24) ); ------------------ \| id \| name \| s1 \| Alison \| 9y \| Robert ------------------ MariaDB > create table people ( id varchar(10), name varchar(24) ) ENGINE=CONNECT TABLE_TYPE=ODBC multiple=1 tabname='users' CONNECTION='DSN=SQLite3 Datasource;DBQ=/usr/local/data/SQLiteDB_.dat;'; MariaDB > select from people: ---------------- \| id \| name \| f3 \| Alice \| 6g \| Bob \| f3 \| Alice \| 6g \| Bob ----------------

Brandon Darbyson made changes - 2015-10-02 18:36

Description

We're trying to merge two sqlite databases that are similar to those outlined below, and see the following when we try to query the resulting table in MariaDB:

/usr/local/data/SQLiteDB_1.dat < create table users ( id varchar(10), name varchar(24) );
-----------------
| id | name
| f3 | Alice
| 6g | Bob
-----------------

/usr/local/data/SQLiteDB_2.dat < create table users( id varchar(10), name varchar(24) );
------------------
| id | name
| s1 | Alison
| 9y | Robert
------------------

MariaDB > create table people ( id varchar(10), name varchar(24) ) ENGINE=CONNECT TABLE_TYPE=ODBC multiple=1 tabname='users' CONNECTION='DSN=SQLite3 Datasource;DBQ=/usr/local/data/SQLiteDB_*.dat;';

MariaDB > select * from people:

----------------
| id | name
| f3 | Alice
| 6g | Bob
| f3 | Alice
| 6g | Bob
----------------

We're trying to merge two sqlite databases that are similar to those outlined below, and see the following when we try to query the resulting table in MariaDB:

/usr/local/data/SQLiteDB_1.dat < create table users ( id varchar(10), name varchar(24) );
-----------------
| id | name
| f3 | Alice
| 6g | Bob

/usr/local/data/SQLiteDB_2.dat < create table users( id varchar(10), name varchar(24) );
------------------
| id | name
| s1 | Alison
| 9y | Robert

MariaDB > create table people ( id varchar(10), name varchar(24) ) ENGINE=CONNECT TABLE_TYPE=ODBC multiple=1 tabname='users' CONNECTION='DSN=SQLite3 Datasource;DBQ=/usr/local/data/SQLiteDB_*.dat;';

MariaDB > select * from people:

----------------
| id | name
| f3 | Alice
| 6g | Bob
| f3 | Alice
| 6g | Bob

Elena Stepanova made changes - 2015-10-21 01:09

Description

We're trying to merge two sqlite databases that are similar to those outlined below, and see the following when we try to query the resulting table in MariaDB:

/usr/local/data/SQLiteDB_1.dat < create table users ( id varchar(10), name varchar(24) );
-----------------
| id | name
| f3 | Alice
| 6g | Bob

/usr/local/data/SQLiteDB_2.dat < create table users( id varchar(10), name varchar(24) );
------------------
| id | name
| s1 | Alison
| 9y | Robert

MariaDB > create table people ( id varchar(10), name varchar(24) ) ENGINE=CONNECT TABLE_TYPE=ODBC multiple=1 tabname='users' CONNECTION='DSN=SQLite3 Datasource;DBQ=/usr/local/data/SQLiteDB_*.dat;';

MariaDB > select * from people:

----------------
| id | name
| f3 | Alice
| 6g | Bob
| f3 | Alice
| 6g | Bob

We're trying to merge two sqlite databases that are similar to those outlined below, and see the following when we try to query the resulting table in MariaDB:

{noformat}
/usr/local/data/SQLiteDB_1.dat < create table users ( id varchar(10), name varchar(24) );
-----------------
| id | name
| f3 | Alice
| 6g | Bob
{noformat}

{noformat}
/usr/local/data/SQLiteDB_2.dat < create table users( id varchar(10), name varchar(24) );
------------------
| id | name
| s1 | Alison
| 9y | Robert
{noformat}

{noformat}
MariaDB > create table people ( id varchar(10), name varchar(24) ) ENGINE=CONNECT TABLE_TYPE=ODBC multiple=1 tabname='users' CONNECTION='DSN=SQLite3 Datasource;DBQ=/usr/local/data/SQLiteDB_*.dat;';

MariaDB > select * from people:

----------------
| id | name
| f3 | Alice
| 6g | Bob
| f3 | Alice
| 6g | Bob
{noformat}

Elena Stepanova made changes - 2015-10-21 01:10

Description

We're trying to merge two sqlite databases that are similar to those outlined below, and see the following when we try to query the resulting table in MariaDB:

{noformat}
/usr/local/data/SQLiteDB_1.dat < create table users ( id varchar(10), name varchar(24) );
-----------------
| id | name
| f3 | Alice
| 6g | Bob
{noformat}

{noformat}
/usr/local/data/SQLiteDB_2.dat < create table users( id varchar(10), name varchar(24) );
------------------
| id | name
| s1 | Alison
| 9y | Robert
{noformat}

{code:sql}
MariaDB > create table people ( id varchar(10), name varchar(24) ) ENGINE=CONNECT TABLE_TYPE=ODBC multiple=1 tabname='users' CONNECTION='DSN=SQLite3 Datasource;DBQ=/usr/local/data/SQLiteDB_*.dat;';

MariaDB > select * from people:

----------------
| id | name
| f3 | Alice
| 6g | Bob
| f3 | Alice
| 6g | Bob
{code}

Elena Stepanova made changes - 2015-10-21 01:10

Assignee

Olivier Bertrand [ bertrandop ]

Olivier Bertrand made changes - 2015-10-21 20:42

Fix Version/s		10.0.22 [ 19700 ]
Fix Version/s		10.1.9 [ 20301 ]
Resolution		Fixed [ 1 ]
Status	Open [ 1 ]	Closed [ 6 ]

Olivier Bertrand added a comment - 2015-10-21 20:54 - edited

The problem is that the sqlite ODBC driver does not recognize the DBQ keyword. The merge table should be created as:

create table people ( id varchar(10), name varchar(24) )

ENGINE=CONNECT TABLE_TYPE=ODBC multiple=1 tabname='users'

CONNECTION='DSN=SQLite3 Datasource;Database=/usr/local/data/SQLiteDB_*.dat;';

This is all right for simple tables but for multiple tables CONNECT failed to retrieve the database name because only looking for the DBQ keyword. This is what will be fixed in next versions of MariaDB.

Meanwhile, you can alternatively merge the tables using the TBL table type:

create table users1 (id varchar(10), name varchar(24)) engine=connect table_type=ODBC tabname='users'

CONNECTION='Driver=SQLite3 ODBC Driver;Database=/usr/local/data/SQLiteDB_1.dat';

create table users2 (id varchar(10), name varchar(24)) engine=connect table_type=ODBC tabname='users'

CONNECTION='Driver=SQLite3 ODBC Driver;Database=/usr/local/data/SQLiteDB_2.dat';

create table people2 (id varchar(10), name varchar(24)) ENGINE=CONNECT TABLE_TYPE=TBL table_list='users1,users2';

Olivier Bertrand added a comment - 2015-10-21 20:54 - edited The problem is that the sqlite ODBC driver does not recognize the DBQ keyword. The merge table should be created as: create table people ( id varchar(10), name varchar(24) ) ENGINE=CONNECT TABLE_TYPE=ODBC multiple=1 tabname='users' CONNECTION='DSN=SQLite3 Datasource;Database=/usr/local/data/SQLiteDB_*.dat;'; This is all right for simple tables but for multiple tables CONNECT failed to retrieve the database name because only looking for the DBQ keyword. This is what will be fixed in next versions of MariaDB. Meanwhile, you can alternatively merge the tables using the TBL table type: create table users1 (id varchar(10), name varchar(24)) engine=connect table_type=ODBC tabname='users' CONNECTION='Driver=SQLite3 ODBC Driver;Database=/usr/local/data/SQLiteDB_1.dat'; create table users2 (id varchar(10), name varchar(24)) engine=connect table_type=ODBC tabname='users' CONNECTION='Driver=SQLite3 ODBC Driver;Database=/usr/local/data/SQLiteDB_2.dat'; create table people2 (id varchar(10), name varchar(24)) ENGINE=CONNECT TABLE_TYPE=TBL table_list='users1,users2';

Sergei Golubchik made changes - 2021-12-06 21:42

Workflow

MariaDB v3 [ 71855 ]

MariaDB v4 [ 149674 ]

People

Assignee:: Olivier Bertrand

Reporter:: Brandon Darbyson

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 2015-10-02 18:33

Updated:: 2015-10-21 20:57

Resolved:: 2015-10-21 20:42

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server

Details

Description

Attachments

Activity

People

Dates

Git Integration