[MDEV-7624] Cannot query Oracle Table with VARCHAR2 data type Created: 2015-02-24  Updated: 2022-11-20  Resolved: 2022-11-20

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - Connect
Affects Version/s: 10.0.15
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Joffrey MICHAIE (Inactive) Assignee: Olivier Bertrand
Resolution: Not a Bug Votes: 0
Labels: None


 Description   

Hi,

I managed to connect an Oracle Table using Connect and the following options:

connect_work_size=2
connect_type_conv=1
optimizer_switch='engine_condition_pushdown=on' 

Trying to do automatic discovery:

MariaDB [daybreak_mariadb]> CREATE TABLE mytbl_oracle ENGINE=CONNECT TABLE_TYPE=ODBC tabname='mytbl' CONNECTION='DSN=db1;UID=user;PWD=pass';
ERROR 1105 (HY000): Cannot get columns from mytbl
MariaDB [daybreak_mariadb]> show errors;
+-------+------+-------------------------------------------------------------------------------------+
| Level | Code | Message |
+-------+------+-------------------------------------------------------------------------------------+
| Error | 1105 | Cannot get columns from mytbl                                                        |
| Error | 1030 | Got error 122 "Internal (unspecified) error in handler" from storage engine CONNECT |
+-------+------+-------------------------------------------------------------------------------------+
2 rows in set (0.00 sec) 

I know the columns, so I create the table manually (with varchar(xx))

Then, SELECT * Doesn't work:

ERROR 1296 (HY000): Got error 122 '[Oracle][ODBC]Error in assignment.' from CONNECT

Manual select of non varchar2 columns works.

I created a mapping table:

MariaDB> create table mytbl_columns engine=connect table_type=ODBC tabname=mytbl
catfunc=columns Connection='DSN=db1;UID=user;PWD=pass';

Problematic column is on the following type:

MariaDB [daybreak_mariadb]> select * from mytbl_columns where Column_Name='account';
+-----------+--------------+------------+-------------+-----------+-----------+-------------+---------------+----------------+-------+---------- +---------+
| Table_Cat | Table_Schema | Table_Name | Column_Name | Data_Type | Type_Name | Column_Size | Buffer_Length | Decimal_Digits | Radix | Nullable | Remarks | +-----------+--------------+------------+-------------+-----------+-----------+-------------+---------------+----------------+-------+---------- +---------+
| |DB |mytbl |account 12|VARCHAR2 | 30| 30| 0| 0| 0| | +-----------+--------------+------------+-------------+-----------+-----------+-------------+---------------+----------------+-------+---------- +---------+

Isn't there a mapping issue between varchar2 / oracle and varchar / mariadb ?

Thanks,
Joffrey



 Comments   
Comment by Olivier Bertrand [ 2015-02-25 ]

Options in MariaDB:
> connect_work_size=2
Bizarre, fortunately CONNECT takes a minimum value (This is actually the work memory size)
The variable giving the size of TEXT column is connect_conv_size but should not ne 2 anyhow.

> connect_type_conv=1
can also be given the value 'skip' (2) and the table will be created by discovery without the "wrong" column.

> optimizer_switch='engine_condition_pushdown=on'
Useless. This is now the default for MariaDB.

Let's come back to your problem.

This is very strange, type 12 is defined as VARCHAR and VARCHAR2 does not exist neither in sql.h nor in sqlext.h. Because the CONNECT type translate function works from the type number, not from its name, it should accept it (12 is defined as SQL_VARCHAR)

The problem is that I have no such table to debug this case. Can you provide one?

Comment by Olivier Bertrand [ 2015-02-25 ]

Sorry, I have many tables with VARCHAR2 columns, for instance:

create table countries_col
ENGINE=CONNECT CONNECTION='DSN=ORACLE_TEST;UID=system;DB=HR;PWD=manager' TABLE_TYPE=ODBC TABNAME=COUNTRIES CATFUNC=COL;
select * from countries_col;

Table_Cat Table_Schema Table_Name Column_Name Data_Type Type_Name Column_Size Buffer_Length Decimal_Digits Radix Nullable Remarks
<null> HR COUNTRIES COUNTRY_ID 1 CHAR 2 2 0 0 0 <null>
<null> HR COUNTRIES COUNTRY_NAME 12 VARCHAR2 40 40 0 0 1 <null>
<null> HR COUNTRIES REGION_ID 6 NUMBER 38 40 0 0 1 <null>

Now I create the table:

create table countries
ENGINE=CONNECT CONNECTION='DSN=ORACLE_TEST;UID=system;PWD=manager' TABLE_TYPE=ODBC TABNAME='HR.COUNTRIES';

It was Ok, then:

select * from countries;

COUNTRY_ID COUNTRY_NAME REGION_ID
AR Argentina 2
AU Australia 3
BE Belgium 1
BR Brazil 2
CA Canada 2
CH Switzerland 1
CN China 3
DE Germany 1
DK Denmark 1
EG Egypt 4
FR France 1
IL Israel 4
IN India 3
IT Italy 1
JP Japan 3
KW Kuwait 4
ML Malaysia 3
MX Mexico 2
NG Nigeria 4
NL Netherlands 1
SG Singapore 3
UK United Kingdom 1
US United States of America 2
ZM Zambia 4
ZW Zimbabwe 4

Everything Ok.
I am running Windows 7. My ORACLE ODBC driver is:
Oracle in XE version 11.02.00.02 Oracle Corporation SQORA32.DLL 02/11/2013
My data source ORACLE_TESTis defined as:
Description: Test Oracle
UserID: SYSTEM
Enable Result Sets
Enable Query Timeout
Enable Thread Safety
Commit only if all statement succeed
Use US settings
Fetch Buffer Size: 64000
Enable LOBs
Cache Buffer Size: 20
Enable Failover
Retry: 10
Delay: 10
Disable Microsoft Transaction Server
Disable RULE Hint

Comment by Christopher Halbersma [ 2015-04-13 ]

Hi,
I'm with the company Joffrey was trying this out on. I've replicated what I believe to be the issue. Destination instance is MariaDB 10.0.17-MariaDB-log source is Oracle 10.2.0.5.0. I've setup connect engines in a manner similar to how I believe Joffrey setup his system. I'm trying to bring in the ACCOUNTS tables into a empty database called remote_Daybreak.

Accounts setup in Oracle:

ColumnName,ColID,PK,IndexPos,Null,DataType,Default,Histogram,NumDistinct,NumNulls,Density,EncryptionAlg,Salt,Trigger,Virtual
ACC_AAD_ID,1,,5, 3, 2, 2, 1, 3,N,NUMBER,,Height Balanced,1113943,0,0,,,,
ACC_SAC_ID,2,,,N,NUMBER,0,None,1,0,1,,,,
ACC_PTC_COMPANY,3,,,N,VARCHAR2 (30 Byte),'UNDEFINED',Frequency,2,0,0,,,,
ACC_PCB_BRANCH_ORG,4,,,N,VARCHAR2 (30 Byte),'UNDEFINED',None,1,0,1,,,,
ACC_PCB_BRANCH,5,,,N,VARCHAR2 (30 Byte),'UNDEFINED',Frequency,3,0,0,,,,
ACC_APP_DT,6,,,N,DATE,SYSDATE,None,3770,0,0.00027,,,,
ACC_APP_NBR,7,,,N,VARCHAR2 (30 Byte),'UNDEFINED',None,1113943,0,0,,,,
ACC_APP_PURPOSE_CD,8,,,Y,VARCHAR2 (30 Byte),,None,0,1113959,0,,,,
ACC_APP_SOURCE_CD,9,,,N,VARCHAR2 (30 Byte),'UNDEFINED',None,1,0,1,,,,
ACC_APP_STATUS_CD,10,,,N,VARCHAR2 (30 Byte),'NEW',None,1,0,1,,,,
ACC_APP_SUB_STATUS_CD,11,,,N,VARCHAR2 (30 Byte),'UNDEFINED',None,1,0,1,,,,
ACC_APP_STATUS_CHG_DT_LAST,12,,,N,DATE,TO_DATE('01/01/1800',
'MM/DD/YYYY'),None,1,0,1,,,,
ACC_NBR,13,,1, 2,N,VARCHAR2 (30 Byte),'UNDEFINED',Height Balanced,1113943,0,0,,,,
+400ish lines. Can send the full thing if desired.

In MariaDB going in as the root user. Trying to create table.
[root@wfslxvddbcompare1 ~]# mysql
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 9
Server version: 10.0.17-MariaDB-log MariaDB Server

Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> use remote_Daybreak;
Database changed
MariaDB [remote_Daybreak]> create table accounts ENGINE=CONNECT TABLE_TYPE=ODBC tabname='ACCOUNTS' CONNECTION='DSN=ODBC;UID=daybreak;PWD=daybreak';
ERROR 1939 (HY000): Engine CONNECT failed to discover table `remote_Daybreak`.`accounts` with 'CREATE TABLE whatever (`ACC_AAD_ID` DOUBLE(22,0) NOT NULL,`ACC_SAC_ID` DOUBLE(22,0) NOT NULL,`ACC_PTC_COMPANY` VARCHAR(30) NOT NULL,`ACC_PCB_BRANCH_ORG` VARCHAR(30) NOT NULL,`ACC_PCB_BRANCH` VARCHAR(30) NOT NULL,`ACC_APP_DT` TIMESTAMP NOT NULL,`ACC_APP_NBR` VARCHAR(30) NOT NULL,`ACC_APP_PURPOSE_CD` VARCHAR(30),`ACC_APP_SOURCE_CD` VARCHAR(30) NOT NULL,`ACC_APP_STATUS_CD` VARCHAR(30) NOT NULL,`ACC_APP_SUB_STATUS_CD` VARCHAR(30) NOT NU
MariaDB [remote_Daybreak]> show errors;
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Level Code Message

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Error 1060 Duplicate column name 'ACC_AAD_ID'
Error 1939 Engine CONNECT failed to discover table `remote_Daybreak`.`accounts` with 'CREATE TABLE whatever (`ACC_AAD_ID` DOUBLE(22,0) NOT NULL,`ACC_SAC_ID` DOUBLE(22,0) NOT NULL,`ACC_PTC_COMPANY` VARCHAR(30) NOT NULL,`ACC_PCB_BRANCH_ORG` VARCHAR(30) NOT NULL,`ACC_PCB_BRANCH` VARCHAR(30) NOT NULL,`ACC_APP_DT` TIMESTAMP NOT NULL,`ACC_APP_NBR` VARCHAR(30) NOT NULL,`ACC_APP_PURPOSE_CD` VARCHAR(30),`ACC_APP_SOURCE_CD` VARCHAR(30) NOT NULL,`ACC_APP_STATUS_CD` VARCHAR(30) NOT NULL,`ACC_APP_SUB_STATUS_CD` VARCHAR(30) NOT NU
Error 1030 Got error 168 "Unknown (generic) error from engine" from storage engine CONNECT

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

3 rows in set (0.01 sec)
In /etc/odbinst.ini i had setup trace level logging.
{{[ODBC]
Trace = Yes
TraceFile = /tmp/odbc.log
ForceTrace = Yes
Pooling = No
DEBUG = 1}}
Here is the output for that:

[ODBC][28547][1428949088.341420][__handles.c][460]
Exit:[SQL_SUCCESS]
Environment = 0x7f334eaeea00
[ODBC][28547][1428949088.341569][SQLAllocHandle.c][375]
Entry:
Handle Type = 2
Input Handle = 0x7f334eaeea00
[ODBC][28547][1428949088.341647][SQLAllocHandle.c][493]
Exit:[SQL_SUCCESS]
Output Handle = 0x7f334eb78000
[ODBC][28547][1428949088.341701][SQLSetConnectOption.c][345]
Entry:
Connection = 0x7f334eb78000
Option = SQL_ATTR_ACCESS_MODE
Value = 1
[ODBC][28547][1428949088.341740][SQLSetConnectOption.c][508]
Exit:[SQL_SUCCESS]
+5k ish more lines

I know the error here is complaining about a duplicate column ("| Error | 1060 | Duplicate column name 'ACC_AAD_ID'"). But when I look at the definitions I got from oracle there seems to only be one column in this table.

I thought it could be a general setup issue but I was able to setup a smaller table (only 3 columns & 4 rows) in our environment. and it appears to pull data correctly.

CRH

Comment by Olivier Bertrand [ 2015-04-14 ]

A possible cause for "duplicate column name" is when your data source contains several tables with the same name in different databases (schema)
You can check this with a catalog table, for instance:

create table accounts ENGINE=CONNECT TABLE_TYPE=ODBC tabname='ACCOUNTS' CONNECTION='DSN=ODBC;UID=daybreak;PWD=daybreak' catfunc=table;  /* or catfunc=col */

If this is the case, specify the schema name of your table with the option dbname:

create table accounts ENGINE=CONNECT TABLE_TYPE=ODBC tabname='ACCOUNTS' DBNAME='schema_name' CONNECTION='DSN=ODBC;UID=daybreak;PWD=daybreak';

Generated at Thu Feb 08 07:21:01 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.