[MDEV-5144] Using connect engine to connect to SQL Server 2008 R2 from mysql on linux Created: 2013-10-15  Updated: 2013-11-01  Resolved: 2013-11-01

Status: Closed
Project: MariaDB Server
Component/s: None
Affects Version/s: 10.0.4
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Eric Hernandez Assignee: Alexander Barkov
Resolution: Not a Bug Votes: 0
Labels: connect-engine
Environment:

Debian 7.1 and CentOS 6.4



 Description   

I am trying to connect to SQL Server 2008 R2 using the connect engine and running mysql from linux

I have tried on two different platforms to make this work
Debian 7.1 64-bit and CentOS 6.4 64 bit.

############
Debian 7.1 details
Linux db149 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1+deb7u1 x86_64 GNU/Linux

relevant packages installed:

ii  libmariadbclient18                 10.0.4+maria-1~wheezy         amd64        MariaDB database client library
ii  libmysqlclient18                   10.0.4+maria-1~wheezy         amd64        Virtual package to satisfy external depends
ii  mariadb-client-10.0                10.0.4+maria-1~wheezy         amd64        MariaDB database client binaries
ii  mariadb-client-core-10.0           10.0.4+maria-1~wheezy         amd64        MariaDB database core client binaries
ii  mariadb-common                     10.0.4+maria-1~wheezy         all          MariaDB database common files (e.g. /etc/mysql/conf.d/mariadb.cnf)
iU  mariadb-connect-engine-10.0        10.0.4+maria-1~wheezy         all          Connect storage engine for MariaDB
iF  mariadb-server-10.0                10.0.4+maria-1~wheezy         amd64        MariaDB database server binaries
ii  mariadb-server-core-10.0           10.0.4+maria-1~wheezy         amd64        MariaDB database core server files
ii  libodbc1:amd64                     2.2.14p2-5                    amd64        ODBC library for Unix
ii  odbcinst                           2.2.14p2-5                    amd64        Helper program for accessing odbc ini files
ii  odbcinst1debian2:amd64             2.2.14p2-5                    amd64        Support library for accessing odbc ini files
ii  tdsodbc:amd64                      0.91-2                        amd64        ODBC driver for connecting to MS SQL and Sybase SQL servers
ii  unixodbc                           2.2.14p2-5                    amd64        Basic ODBC tools
ii  unixodbc-dev                       2.2.14p2-5                    amd64        ODBC libraries for UNIX (development files)
ii  freetds-bin                        0.91-2+deb7u1                 amd64        FreeTDS command-line utilities
ii  freetds-common                     0.91-2                        all          configuration files for FreeTDS SQL client libraries

cat /etc/odbc.ini
[MSSQLTestServer]
Driver  = FreeTDS
Server      = 192.168.1.20
Port      = 1433

cat /etc/odbcinst.ini
[FreeTDS]
Description	= TDS driver (Sybase/MS SQL)
Driver		= libtdsodbc.so
Setup		= libtdsS.so
CPTimeout	=
CPReuse		=
Trace   = yes
TraceFile    = /tmp/sql.log
ForceTrace   = Yes
Database        = master

Proof that odbc is working from linux to Sql server via the command line. Note that credentials have to be supplied. I have sanitized the username and password.

isql -v MSSQLTestServer user password 
+---------------------------------------+
| Connected!                            |
|                                       |
| sql-statement                         |
| help [tablename]                      |
| quit                                  |
|                                       |
+---------------------------------------+
SQL> select * from connect_table;
+------------+
| id         |
+------------+
| 1001       |
| 1002       |
+------------+
SQLRowCount returns 2
2 rows fetched

Attempt to create connect table from within maria

(192.168.1.20:test:5)$ CREATE TABLE connect_table (
    ->   id int(10) NOT NULL
    -> ) ENGINE=CONNECT DEFAULT CHARSET=latin1 CONNECTION='DSN=MSSQLTestServer' table_type=odbc block_size=10 tabname='connect_table';
Query OK, 0 rows affected (0.00 sec)
 
(192.168.1.20:test:6)$ select * from connect_table;
ERROR 1296 (HY000): Got error 174 '[unixODBC][FreeTDS][SQL Server]Unable to connect to data source' from CONNECT
(192.168.1.20:test:7)$

##################################
CentOs 6.4 details

CentOS release 6.4 (Final)
Linux maria1 2.6.32-358.el6.x86_64 #1 SMP Fri Feb 22 00:31:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

MariaDB-common-10.0.4-1.x86_64
MariaDB-connect-engine-10.0.4-1.x86_64
MariaDB-compat-10.0.4-1.x86_64
MariaDB-server-10.0.4-1.x86_64
MariaDB-client-10.0.4-1.x86_64
freetds-0.64-1.el6.rf.x86_64
unixODBC-2.2.14-12.el6_3.x86_64

cat /etc/odbc.ini
[MSSQLTestServer]
Driver  = ODBC Driver 11 for SQL Server
Server      = 192.168.1.20
Port      = 1433

cat /etc/odbcinst.ini
[ODBC Driver 11 for SQL Server]
Description=Microsoft ODBC Driver 11 for SQL Server
Driver=/opt/lib/libmsodbcsql-11.0.so.2270.0
Threading=1
UsageCount=1

Ok on CentOS I first tried to setup Integrated Authentication from Linux to Microsoft SQL Server and I was successful.
I followed this instruction http://msdn.microsoft.com/en-us/library/hh568451.aspx

Here is the proof that it works.

./sqlcmd -E -S SVDWDBCLN1.sellingsource.local -d master
1> select * from connect_table;
2> go
id
-----------
       1001
       1002
 
(2 rows affected)

Now keep in mind that in prior test on debian i used freetds as my SQL Server driver.

Here is the result from MySQL Client

MariaDB [test]> CREATE TABLE `connect_table` (
    ->   `id` int(10) NOT NULL
    -> ) ENGINE=CONNECT DEFAULT CHARSET=latin1 CONNECTION='DSN=MSSQLTestServer' `table_type`=odbc `block_size`=10 `tabname`='connect_table';
Query OK, 0 rows affected (0.00 sec)
 
MariaDB [test]> show tables;
+----------------+
| Tables_in_test |
+----------------+
| connect_table  |
+----------------+
1 row in set (0.00 sec)
 
MariaDB [test]> select * from connect_table;
ERROR 1296 (HY000): Got error 174 '[unixODBC][Driver Manager]Can't open lib '/opt/lib/libmsodbcsql-11.0.so.2270.0' : file not found' from CONNECT



 Comments   
Comment by Alexander Barkov [ 2013-10-23 ]

Can you please specify UID and PWD in the connection string on Debian, like this:

CREATE TABLE connect_table (
id int(10) NOT NULL
) ENGINE=CONNECT DEFAULT CHARSET=latin1 CONNECTION='DSN=MSSQLTestServer;UID=sa;PWD=sapwd' table_type=odbc block_size=10 tabname='connect_table';

Note, ConnectSE supports so called discovery, so in many cases it's ok not to specify table structure,
it should be detected automatically. Also, if the local and the remote table names are the same,
then no needs to specify the tabname option. This is the minimum possible create statetent:

CREATE TABLE connect_table ENGINE=CONNECT CONNECTION='DSN=MSSQLTestServer;UID=sa;PWD=passwd' table_type=odbc;

Comment by Alexander Barkov [ 2013-10-23 ]

I will check with the native MS SQL driver for Linux and report back separately.

Comment by Alexander Barkov [ 2013-10-23 ]

Btw, did you reinstall unixODBC to the version 2.3.0, which the MS driver requires ?
CentOS-6.4 seems to have unixODBC-2.2.14.

Comment by Eric Hernandez [ 2013-10-23 ]

Yes i did custom compile unixODBC 2.3.0 as per instruction on the microsoft site and I was successfully able to do integrated authentication from the command line (on centos).

I will try again with the UID and PWD soon.

Comment by Eric Hernandez [ 2013-10-23 ]

So that worked!!! I guess all i was missing was the credentials in the table definition.
Was that in the documentation?

Comment by Alexander Barkov [ 2013-10-23 ]

Eric, excellent. Good to know. Olivier, please consider documenting this detail.

Comment by Alexander Barkov [ 2013-10-23 ]

Eric, can you please clarify why you're trying to use the Microsoft native driver on the CentOS box,
instead of the FreeTDS driver. The latter is a part of the CentOS distribution, thus consistently works
with unixODBC and the other software involved (no needs to do custom compule etc).

Is there anything in the MS driver that FreeTDS does not support, or are you trying it
mostly out of curiosity?

Comment by Olivier Bertrand [ 2013-10-24 ]

I am mostly familiar with Windows ODBC (the original product) but unixODBC has probably the same features:

ODBC connections can be specified in two ways:
1) Specifying the driver to use (via DRIVER=... in the connection string)
2) Specifying a data source (via DSN=... in the connection string)
The first way is more complicated because all parameters required for the connection must be specified in the connection string.
When using the second way, ODBC uses the parameters defined for the data source and you have to add only the ones not yet defined in the data source description.
For unixODBC, the data source is defined as an entry in the odbc.ini configuration file. It is quite possible to define several data sources using the same driver. For instance:

[MSSQLTestServer]
Driver = ODBC Driver 11 for SQL Server
Server = 192.168.1.20
Port = 1433

[MyMSSQL]
Driver = ODBC Driver 11 for SQL Server
Server = 192.168.1.20
Port = 1433
UID = sa
PWD = sapwd

Doing so you should be able to create your table just saying:

CREATE TABLE connect_table ENGINE=CONNECT table_type=odbc CONNECTION='DSN=MyMSSQL';

The equivalent works fine on Windows, I did not yet test it on Linux (I am using ubuntu)

Note that the parameters required for a connection depend on the ODBC connector you use. They cannot be described in the CONNECT documentation and you should refer to the used connector documentation to know what they are.

Comment by Eric Hernandez [ 2013-10-24 ]

Alex,
I was just troubleshooting and was just trying to see if the FreeTDS was the issue but it was not.
I don't even normally run CentOS i mostly use debian but the microsoft site only had rpms for their driver.

Comment by Alexander Barkov [ 2013-10-24 ]

Olivier,
according to the FreeTDS documentation:
http://freetds.schemamania.org/userguide/odbcconnattr.htm
UID and PWD can only be on the connection string,
and cannot be used in odbc.ini in the DSN configuration.

It would be nice to have a new manual section with tips how to use various ODBC drivers.

Comment by Alexander Barkov [ 2013-10-24 ]

Eric,

Have you solved all the problems you had? Can we close the issue?

Thanks for fast feedback!

Comment by Eric Hernandez [ 2013-10-25 ]

Yes this problem is solved now. Thank you for the help.

However I am probably going to open up some other tickets because I am
having other issues. Like my where clause not being passed to SQL Server or
every time i do a query to SQL Server the first thing it does is a select
count which takes a long time on the sql server side and I am getting
some random crashes when i do a select * from table ..

On Wed, Oct 23, 2013 at 11:31 PM, Alexander Barkov (JIRA) <


Eric H.
eric.hardway@gmail.com

Comment by Olivier Bertrand [ 2013-10-25 ]

To access a table via ODBC, CONNECT rephrases the original query and send it to the
data source. The issue is that the entered query if firstly parsed by MariaDB and
therefore must obey the MySQL syntax, while the rephrased query must obey the data
source syntax.

Because both are using SQL, this is generally not a problem. However, because some
details of the syntax can differ, for instance the name of scalar functions, the
rephrased query just ask the used columns values. For instance:

select upper(`first name`), `Salary` * 0.95 from `Employee`;

could be rephrased as:

SELECT "first name", "Salary" FROM "Employee"

The scalar function and expression will be locally calculated by MariaDB. See below
for similar restrictions regarding the WHERE clause.

Where clause not being passed to the data source:
-------------------------------------------------
There can be two causes:
1 - To be able to retrieve the where clause, the storage engine function cond_push
must be called by MariaDB. Since MariaDB 5.5 the engine condition pushdown is OFF by default.
It is therefore necessary to set it ON, for instance by:

set optimizer_switch='engine_condition_pushdown=on';

Or starting mysqld with this parameter set to ON, for instance:

mysqld --console --engine_condition_pushdown=on

Note 1: specifying -console is important to have some error messages from CONNECT printed
because MariaDB does not always retrieve them.

Note 2: since MariaDB 10.0.4, the CONDITION_PUSHDOWN argument is no more accepted.
However, it is no more needed because CONNECT uses condition pushdown unconditionally.

2 - The technique used above (getting only column values) obviously cannot be used here because
the where clause must be executed by the data source. This is why CONNECT extracts only the
"compatible" part of query WHERE clauses and add it to the ODBC query. Currently, clauses
containing scalar functions, expressions or subselect are not passed to the data source.

select Count from table
--------------------------
This was done to answer the info internal engine function. However, this info is not really
important and returning a fixed guess is enough. Therefore, this will be suppressed in the
next version of CONNECT.

Random crashes
--------------
This must be treated by bug reports giving all details allowing to trace the cause of them.

New version
-----------
The new version of CONNECT (in MariaDB 10.0.5) will have many new features addressing in
particular most of the issues listed above.

Comment by Alexander Barkov [ 2013-11-01 ]

Eric, I'm closing this ticket.
Please file separate tickets for the remaining problems.
Thanks.

Generated at Thu Feb 08 07:02:01 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.