[CONC-281] mysql_stmt_fetch_column() should work without mysql_stmt_store_result() Created: 2017-09-24  Updated: 2019-01-15  Resolved: 2017-09-25

Status: Closed
Project: MariaDB Connector/C
Component/s: None
Affects Version/s: 2.3.3, 3.0.2
Fix Version/s: 2.3.4, 3.0.3

Type: Bug Priority: Major
Reporter: Kirill Müller Assignee: Georg Richter
Resolution: Fixed Votes: 0
Labels: None
Environment:

Ubuntu Linux, test compiled from source, run against MySQL 5.7.19


Issue Links:
Relates
relates to CONC-282 Provide additional version number for... Closed
relates to CONC-301 Using mysql_stmt_bind_result() in pre... Closed

 Description   

It seems that fetching columns of blob or varchar type via mysql_stmt_fetch_column() requires a call to mysql_stmt_store_result() after mysql_stmt_execute(). Otherwise all attempts to fetch the data result in CR_NO_DATA. This is different in the MySQL C connector, and I don't see why I need to bring the entire result set to the client just to make available all blobs in a row. (At least this is how I understand the documentation.) This might severely impact the performance of the new RMariaDB database backend for R.

Removing all calls to mysql_stmt_store_result() from the tests gives two failures, see [1] for a patch:

  1. port: 0
  2. socketname: (null)
    1..13
  3. Testing against MySQL Server 5.7.19-0ubuntu0.17.04.1
  4. Host: Localhost via UNIX socket
  5. Client library: 10.2.6
  6. Error: (/home/muelleki/git/mariadb-connector-c/unittest/libmariadb/fetch.c: 241)
    not ok 1 - test_fetch_seek
  7. truncation: 1
  8. Error: (/home/muelleki/git/mariadb-connector-c/unittest/libmariadb/fetch.c: 316)
    not ok 2 - test_fetch_offset
    ok 3 - test_fetch_column
    ok 4 - test_fetch_nobuffs
    ok 5 - test_fetch_null
    ok 6 - test_fetch_date
    ok 7 - test_fetch_str
    ok 8 - test_fetch_long
    ok 9 - test_fetch_short
    ok 10 - test_fetch_tiny
    ok 11 - test_fetch_bigint
    ok 12 - test_fetch_float
    ok 13 - test_fetch_double
  9. close default
  10. Failed 2 tests!

[1] https://github.com/krlmlr/mariadb-connector-c/commit/5db25b0e0448046adec7bdf8ee8cbf47228be5ac



 Comments   
Comment by Georg Richter [ 2017-09-25 ]

The patched tests fail of course, since you can't use mysql_stmt_seek with an unbuffered result set.
If you want to use mysql_stmt_fetch_column with an unbuffered result set, you have to fetch data first:

mysql_stmt_execute(stmt);
mysql_stmt_fetch(stmt); /* load first row into buffer */
mysql_stmt_fetch_column(stmt, ....

Please also note, that MySQL Connector/C will fail with same errors (both tests are part of mysql_client_test.c)

  DBUG_ENTER("mysql_stmt_fetch_column");
 
  if ((int) stmt->state < (int) MYSQL_STMT_FETCH_DONE)
  {
    set_stmt_error(stmt, CR_NO_DATA, unknown_sqlstate, NULL);
    DBUG_RETURN(1);
  }

Comment by Kirill Müller [ 2017-09-25 ]

I'm pretty sure this is what I had done before - execute, fetch, fetch column. This worked for all fixed-size data types, but consistently failed for varchar and blob, and only succeeded after I changed the code to execute, store, fetch, fetch column. I spent the better part of yesterday isolating the problem.

The test cases might not reflect the problem I was experiencing. I'll follow up with more information, but perhaps you can replicate?

Comment by Georg Richter [ 2017-09-25 ]

Yes, it's reproducable.- if mysql_stmt_bind_result was not called before, the internal stmt->bind structure has incorrect length and null values.

Comment by Kirill Müller [ 2017-09-25 ]

Awesome, thanks for taking another look!

Comment by Georg Richter [ 2017-09-25 ]

connector_c_2.3 branch:
commit a028307e4182eb7f07c14e5cf7f32b358a3ccd1c

master_branch (3.0.3)
commit 17110fb9298fb328759bef287ceb72f97cc81a04

Comment by Kirill Müller [ 2017-09-25 ]

Wow! Thanks again for the fast resolution.

Is there a way to retrieve the connector version in a robust way (e.g. in my `configure` script), so that I can keep the workaround for old versions?

Comment by Georg Richter [ 2017-09-25 ]

You can use

{mariadb_config}

$ bin/mariadb_config 
Copyright 2011-2017 MariaDB Corporation AB
Get compiler flags for using the MariaDB Connector/C.
Usage: mariadb_config/mariadb_config [OPTIONS]
  --cflags        [-I/usr/local/include/mariadb -I/usr/local/include/mariadb/mysql]
  --include       [-I/usr/local/include/mariadb -I/usr/local/include/mariadb/mysql]
  --libs          [-L/usr/local/lib/mariadb/ -lmariadb -lpthread -ldl -lm -lssl -lcrypto]
  --libs_r        [-L/usr/local/lib/mariadb/ -lmariadb -lpthread -ldl -lm -lssl -lcrypto]
  --libs_sys      [-lpthread -ldl -lm -lssl -lcrypto]
  --version       [10.2.9]
  --socket        [/tmp/mysql.sock]
  --port          [3306]
  --plugindir     [/usr/local/lib/mariadb/plugin]
  --tlsinfo       [OpenSSL 1.1.1]

Comment by Kirill Müller [ 2017-09-25 ]

Thanks. But how does the 10.2.9 returned by `mariadb_config` correspond to the C connector version (2.3.x or 3.0.x)?

Comment by Kirill Müller [ 2017-10-01 ]

I'd like to request clarification:

  • Is there a workaround for this problem? Unfortunately, storing the entire result at the client seems to have other problems, see https://github.com/rstats-db/RMariaDB/issues/60 and https://github.com/rstats-db/RMariaDB/issues/61.
  • Which versions are affected, only 2.3.3 or several versions prior to 2.3.3? And for version 3, only 3.0.2 or also 3.0.1 and 3.0.0? Also, I can't seem to replicate this problem when building my library against libmariadb-dev 2.3.2-2 on Ubuntu – why?
  • How do I reliably determine from the output of `mariadb_config` if the installed version is affected by the problem?

Thanks.

Comment by Brad House [ 2018-07-18 ]

Is it at all possible this issue is back in MariaDB Connector/C 3.0.5? We recently updated and have customers reporting this error being emitted:
stmt fetch (oversized) column failed: 2051:

Code that generated this error is here:
https://github.com/Monetra/mstdlib/blob/master/sql/mysql/mstdlib_sql_mysql.c#L856

Comment by Georg Richter [ 2018-07-19 ]

Brad,

I don't think this is related to the original issue (wrong length calculation). The CR_NO_DATA error (2051) is immediately returned at the beginning of the function mysql_stmt_fetch_column():

if (stmt->state < MYSQL_STMT_USER_FETCHING || column >= stmt->field_count ||
      stmt->state == MYSQL_STMT_FETCH_DONE)  {
    SET_CLIENT_STMT_ERROR(stmt, CR_NO_DATA, SQLSTATE_UNKNOWN, 0);
    return(1);
  }

It would be helpful to start a debug session and check which of the conditions above will fail.

Comment by Brad House [ 2018-07-19 ]

Unfortunately I don't have access to the customer's systems and I'm not able to reproduce myself. I'm testing on Percona XtraDB Cluster 5.7.22, but the customer is on the official Oracle Enterprise MySQL 5.6.40. Reverting back to the 3.0.3 release resolved the customer issue.

I can say we do reuse statement handles, so it could be related to CONC-344 possibly somehow.

If we can figure out how to reproduce, I'll open another ticket.

Comment by Mauro Leggieri [ 2018-12-03 ]

Hi, it is easy to repro with connector 3.0.7 and MySQL 5.7.22

Just fetch any table with a varchar binding with `buffer_length == 0`. Resize the buffer and update length after `mysql_stmt_fetch` and `mysql_stmt_fetch_column` will always returns `CR_NO_DATA`.

Regards,
Mauro.

Comment by Heo Yuna [ 2019-01-15 ]

It was fixed in connector 3.0.3, but at 3.0.4 or later (I've also checked versions 3.0.7 and 3.0.8), the issue reoccurs. As Mauro said, this issue can easily be reproduced.

Comment by Mauro Leggieri [ 2019-01-15 ]

As a workaround set MYSQL_REPORT_DATA_TRUNCATION option to zero in the connection options. Checked with source code.

Generated at Thu Feb 08 03:04:11 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.