[MDEV-9347] Not all rows returned by the C API Created: 2015-12-30  Updated: 2016-01-27  Resolved: 2016-01-27

Status: Closed
Project: MariaDB Server
Component/s: Optimizer, Storage Engine - Aria
Affects Version/s: 10.1
Fix Version/s: 10.1.11

Type: Bug Priority: Major
Reporter: Tim Van den Eynde Assignee: Michael Widenius
Resolution: Fixed Votes: 0
Labels: None
Environment:

Linux x64

Attachments: File mariadb-bug.tar.gz    
Sprint: 10.1.11

 Description   

We noticed after upgrading from MySQL 5.5 to MariaDB 10.1 that looping over a table through the C API doesn't return all rows. It only happens on a sufficiently large table.

Attached is a small Python script which produces such a table, and a C program that triggers the bug. Running make will compile the C file and produce a file setup.sql which you can import in your database. When it is imported, MYSQLPASSWORD= ./main -s (other flags might be needed, see ./main -h) shows the number of rows received, which should of course agree with SELECT COUNT(*) FROM foo; from the mysql client.

We have bisected the issue to this commit: https://github.com/MariaDB/server/commit/7a387c01586ae5107f2c2c64d3cdd420de83e168

 Comments   
Comment by Sergei Golubchik [ 2016-01-24 ]

What happens here:

  • a large table (1*10 6 rows)
  • main.c prepares a simple SELECT columns FROM table
  • and executes it with the CURSOR_TYPE_READ_ONLY flag
  • on the server this goes to mysql_open_cursor() that creates a Select_materialize object
  • Select_materialize::send_result_set_metadata() invokes create_result_table() with keep_row_order=TRUE
  • before my commit (referenced above) keep_row_order caused Aria tables to be created with the DYNAMIC record format
  • with the google encryption patch Aria can preserve the row order in the BLOCK format too
  • so now all temporary tables use BLOCK format to avoid leaking unencrypted data to disk (and BLOCK is faster anyway)
  • in the debug build that query hits an assert in allocate_head() function in ma_bitmap.c: 

      if (insert_order)
    		 
      {
    		 
        uint last_insert_page= share->last_insert_page;
    		 
        uint byte= 6 * (last_insert_page / 16);
    		 
        first_pattern= last_insert_page % 16;
    		 
        DBUG_ASSERT(data + byte < end);
    		 
        data+= byte;
    		 
      }

Comment by Michael Widenius [ 2016-01-27 ]

roblem was that insert-order (enforced by the optimizer) did not handle the case where the bitmap changed to a new one.

Fixed by remembering the last bitmap page used and to force usage of this when inserting new rows

Comment by Michael Widenius [ 2016-01-27 ]

Fix pushed

Generated at Thu Feb 08 07:33:59 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.