Assertion `bytes_read' failed in merge_walk() when GROUP_CONCAT(DISTINCT) with an OFFSET past the end of the group is evaluated more than once

GROUP_CONCAT(DISTINCT x LIMIT n OFFSET m) builds its result in Item_func_group_concat::val_str() by walking the Unique object that filtered the duplicates out. When m is at least as large as the number of distinct rows in the group, every row is consumed by the offset, the aggregate returns an empty result, and the object is left looking as though it had never been evaluated. A second evaluation of the same group then walks the same Unique a second time. That is not allowed. Unique::walk() consumes the object, and Unique::reset() carries the contract in a comment above it: "You must call reset() if you want to reuse Unique after walk()", and "walk() does not reset any Unique member".

A debug build aborts on an assertion in merge_walk(). A build without assertions carries on into the merge with a chunk that holds no keys.

A second evaluation of the same group happens whenever the aggregate is both used in a HAVING clause and returned to the client, which is the shape of the repeat below. The crash also needs the duplicate filter to have spilled to disk, since a Unique that still holds everything in its tree is walked by a plain tree_walk() and a second walk of it is harmless.

How to repeat

The example uses the sequence engine for the data. Any table with more distinct values per group than the offset skips will do.

CREATE TABLE t1 (g INT, a VARCHAR(100));

INSERT INTO t1 SELECT seq MOD 4, LPAD(seq, 100, '0') FROM seq_1_to_2000;

SET @@tmp_memory_table_size=0;

SELECT g, GROUP_CONCAT(DISTINCT a LIMIT 5 OFFSET 1000) AS gc

  FROM t1 GROUP BY g HAVING gc <> 'x';

mariadbd: sql/uniques.cc:558: bool merge_walk(uchar*, size_t, uint, Merge_chunk*, Merge_chunk*, tree_walk_action, void*, qsort_cmp2, void*, IO_CACHE*, bool): Assertion `bytes_read' failed.

Server version: 13.1.0-MariaDB-debug-log

sql/uniques.cc:559(merge_walk(...))

sql/uniques.cc:697(Unique::walk(TABLE*, int (*)(void*, unsigned int, void*), void*))

sql/item_sum.cc:4737(Item_func_group_concat::val_str(String*))

sql/sql_type.cc:7729(Type_handler::Item_send_str(Item*, Protocol*, st_value*) const)

sql/sql_type.h:5813(Type_handler_string_result::Item_send(Item*, Protocol*, st_value*) const)

sql/item.h:1250(Item::send(Protocol*, st_value*))

sql/protocol.cc:1358(Protocol::send_result_set_row(List<Item>*))

sql/sql_class.cc:3411(select_send::send_data(List<Item>&))

sql/sql_select.cc:26803(end_send_group(JOIN*, st_join_table*, bool))

sql/sql_select.cc:25312(sub_select(JOIN*, st_join_table*, bool))

sql/sql_select.cc:24786(do_select(JOIN*, Procedure*))

The stack is abridged below do_select(). Note the frame above val_str(): the walk that aborts is the one performed while sending the row, after the HAVING clause has already evaluated the same aggregate for the same group.

Root cause

Whether the walk runs at all is decided by a single flag:

/* sql/item_sum.cc */

  if (!result_finalized) // Result yet to be written.

  {

    if (tree != NULL) // order by

      tree_walk(tree, &dump_leaf_key, this, left_root_right);

    else if (distinct) // distinct (and no order by).

      unique_filter->walk(table, &dump_leaf_key, this);

    else if (row_limit && copy_row_limit == (ulonglong)row_limit->val_int())

      return &result;

    else

      DBUG_ASSERT(false); // Can't happen

  }

result_finalized is written only inside dump_leaf_key(), the callback that the walk invokes for each row, and only on the paths that either append something to the result or stop because the row limit is exhausted. The path taken while the offset is being consumed sets nothing:

/* sql/item_sum.cc, dump_leaf_key() */

  ulonglong *offset_limit= &item->copy_offset_limit;

  ulonglong *row_limit = &item->copy_row_limit;

  if (item->limit_clause && !(*row_limit))

  {

    item->result_finalized= true;

    return 1;

  }

  tmp.length(0);

  if (item->limit_clause && (*offset_limit))

  {

    item->row_count++;

    (*offset_limit)--;

    return 0;

  }

  if (!item->result_finalized)

    item->result_finalized= true;

When the offset is not exhausted before the rows run out, every invocation returns through the middle branch. The walk therefore completes normally, the result stays empty, and result_finalized stays false, which is indistinguishable from a group that has not been evaluated yet.

The second walk is destructive because the first one emptied the tree. Unique::walk() returns early only while nothing has been spilled:

/* sql/uniques.cc */

  if (elements == 0)                       /* the whole tree is in memory */

    return tree_walk(&tree, action, walk_action_arg, left_root_right);

  sort.return_rows= elements+tree.elements_in_tree;

  /* flush current tree to the file to have some memory for merge buffer */

  if (flush())

    return 1;

flush() appends one chunk per call, with whatever the tree currently holds, and then deletes the tree:

/* sql/uniques.cc */

bool Unique::flush()

{

  Merge_chunk file_ptr;

  elements+= tree.elements_in_tree;

  file_ptr.set_rowcount(tree.elements_in_tree);

  file_ptr.set_file_position(my_b_tell(&file));

  ...

  if (tree_walk(&tree, action,

		(void*) this, left_root_right) ||

      insert_dynamic(&file_ptrs, (uchar*) &file_ptr))

    return 1;

  delete_tree(&tree, 0);

  return 0;

}

On the second walk the tree is empty, so the chunk appended holds no rows. merge_walk() reads that chunk and gets nothing back:

/* sql/uniques.cc, merge_walk() */

  for (top= begin; top != end; ++top)

  {

    top->set_buffer(merge_buffer + (top - begin) * piece_size,

                    merge_buffer + (top - begin) * piece_size + piece_size);

    top->set_max_keys(max_key_count_per_piece);

    bytes_read= read_to_buffer(file, top, &sort_param, false);

    if (unlikely(bytes_read == (ulong) -1))

      goto end;

    DBUG_ASSERT(bytes_read);

    queue_insert(&queue, (uchar *) top);

  }

The value zero is neither the error value that is checked for nor a valid read, so the assertion is the only thing standing between it and the merge loop, which goes on to take keys from the chunk.

Affected versions

MDEV-11563 (a006e88cac0, Varun Gupta, 2020-03-21) changed GROUP_CONCAT(DISTINCT) without ORDER BY to build its result from the duplicate filter in val_str(). The guard has looked the same since, and the block is byte identical on every branch checked, verified by reading each one:

• 10.11, sql/item_sum.cc:4491
• 11.4, sql/item_sum.cc:4491
• 11.8, sql/item_sum.cc:4480
• 12.0, sql/item_sum.cc:4476
• 12.3, sql/item_sum.cc:4495
• main, sql/item_sum.cc:4495

The symptom above was reproduced on a debug build of the current development branch only. Other branches were not run.

Note

Found while working on MDEV-21879. On main the reach is wider than on the release branches: since MDEV-21879 the DISTINCT with ORDER BY combination also builds its result by walking the duplicate filter, so the same query with an ORDER BY added aborts there as well, where on the release branches it takes the tree_walk() path and does not.