The problem is that we try to run the query as Columnstore, even though there's no columnstore table involved. During this process, we execute:

// @InfiniDB We don't need tmp table for vtable create phase. Plus
// to build tmp table may corrupt some field table_name & db_name (for some reason)
if (thd->infinidb_vtable.vtable_state == THD::INFINIDB_CREATE_VTABLE)
need_tmp = false;

in the optimizer. This has the effect of setting things up such that the assert happens during this Columnstore attempt.

Not sure if the best thing to do is remove the assert (things appear to work without it) or something else.

After some gdb poking, the best thing to do is not do the assert during the initial Columnstore attempt.

Build tested: GitHub source, tag=columnstore-1.1.7

Created a build with debug flag and reproduced the issue

[root@localhost ~]# mcsmysql mytest
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 12
Server version: 10.2.22-MariaDB-debug-log Columnstore 1.1.7-1

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [mytest]> CREATE TABLE s3 (
-> k1 bigint NOT NULL,
-> c1 int DEFAULT NULL,
-> PRIMARY KEY (k1)
-> );
Query OK, 0 rows affected (0.01 sec)

MariaDB [mytest]> SELECT k1, c1, avg(c1) OVER (ORDER BY c1) FROM s3 WHERE k1 < 100 ORDER BY k1;
ERROR 2013 (HY000): Lost connection to MySQL server during query

Build verified: Github 1.1.8-1 source

/root/columnstore/mariadb-columnstore-server
commit 09faec83167fbecf3693b703ea287058364e4b2c
Merge: 01cc1ef 890088e
Author: Roman Nozdrin <drrtuy@gmail.com>
Date: Thu May 16 20:13:41 2019 +0300

Merge pull request #159 from mariadb-corporation/MCOL-3307

MCOL-3307 don't assert during Columnstore processing

/root/columnstore/mariadb-columnstore-server/mariadb-columnstore-engine
commit 46283467468fef60996931096e5da9d2cfd27e28
Merge: 0af6994 01ff860
Author: Andrew Hutchings <andrew@linuxjedi.co.uk>
Date: Fri May 17 20:34:26 2019 +0100

Merge pull request #766 from mariadb-corporation/MCOL-3314-1.1

MCOL-3314 When setting up statistics functions for the UM, use values…

Made build from the latest develop-1.1 source with debug flag and verified that assertion no longer occurs.

[root@localhost ~]# mcsmysql mytest
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 12
Server version: 10.2.22-MariaDB-debug-log Columnstore 1.1.8-1

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [mytest]> CREATE TABLE s3 (
-> k1 bigint NOT NULL,
-> c1 int DEFAULT NULL,
-> PRIMARY KEY (k1)
-> );
Query OK, 0 rows affected (0.04 sec)

MariaDB [mytest]> SELECT k1, c1, avg(c1) OVER (ORDER BY c1) FROM s3 WHERE k1 < 100 ORDER BY k1;
Empty set (0.01 sec)

MariaDB [mytest]> quit

Still waiting for 1.2.4-1 source to merge up.

Build verified: 1.2.4-1 GitHub source

/root/columnstore/mariadb-columnstore-server
commit 05f10f4541177e565963d32ed3de1f71fad7b837
Merge: edd358d 43d3662
Author: Roman Nozdrin <drrtuy@gmail.com>
Date: Mon May 20 22:11:32 2019 +0300

Merge pull request #161 from mariadb-corporation/MCOL-3318

Fix warning during RPM install

/root/columnstore/mariadb-columnstore-server/mariadb-columnstore-engine
commit 122038e36a8a4d4bd632eb137bde31a09a88d2b9
Merge: 8afc3f8 ea2ff9c
Author: Roman Nozdrin <drrtuy@gmail.com>
Date: Mon May 20 13:50:36 2019 +0300

Merge pull request #767 from mariadb-corporation/develop-1.2-merge-up-20190517

Merge develop-1.1 into develop-1.2