[MDEV-19270] function as Window function causes debug assert Created: 2019-04-17  Updated: 2019-05-13  Resolved: 2019-05-13

Status: Closed
Project: MariaDB Server
Component/s: Optimizer - Window functions
Affects Version/s: 10.3.14
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: David Hall (Inactive) Assignee: Varun Gupta (Inactive)
Resolution: Not a Bug Votes: 0
Labels: None
Environment:

CentOS 7


Issue Links:
PartOf
is part of MCOL-3307 Non-Columnstore Window function caus... Closed
Relates
relates to MDEV-19364 Server crashes on certain window func... Closed

 Description   

When using AVG as a Window function I get a debug assert at sql_window.cc:3054

Window_funcs_computation::setup()
 
  if (tab->filesort && tab->filesort->select)
  {
    sel= tab->filesort->select;
    DBUG_ASSERT(!sel->quick);
  }

To reproduce:

CREATE TABLE s3 (
  k1 bigint NOT NULL,
  c1 int DEFAULT NULL,
  PRIMARY KEY (k1)
);
 
SELECT k1, c1, avg(c1) OVER (ORDER BY c1) FROM s3 WHERE k1 < 100 ORDER BY k1;

If you leave off the final ORDER BY k1 or the WHERE clause, no assert. If k1 is not a key, no assert.



 Comments   
Comment by Alice Sherepa [ 2019-04-23 ]

I can not reproduce it, could you please add more information about how to repeat it (your .cnf file(s)).

MariaDB [test]> select version ();
+-----------------+
| version ()      |
+-----------------+
| 10.3.14-MariaDB |
+-----------------+
1 row in set (0.000 sec)
 
MariaDB [test]> CREATE TABLE s3 (k1 bigint NOT NULL, c1 int DEFAULT NULL, PRIMARY KEY (k1));
Query OK, 0 rows affected (0.032 sec)
 
MariaDB [test]> insert ignore into s3(k1) select floor(rand()*150)+90 from seq_1_to_100;
Query OK, 75 rows affected, 25 warnings (0.012 sec)
Records: 100  Duplicates: 25  Warnings: 25
 
MariaDB [test]> insert ignore into s3 select floor(rand()*150)+90,seq from seq_1_to_1000;
Query OK, 75 rows affected, 925 warnings (0.018 sec)
Records: 1000  Duplicates: 925  Warnings: 925
 
MariaDB [test]> SELECT k1,
    ->        c1,
    ->        avg(c1) OVER (ORDER BY c1)
    -> FROM s3
    -> WHERE k1 < 100
    -> ORDER BY k1;
+----+------+----------------------------+
| k1 | c1   | avg(c1) OVER (ORDER BY c1) |
+----+------+----------------------------+
| 90 |  154 |                   107.0000 |
| 91 | NULL |                       NULL |
| 92 |   27 |                    27.0000 |
| 93 |  140 |                    83.5000 |
| 94 | NULL |                       NULL |
| 95 | NULL |                       NULL |
| 96 | NULL |                       NULL |
| 97 | NULL |                       NULL |
| 98 | NULL |                       NULL |
| 99 | NULL |                       NULL |
+----+------+----------------------------+
10 rows in set (0.001 sec)
 
MariaDB [test]> CREATE or replace TABLE s3 (k1 bigint NOT NULL, c1 int DEFAULT NULL, PRIMARY KEY (k1));
Query OK, 0 rows affected (0.065 sec)
 
MariaDB [test]> SELECT k1,
    ->        c1,
    ->        avg(c1) OVER (ORDER BY c1)
    -> FROM s3
    -> WHERE k1 < 100
    -> ORDER BY k1;
Empty set (0.001 sec)

Comment by David Hall (Inactive) [ 2019-04-23 ]

This won't break if not a debug build. It just asserts and quits under debug.

Server version: 10.3.13-MariaDB-debug-log Columnstore 1.2.4-1

my.cnf:
[client]
port = 3306
socket = /usr/local/mariadb/columnstore/mysql/lib/mysql/mysql.sock

[mysqld]
loose-server_audit_syslog_info = columnstore-1
port = 3306
socket = /usr/local/mariadb/columnstore/mysql/lib/mysql/mysql.sock
datadir = /usr/local/mariadb/columnstore/mysql/db
skip-external-locking
key_buffer_size = 512M
max_allowed_packet = 1M
table_cache = 512
sort_buffer_size = 4M
read_buffer_size = 4M
read_rnd_buffer_size = 16M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size = 0

  1. Try number of CPU's*2 for thread_concurrency
    #thread_concurrency = 8
    thread_stack = 512K
    lower_case_table_names=1
    group_concat_max_len=512
    sql_mode="ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
  1. Enable compression by default on create, set to 0 to turn off
    infinidb_compression_type=2
  1. Default for string table threshhold
    infinidb_stringtable_threshold=20
  1. infinidb local query flag
    infinidb_local_query=0

infinidb_diskjoin_smallsidelimit=0
infinidb_diskjoin_largesidelimit=0
infinidb_diskjoin_bucketsize=100
infinidb_um_mem_limit=0

infinidb_use_import_for_batchinsert=1
infinidb_import_for_batchinsert_delimiter=7

basedir = /usr/local/mariadb/columnstore/mysql/
character-sets-dir = /usr/local/mariadb/columnstore/mysql/share/charsets/
lc-messages-dir = /usr/local/mariadb/columnstore/mysql/share/
plugin_dir = /usr/local/mariadb/columnstore/mysql/lib/plugin

binlog_format=ROW

server-id = 1

log-bin=/usr/local/mariadb/columnstore/mysql/db/mysql-bin
relay-log=/usr/local/mariadb/columnstore/mysql/db/relay-bin
relay-log-index = /usr/local/mariadb/columnstore/mysql/db/relay-bin.index
relay-log-info-file = /usr/local/mariadb/columnstore/mysql/db/relay-bin.info

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash

[isamchk]
key_buffer_size = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M

[myisamchk]
key_buffer_size = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

This may have some bearing on the issue:
Output of select @@optimizer switch
index_merge=off,
index_merge_union=off,
index_merge_sort_union=off,
index_merge_intersection=off,
index_merge_sort_intersection=off,
engine_condition_pushdown=off,
index_condition_pushdown=off,
derived_merge=off,
derived_with_keys=off,
firstmatch=off,
loosescan=off,
materialization=off,
in_to_exists=on,
semijoin=off,
partial_match_rowid_merge=off,
partial_match_table_scan=off,
subquery_cache=off,
mrr=off,
mrr_cost_based=off,
mrr_sort_keys=off,
outer_join_with_cache=off,
semijoin_with_cache=off,
join_cache_incremental=off,
join_cache_hashed=off,
join_cache_bka=off,
optimize_join_buffer_size=off,
table_elimination=off,
extended_keys=off,
exists_to_in=on,
orderby_uses_equalities=off,
condition_pushdown_for_derived=on,
split_materialized=off

Comment by Alice Sherepa [ 2019-04-24 ]

Could you please add error log, information on how exactly the server was built, which commit. Have you tried to reproduce on the recent version? Is that assertion reproducible for you with exactly that test case, that is submitted (empty table)?

Comment by David Hall (Inactive) [ 2019-04-24 ]

The assertion hits with empty file exactly as shown

Commit c8f9b3f915a729ec35c00e92cc534a01271aa6e6
This is the commit from which the latest columnstore fork was built

Startup .err log:
<pre>190424 10:16:02 Columnstore: Started; Version: 1.2.4-1
190424 10:16:02 Columnstore: Started; Version: 1.2.4-1
2019-04-24 10:16:02 0 [Note] InnoDB: Using Linux native AIO
2019-04-24 10:16:02 0 [Note] InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!!
2019-04-24 10:16:02 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-04-24 10:16:02 0 [Note] InnoDB: Uses event mutexes
2019-04-24 10:16:02 0 [Note] InnoDB: Compressed tables use zlib 1.2.7
2019-04-24 10:16:02 0 [Note] InnoDB: Number of pools: 1
2019-04-24 10:16:02 0 [Note] InnoDB: Using SSE2 crc32 instructions
2019-04-24 10:16:02 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2019-04-24 10:16:02 0 [Note] InnoDB: Completed initialization of buffer pool
2019-04-24 10:16:02 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-04-24 10:16:02 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
2019-04-24 10:16:02 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2019-04-24 10:16:02 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2019-04-24 10:16:02 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2019-04-24 10:16:02 0 [Note] InnoDB: Waiting for purge to start
2019-04-24 10:16:02 0 [Note] InnoDB: 10.3.13 started; log sequence number 12863068493; transaction id 24521157
2019-04-24 10:16:02 0 [Note] InnoDB: Loading buffer pool(s) from /usr/local/mariadb/columnstore/mysql/db/ib_buffer_pool
2019-04-24 10:16:02 0 [Note] Plugin 'FEEDBACK' is disabled.
2019-04-24 10:16:02 0 [Warning] /usr/local/mariadb/columnstore/mysql//bin/mysqld: unknown variable 'loose-server_audit_syslog_info=columnstore-1'
2019-04-24 10:16:02 0 [Note] Server socket created on IP: '::'.
2019-04-24 10:16:02 0 [Note] Reading of all Master_info entries succeded
2019-04-24 10:16:02 0 [Note] Added new Master_info '' to hash table
2019-04-24 10:16:02 0 [Note] /usr/local/mariadb/columnstore/mysql//bin/mysqld: ready for connections.
Version: '10.3.13-MariaDB-debug-log' socket: '/usr/local/mariadb/columnstore/mysql/lib/mysql/mysql.sock' port: 3306 Columnstore 1.2.4-1
2019-04-24 10:16:03 0 [Note] InnoDB: Buffer pool(s) load completed at 190424 10:16:03

</pre>

Comment by David Hall (Inactive) [ 2019-04-24 ]

MariaDB [dhall]> SELECT k1, c1, avg(c1) OVER (ORDER BY c1) FROM s3 WHERE k1 < 100 ORDER BY k1;
ERROR 2013 (HY000): Lost connection to MySQL server during query
MariaDB [dhall]>

mysqld: /home/calpont/mariadb-columnstore-server/sql/sql_window.cc:3054: bool Window_funcs_computation::setup(THD*, List<Item_window_func>, JOIN_TAB): Assertion `!sel->quick' failed.
190424 12:17:58 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

Server version: 10.3.13-MariaDB-debug-log
key_buffer_size=536870912
read_buffer_size=4194304
max_used_connections=1
max_threads=153
thread_count=8
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1781190 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x7f9270000b00
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f92e03b9d40 thread_stack 0x80000
/usr/local/mariadb/columnstore/mysql//bin/mysqld(my_print_stacktrace+0x40)[0x556f4e73b280]
/usr/local/mariadb/columnstore/mysql//bin/mysqld(handle_fatal_signal+0x3dc)[0x556f4df93069]
sigaction.c:0(__restore_rt)[0x7f9316a895d0]
:0(__GI_raise)[0x7f931472a207]
:0(__GI_abort)[0x7f931472b8f8]
:0(__assert_fail_base)[0x7f9314723026]
:0(_GI__assert_fail)[0x7f93147230d2]
sql/sql_window.cc:3058(Window_funcs_computation::setup(THD*, List<Item_window_func>, st_join_table))[0x556f4de96f0c]
sql/sql_select.cc:3340(JOIN::make_aggr_tables_info())[0x556f4dcdcfa8]
sql/sql_select.cc:2635(JOIN::optimize_stage2())[0x556f4dcda426]
sql/sql_select.cc:1934(JOIN::optimize_inner())[0x556f4dcd7eaf]
sql/sql_select.cc:1451(JOIN::optimize())[0x556f4dcd6363]
sql/sql_select.cc:4511(mysql_select(THD*, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*))[0x556f4dce09c3]
sql/sql_select.cc:385(handle_select(THD*, LEX*, select_result*, unsigned long))[0x556f4dcd1f1c]
sql/sql_parse.cc:4290(mysql_execute_command(THD*))[0x556f4dc8fbeb]
sql/sql_parse.cc:8142(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x556f4dc9c2f8]
sql/sql_parse.cc:10390(idb_parse_vtable(THD*, String&, THD::infinidb_state))[0x556f4dca18fe]
sql/sql_parse.cc:10955(idb_vtable_process(THD*, unsigned long long, Statement*))[0x556f4dca4176]
sql/sql_parse.cc:1871(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x556f4dc891b9]
sql/sql_parse.cc:1415(do_command(THD*))[0x556f4dc87b9a]
sql/sql_connect.cc:1403(do_handle_one_connection(CONNECT*))[0x556f4ddf6728]
sql/sql_connect.cc:1310(handle_one_connection)[0x556f4ddf647f]
pthread_create.c:0(start_thread)[0x7f9316a81dd5]
/lib64/libc.so.6(clone+0x6d)[0x7f93147f1ead]

Comment by Varun Gupta (Inactive) [ 2019-05-13 ]

Discussed this with David.Hall , the MariaDB server is not affected , there is some bug in Columnstore. So closing this issue.

Generated at Thu Feb 08 08:50:20 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.