[MCOL-1182] Cross-engine join query failing (single node) Created: 2018-01-26 Updated: 2018-04-16 Resolved: 2018-04-16 |
|
| Status: | Closed |
| Project: | MariaDB ColumnStore |
| Component/s: | ExeMgr |
| Affects Version/s: | 1.0.12, 1.1.2 |
| Fix Version/s: | 1.1.4 |
| Type: | Bug | Priority: | Major |
| Reporter: | Andrew Simkovsky | Assignee: | Daniel Lee (Inactive) |
| Resolution: | Fixed | Votes: | 1 |
| Labels: | None | ||
| Environment: |
AWS server, m4.xlarge. Ubuntu 16.04 launched from AMI ami-2581aa40. |
||
| Attachments: |
|
| Sprint: | 2018-08 |
| Description |
|
Hello, I'm having trouble with cross-engine join queries. I have a single node with MariaDB column store on it. This one machine has both UM and PM. Followed the very simple install process here: https://mariadb.com/kb/en/library/preparing-for-columnstore-installation-11x/ The install is extremely vanilla. No configuration changes after install. I loaded some sample data. Lookup tables are innodb, but a large table with bulk data is "columnstore" engine. The intent is to query this columnstore table and join with an innodb lookup table to translate values. The tables in question are:
The keymap table has about 5000 rows. The study_response_1 table has about 7.7 million rows. If I query only the study_response_1 table by itself, it works just fine:
But if I join keymap and study_response_1, the ExeMgr process dies and forces restart. I get this error message:
ERROR 1815 (HY000): Internal error: Lost connection to ExeMgr. Please contact your administrator I've tried this on both 1.0.12 and 1.1.2, both fresh installs, and the same thing happens. I used mariadb-columnstore-1.0.12-1-xenial.x86_64.deb.tar.gz and mariadb-columnstore-1.1.2-1-xenial.x86_64.deb.tar.gz. The only difference between the versions is that version 1.0.12 required CrossEngineSupport to be manually set up in Columnstore.xml, but 1.1.2 already had it set up and working. That section of Columnstore.xml is here (default from 1.1.2, I didn't change anything):
If I convert the keymap table to ENGINE=columnstore, then the query works just fine. So its clearly only an issue when joining two tables with the different storage engines. I'm attaching tails of the various logs from /var/log/mariadb/columnstore. As soon as I execute the query, I see this in warning.log:
I then see this in debug.log:
All the rest of the log output seems to be related ExeMgr restarting itself. Is this a bug or a misconfiguration? Any known way to work around this other than converting all my tables to columnstore? I can provide sample data if needed for reproducing the problem. |
| Comments |
| Comment by Andrew Hutchings (Inactive) [ 2018-01-27 ] | ||||||||||||||||||
|
Unfortunately it looks like ExeMgr crashed when you executed that query. Can you please turn on core files and send us the core file generated? You can turn on core files by following these instructions: If you wish to upload using our FTP server the details are: | ||||||||||||||||||
| Comment by Andrew Simkovsky [ 2018-01-29 ] | ||||||||||||||||||
|
Thanks for the help. I've generated the core file and attached it here. File is named MCOL1182-core.ExeMgr.18965.gz | ||||||||||||||||||
| Comment by Andrew Hutchings (Inactive) [ 2018-01-29 ] | ||||||||||||||||||
|
Many thanks. We will analyse this as soon as possible. | ||||||||||||||||||
| Comment by Andrew Simkovsky [ 2018-02-01 ] | ||||||||||||||||||
|
Hello, any update? | ||||||||||||||||||
| Comment by Andrew Simkovsky [ 2018-02-01 ] | ||||||||||||||||||
|
Hello, thank you for confirming the bug. I see that its due to be fixed in 1.1.4. Do you know when that is targeted for release? | ||||||||||||||||||
| Comment by Andrew Hutchings (Inactive) [ 2018-02-01 ] | ||||||||||||||||||
|
Hi, | ||||||||||||||||||
| Comment by Andrew Hutchings (Inactive) [ 2018-02-01 ] | ||||||||||||||||||
|
We are releasing 1.1.3 first before we schedule 1.1.4. Unfortunately it is too late to fix this for 1.1.3. We typically do a 1.1 release every 1-2 months. | ||||||||||||||||||
| Comment by Andrew Simkovsky [ 2018-02-01 ] | ||||||||||||||||||
|
Ok, understood. Is there any chance that a fix would be backported to 1.1.2. Or is there a simple workaround that does not involve converting all tables to the columnstore engine? | ||||||||||||||||||
| Comment by Andrew Hutchings (Inactive) [ 2018-02-24 ] | ||||||||||||||||||
|
Sorry, completely missed that last comment until now. The last digit in out releases is an incremental bugfix version, we would not re-release a bugfix version. We instead release incremental bugfix releases. We may backport to 1.0 if there is a need for it. I'm not yet aware of a workaround beyond the one you mentioned. | ||||||||||||||||||
| Comment by Sergio Paternoster [ 2018-03-22 ] | ||||||||||||||||||
|
Could this be a possible workaround until fix? select k.value as make, count | ||||||||||||||||||
| Comment by Andrew Hutchings (Inactive) [ 2018-04-09 ] | ||||||||||||||||||
|
Issue appears to be doOuterJoinOnFilter() executes "delete c" on the filter when making it a nullTree (which doesn't appear to be needed) and the filter is then used for the cross engine join. | ||||||||||||||||||
| Comment by Andrew Hutchings (Inactive) [ 2018-04-09 ] | ||||||||||||||||||
|
For QA:
Will crash ExeMgr before patch, empty result after. | ||||||||||||||||||
| Comment by Andrew Hutchings (Inactive) [ 2018-04-09 ] | ||||||||||||||||||
|
Fix not perfect yet apparently... | ||||||||||||||||||
| Comment by Andrew Hutchings (Inactive) [ 2018-04-10 ] | ||||||||||||||||||
|
New patch is much more stable | ||||||||||||||||||
| Comment by Daniel Lee (Inactive) [ 2018-04-16 ] | ||||||||||||||||||
|
Build 1.1.4-1 source /root/columnstore/mariadb-columnstore-server Merge pull request #108 from mariadb-corporation/ /root/columnstore/mariadb-columnstore-server/mariadb-columnstore-engine Merge pull request #438 from mariadb-corporation/ diff --cc writeengine/splitter/we_splitterapp.cpp Reproduced the issue in 1.1.2-1 with only one try |