Details
-
Bug
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
23.02.24
-
None
-
None
Description
Summary
Selecting from information_schema.COLUMNSTORE_TABLES or information_schema.COLUMNSTORE_PARTITIONS (and, on 23.02, COLUMNSTORE_COLUMNS) while ExeMgr/PrimProc is unreachable or unhealthy can kill the whole MariaDB server process with SIGABRT, instead of just failing that one query.
The fill functions for these views call CalpontSystemCatalog::getTables() without a try/catch. When the system catalog lookup fails, getTables() throws logging::IDBExcept. Nothing in the plugin catches it, so the exception crosses into the server, std::terminate() runs, and mariadbd aborts.
Customer impact
Seen at a customer site on 2026-07-14: a plain SELECT ... FROM information_schema.columnstore_columns aborted mariadbd while PrimProc was already unstable. The error code was MCS-2045 (ERR_PRIMPROC_DOWN). A second abort on 2026-07-23 with MCS-2033 (ERR_SYSTEM_CATALOG) fits the same path, but no backtrace survived, so that one is unconfirmed. Monitoring tools and DBAs commonly query these views exactly when the cluster is in trouble, so a backend problem gets turned into a full server outage.
Where it happens (tag 23.02.24, 4e11b0ed9)
Throw sites. getTables() (dbcon/execplan/calpontsystemcatalog.cpp:2740) calls getSysData() at line 2800. getSysData() (calpontsystemcatalog.cpp:736) throws IDBExcept(ERR_LOST_CONN_EXEMGR) at line 822 and IDBExcept(ERR_SYSTEM_CATALOG) at line 829. getSysData_FE() throws more IDBExcept variants at lines 945, 976, 998 and 1002.
Unguarded callers:
- dbcon/mysql/is_columnstore_columns.cpp:66: catalog_tables = csc.getTables(); in is_columnstore_columns_fill() (line 57)
- dbcon/mysql/is_columnstore_tables.cpp:60: catalog_tables = csc.getTables(); in is_columnstore_tables_fill() (line 44)
- dbcon/mysql/is_columnstore_partitions.cpp:66: catalog_tables = csc.getTables(); in is_columnstore_partitions_fill() (line 49)
Each of these functions already wraps its later per-table calls in try/catch (columnRIDs() at columns.cpp:87-91 and partitions.cpp:82-86, tableInfo() at tables.cpp:70-95). Only the first getTables() call is left unprotected.
State on stable-23.10 (ab3d31bd1). Commit 6e464bcd1 ("fix(crash, plugin)... catch missed exception", 2025-09-01) wrapped the call in is_columnstore_columns.cpp (lines 71-82 on stable-23.10). That commit is not in 23.02. The same call is still unguarded on stable-23.10 in is_columnstore_tables.cpp:60 and is_columnstore_partitions.cpp:66.
Secondary, lower confidence
Inside the per-column loops, csc.colName() / csc.colType() are also called outside any try/catch: is_columnstore_columns.cpp:105-106 and is_columnstore_partitions.cpp:102,109. If these can reach getSysData() on a cache miss, a backend failure partway through the query could abort the server the same way. We have not confirmed this path.
How to reproduce (expected)
- Columnstore cluster with at least one Columnstore table.
- Make ExeMgr/PrimProc unreachable from the SQL node, e.g. stop or SIGSTOP PrimProc.
- SELECT * FROM information_schema.COLUMNSTORE_TABLES;
- Expected: the query returns an error. Actual: mariadbd aborts (SIGABRT, std::terminate in the error log).
Possible resolution (up to the assigned engineer)
One option is to do what 6e464bcd1 did for is_columnstore_columns.cpp on stable-23.10: wrap the getTables() call in each fill function, catch IDBExcept and std::exception, and return 1 so the server reports a normal query error. That would mean backporting 6e464bcd1's is_columnstore_columns.cpp hunk to 23.02 and applying the same change to is_columnstore_tables.cpp and is_columnstore_partitions.cpp on both branches. It would also be worth deciding whether to guard the colName()/colType() calls too, or to put a single catch-all around the whole fill body. A friendlier error message, e.g. via push_warning, is another option. The engineer should pick whichever approach fits the plugin's conventions best.