Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-39234

DuckDB as a storage engine

    XMLWordPrintable

Details

    Description

      This PR introduces DuckDB 1.5.2 as an embedded analytical (OLAP) storage engine plugin for MariaDB 11.4. The engine is ported from AliSQL's DuckDB integration and fully adapted to MariaDB's handler API, plugin system, build infrastructure, and packaging conventions. It ships as a loadable module (ha_duckdb.so) for x86_64 and ARM64 architectures.
      Architecture

      The engine is structured as a part of MariaDB at storage/duckdb which itself contains a submodule (third_parties/duckdb) pointing to an upstream DuckDB 1.5.2. DuckDB is built statically with all builtin extensions (ICU, JSON) and linked into a single ha_duckdb.so plugin. A debug-STL ABI mismatch guard (-U_GLIBCXX_DEBUG -U_GLIBCXX_ASSERTIONS) is applied in debug builds to prevent SIGSEGV from sizeof(std::vector) divergence between the plugin and the server.
      Supported Operations

      DDL: CREATE TABLE, DROP TABLE, ALTER TABLE, RENAME TABLE — SQL is translated from MariaDB's internal structures to DuckDB-dialect DDL via ddl_convertor.cc, including expression defaults, column types, and engine conversions (ALTER TABLE ... ENGINE=DuckDB).
      DML — INSERT: supporting all major MariaDB column types.
      DML — SELECT: Full query pushdown to DuckDB via MariaDB's select_handler interface (ha_duckdb_pushdown.cc), with result-set conversion back to MariaDB row format (duckdb_select.cc).
      DML — UPDATE/DELETE: Direct UPDATE and DELETE translated to DuckDB SQL (dml_convertor.cc).
      Configuration: DuckDB-specific server variables — memory limit, thread count, operating mode, etc. (duckdb_config.cc).
      Timezone & Charset: Mapping between MariaDB and DuckDB timezone names and charset/collation conventions.
      UDFs: Registration of DuckDB-side user-defined functions accessible from MariaDB.
      Per-thread context: DuckdbThdContext managed via thd_get_ha_data/thd_set_ha_data (replacing AliSQL's THD::get_duckdb_context()).

      Packaging

      There is no separate package but the server package delivers ha_duckdb.so that must be explicitly loaded b/c of its maturity level.
      PLUGIN_DUCKDB=NO by default in native Debian builds (debian/rules); enabled automatically by autobake-deb.sh for MariaDB.org release builds when the submodule is present.
      RPM packaging scaffolding included in the submodule.

      Testing

      An MTR test suite is included under mysql-test/plugin/duckdb/ covering DDL, DML, type conversions, ALTER operations, and basic query pushdown scenarios. A disable.def file documents tests that are not yet passing or are intentionally skipped.

      Attachments

        Issue Links

          Activity

            People

              gkodinov Georgi Kodinov
              gkodinov Georgi Kodinov
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.