DuckDB: parse forwarded MariaDB SQL using a custom PEG frontend

Background

The DuckDB storage engine currently forwards the original MariaDB query text from THD::query() to DuckDB. The pushdown layer applies a growing set of lexer-aware and textual rewrites for MariaDB-specific syntax before DuckDB parses the query with its standard SQL parser.

This approach does not preserve MariaDB token semantics in all cases. For example, with ANSI_QUOTES disabled, MariaDB treats a double-quoted token as a string literal:

SELECT id, product, amount

FROM invoice

WHERE product = "MariaDB mug";

The query text reaches DuckDB unchanged, where "MariaDB mug" is interpreted as an identifier and binding fails with Referenced column "MariaDB mug" not found.

Other compatibility cases currently require individual rewrites or remain unsupported, including MariaDB quoting and escaping rules, executable comments, LIMIT offset,count, REGEXP/RLIKE, GROUP BY ... WITH ROLLUP, index hints, and MariaDB-specific function syntax.

Goal

Implement a MariaDB SQL frontend using the PEG parser infrastructure in DuckDB. MariaDB itself will not use or embed the PEG parser and its parser architecture must remain unchanged.

The new frontend should accept the original MariaDB query text, tokenize and parse it according to MariaDB syntax and session semantics, transform it into DuckDB SQLStatement AST nodes, and pass those statements to the existing DuckDB binder, planner, and executor.

THD::query()

  -> MariaDB-compatible tokenizer

  -> MariaDB-compatible lexer

  -> MariaDB PEG grammar

  -> DuckDB SQLStatement AST

  -> DuckDB binder/planner/executor

Requirements

• Implement a custom tokenizer, lexer, and grammar for MariaDB SQL using DuckDB's PEG parser infrastructure.
• Produce native DuckDB SQLStatement and expression nodes; do not serialize the parsed statement back to SQL for normal execution.
• Pass relevant MariaDB session context from THD into the frontend, including at least:
  • ANSI_QUOTES;
  • NO_BACKSLASH_ESCAPES;
  • MariaDB server version where it affects executable comments.
• Preserve MariaDB semantics for single-quoted strings, double-quoted strings or identifiers, backtick-quoted identifiers, embedded quote escaping, comments, and charset introducers.
• Parse the MariaDB syntax currently handled by textual rewrites in ha_duckdb_pushdown.cc, including LIMIT offset,count, REGEXP/RLIKE, GROUP BY ... WITH ROLLUP, STRAIGHT_JOIN, and index hints.
• Apply the MariaDB PEG frontend only to SQL forwarded from MariaDB. SQL generated internally by the DuckDB storage engine must continue to use the standard DuckDB parser.
• Keep the implementation compatible with upstream DuckDB and avoid source patches to the DuckDB binder or parser where an extension or engine-side integration point is sufficient.
• Return controlled compatibility or fallback errors for syntactically valid MariaDB constructs that cannot yet be represented or executed by DuckDB.

Acceptance criteria

1. With ANSI_QUOTES disabled, WHERE product = "MariaDB mug" is represented as a DuckDB string constant and returns the same result as a MariaDB table.
2. With ANSI_QUOTES enabled, a double-quoted token is represented as a quoted identifier.
3. Backtick-quoted MariaDB identifiers are represented as DuckDB identifiers without textual replacement.
4. Quoting, escaping, comments, and statement boundaries are covered by automated tests for both NO_BACKSLASH_ESCAPES modes.
5. Existing pushdown tests continue to pass through the PEG frontend.
6. Tests cover each MariaDB construct that replaces an existing textual rewrite.
7. Internally generated DuckDB SQL remains parsed by the standard DuckDB frontend.
8. The migrated textual rewrites are removed after equivalent PEG parsing and AST transformation are covered by tests.

Non-goals

• Replacing the MariaDB server parser with PEG.
• Changing MariaDB's parser architecture.
• Teaching the standard DuckDB SQL dialect to interpret double quotes as strings.
• Adding binder heuristics that reinterpret unresolved quoted identifiers as string literals.