Rationale and traditional approach to JIT-ting queries

In the current implementation of the DML statements (like update_single_table, sql_update.cc) the row-processing loop contains a large number of conditional checks (triggers, constraints, etc.), which leads to frequent branch mispredictions and wasted CPU cycles.

We need a mechanism that, during the row-processing planning phase, builds a simplified intermediate representation (IR) of the "hot" portion of the loop dead branches eliminated and small functions inlined—and then, on first execution, compiles that IR to native code (JIT), replacing the generic, heavy-loaded loop with a specialized function.

Traditional approach to JIT, used e.g. by PostgreSQL, rewrites the expression tree into a single node. Usually, this doesn't affect the main row-loop, unless it is generated as an "execution plan" itself, which is a list of virtual "Executable" objects picked to call, which may be harder to understand and to debug.

Our approach

Instead, we will try to preserve the more "declarative" way of code's presentation (basically, as it is now), and delegate the optimization to modern (at the same time, low-latency!) code generators.

Let's extract the entire row loop (i.e. update_single_table(), former mysql_update()) into a standalone C/C++ module with a signature like:

int mysql_update_inner_loop(bool has_before_triggers, bool has_after_triggers, 

                            bool has_generated_columns, bool binlog_enabled, ...)

{

  while (!(error=info.read_record()) && !thd->killed)

  {

    if (!has_before_triggers) invoke_before_triggers(...);

    ...

    if (has_periods) {/* do relevant uperations */}

    ... further update row

  }

}

At build-time, we will pre-translate it into a JIT-compiler's intermediate representation.

A specialized "invoker" function is then generated at runtime with all relevant parameters statically set to known constants (e.g. all false on simple tables):

int mysql_update_inner_loop_execute()

{

  return mysql_update_inner_loop(false, false, false, ....);

}

Compiling this together allows full inlining and dead code elimination (DCE) to produce minimal, branch-free code.

Our choice to JIT-compilation system.

The crux of jit-compilation is runtime latency. We will utilize a few ideas to reduce it to, supposedly, insignificance.

First, all the C/C++ code will be pre-translated to binary IR as part of CMake script at build time.
The invocation part (in our example, mysql_update_inner_loop_execute) can be generated directly into binary IR using the JIT-compilers' API.

Second, llvm, though have become an industrial default choice, is known for lengthy executions, even in the most stripped configs.

This is why we have chosen MIR as our initial JIT backend. MIR [github] can compile C to IR and has a much lighter footprint than LLVM. It also uses a MIT licanse, which is compatible with GPLv2.

The downside is that this row-processing code will have to be written in C, since MIR has no C++ support, nor there is a viable way to translate C++ to C.

Upper-level overview

The system will be designed to allow any JIT compiler to be plugged in in future.

At runtime, the engine will work directly with MIR IR and perform JIT compilation, taking advantage of dead-code elimination (DCE) to strip out unnecessary branches.

Overall, the design allows to easily opt-out jit approach at compile-time and follow the usual approach, by compiling the row-processing path in a usual way. Optionally, a runtime variable can be implemented (e.g. optimize_jit_enable).