I wrote about this in January:

http://kristiannielsen.livejournal.com/17676.html
http://kristiannielsen.livejournal.com/18168.html

When profiling simple queries, we see that icache misses is a major
performance bottleneck. Most of the time in the CPU is spent waiting for new
instructions to be fetched and decoded.

The problem is partly just executing too many instructions during a simple
query (typical icache size is 32 kB), and partly too many ifs/branches
(jumping in the middle of an icache line wastes the rest of that line).

Looking at some random function like make_join_statistics(), it seems we have
lots and lots of small conditions that apply only in special cases or for more
complex queries. So most of these are not needed for a typical simple query;
yet every single one still cause execution of some instructions and possibly a
branch out of the cache line.

The idea is to split our code into a "fast" path and a "slow" path. The parser
will mark simple queries for the "fast" path, and remaining queries for the
"slow" path. If a query is marked "fast", we will know that a lot of the
special or complex cases do not apply, so we can skip them all with just a
single conditional that does not cause a branch in the "fast" case.

This way, we should be able to eliminate a lot of executed instructions for
the simple queries (while adding a few extra conditionals to the more complex
queries). If we can do this to the point that the code for eg. a simple
primary-key update fits completely in the icache, then we should see a large
speedup in execution time.