I've built MariaDB server without-aria-tmp-tables (but still with Aria itself):

CONF_COMMAND = ./configure '-prefix=/usr/local/mysql' 'enable-assembler' 'with-extra-charsets=complex' 'enable-thread-safe-client' 'with-big-tables' 'with-plugin-aria' 'without-aria-tmp-tables' 'without-plugin-innodb_plugin' 'with-mysqld-ldflags=-static' 'with-client-ldflags=-static' 'with-readline' 'with-ssl' 'with-plugins=max' 'with-embedded-server' 'with-libevent' '-enable-local-infile' 'CC=ccache gcc' 'CFLAGS=-Wall -Wextra -Wunused -Wwrite-strings -Wno-uninitialized -mtune=native -O3 -fno-omit-frame-pointer -g ' 'CXX=ccache g++' 'CXXFLAGS=-Wall -Wextra -Wunused -Wwrite-strings -Wno-uninitialized -Wno-unused-parameter -Wnon-virtual-dtor -felide-constructors -fno-exceptions -fno-rtti -mtune=native -O3 -fno-omit-frame-pointer -g '

When I start server with Aria and check the variables, it says, as expected,
....

....

If I try to start it with skip-aria, it starts, also as expected, saying in the error log

111220 21:56:25 [Note] Plugin 'Aria' is disabled.

but upon a connection it crashes:

#3 0x082735dd in handle_segfault (sig=11) at mysqld.cc:2838
#4 <signal handler called>
#5 0x0825e616 in thd_ha_data (thd=0x91c9608, hton=0x0) at sql_class.cc:347
#6 0x084c2e64 in ha_maria::implicit_commit (thd=0x91c9608, new_trn=false) at ha_maria.cc:2824
#7 0x0828e79a in dispatch_command (command=COM_QUERY, thd=0x91c9608, packet=0x9216599 "", packet_length=28)
at sql_parse.cc:1611
#8 0x0828ff3b in do_command (thd=0x91c9608) at sql_parse.cc:922
#9 0x0827e848 in handle_one_connection (arg=0x91c9608) at sql_connect.cc:1193
#10 0x400c2b25 in start_thread () from /lib/libpthread.so.0

the corresponding lines at sql_parse.cc:

#ifdef WITH_ARIA_STORAGE_ENGINE
ha_maria::implicit_commit(thd, FALSE);
#endif

Reproducible on 5.2 and 5.3, did not check 5.1.

To reproduce it, it's enough to build as described and run
mtr --skip-aria 1st
(or just try to connect to the server using the client).