Aria: a write error during OPTIMIZE/REPAIR TABLE (maria_repair_by_sort) makes maria_close() of the new data handle fail; the error path then uses and frees the already freed MARIA_HA (heap-use-after-free, double free, server abort)

OPTIMIZE TABLE (and REPAIR TABLE) on an Aria table runs maria_repair_by_sort(),
which writes the compacted rows to the temporary data file <table>.TMD through
a second handle, sort_info.new_info. If one pwrite() to the .TMD fails in the
middle of the repair (not the first one; see the last section), the repair
goes on, and the error is only returned when that handle is closed (logged with
--log-warnings=3):

[ERROR] test.t2: Got error 5 on close

maria_close() frees the MARIA_HA even when it returns an error. The caller
jumps to its error label with sort_info.new_info still pointing at the freed
handle, calls end_io_cache() on it (heap-use-after-free) and then
maria_close() on it a second time (double free). On release builds glibc
aborts the process ("free(): double free detected in tcache 2" or "double
free or corruption (out)"), so every connection is dropped.

After the restart the table is marked crashed ("Last repair was aborted before
finishing", CHECK TABLE: "Found 627 keys of 2011 / Corrupt"). With the default
aria_recover_options=BACKUP,QUICK, the first access repairs it automatically,
and all 2011 rows come back, so in my runs no data was lost. The problem is
that one transient write error in a maintenance statement takes the whole
server down, and the heap gets corrupted on the way.

How to repeat:

1. The fault injector. It fails the Nth pwrite()/pwrite64() to a file whose
path ends in $FAIL_SUFFIX once, with EIO:

cat > failpwrite.c <<'EOF'
#define _GNU_SOURCE
#include <dlfcn.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
static ssize_t (*real_pwrite)(int, const void *, size_t, off_t);
static ssize_t (*real_pwrite64)(int, const void *, size_t, off_t);
static long counter;
static int fire(int fd, size_t len, off_t off) {
char p[64], t[4096]; const char *suf = getenv("FAIL_SUFFIX"), *done = getenv("FAIL_DONE_FILE");
snprintf(p, sizeof p, "/proc/self/fd/%d", fd);
ssize_t n = readlink(p, t, sizeof t - 1); if (n <= 0 || !suf) return 0; t[n] = 0;
size_t ls = strlen(suf); if ((size_t)n < ls || strcmp(t + n - ls, suf)) return 0;
if (__atomic_add_fetch(&counter, 1, __ATOMIC_SEQ_CST) != atol(getenv("FAIL_WHEN"))) return 0;
if (done)

{ if (access(done, F_OK) == 0) return 0; FILE *f = fopen(done, "w"); if (f) fclose(f); }

fprintf(stderr, "failpwrite: EIO on pwrite fd=%d len=%zu off=%lld\n", fd, len, (long long)off);
return 1;
}
ssize_t pwrite(int fd, const void *b, size_t n, off_t o) {
if (!real_pwrite) real_pwrite = dlsym(RTLD_NEXT, "pwrite");
if (fire(fd, n, o))

{ errno = EIO; return -1; } return real_pwrite(fd, b, n, o); }
ssize_t pwrite64(int fd, const void *b, size_t n, off_t o) {
if (!real_pwrite64) real_pwrite64 = dlsym(RTLD_NEXT, "pwrite64");
if (fire(fd, n, o)) { errno = EIO; return -1; }

return real_pwrite64(fd, b, n, o); }
EOF
gcc -O2 -shared -fPIC -o failpwrite.so failpwrite.c -ldl

2. A server that fails the 73rd write to t2.TMD (FAIL_DONE_FILE makes it fire
only once, so the restarted server runs clean):

docker run -d --name aria014 -v $PWD/failpwrite.so:/failpwrite.so:ro \
-e LD_PRELOAD=/failpwrite.so -e FAIL_SUFFIX=t2.TMD -e FAIL_WHEN=73 \
-e FAIL_DONE_FILE=/var/lib/mysql/failpwrite.done \
-e MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1 -e MARIADB_DATABASE=test \
quay.io/mariadb-foundation/mariadb-devel:verylatest

1. (wait until the server is ready)
   q() { docker exec aria014 mariadb -uroot test -e "$1"; }

   q "CREATE TABLE t2 (id BIGINT PRIMARY KEY, k BIGINT, n BIGINT, v TEXT) ENGINE=Aria TRANSACTIONAL=1 ROW_FORMAT=PAGE;
   INSERT INTO t2 SELECT seq, seq, seq, RPAD(MD5(seq), 900, MD5(seq+1)) FROM seq_1_to_2016;
   DELETE FROM t2 WHERE id IN (954, 987, 1523, 1780, 1849);"
   q "OPTIMIZE TABLE t2" # REPAIR TABLE t2 gives the same result
   docker logs aria014 2>&1 | tail -60
   docker start aria014; q "CHECK TABLE t2"; q "SELECT COUNT FROM t2"

Result (identical on 13.2 main and 13.0.2; also with FAIL_WHEN=200, and with
REPAIR TABLE instead of OPTIMIZE TABLE):

OPTIMIZE TABLE t2 -> the server aborts; the docker exec client ends with exit code 137 (a client
connected over TCP gets ERROR 2013 Lost connection to server during query)
error log:
failpwrite: EIO on pwrite fd=53 len=8192 off=581632
[ERROR] test.t2: Got error 5 on close (logged only with --log-warnings=3)
free(): double free detected in tcache 2
[ERROR] mariadbd got signal 6 ;
mariadbd(_ZN8ha_maria6repairEP3THDP22st_handler_check_paramb+0xe93)
mariadbd(_ZN8ha_maria8optimizeEP3THDP15st_ha_check_opt+0xae)
mariadbd(_ZN22Sql_cmd_optimize_table7executeEP3THD+0xfa)
after the restart:
Table './test/t2' is crashed, skipping it. Please repair it with aria_chk -r
CHECK TABLE t2 -> warning Last repair was aborted before finishing
error Found 627 keys of 2011
error Corrupt
SELECT COUNT FROM t2 -> 2011 (automatic repair, aria_recover_options=BACKUP,QUICK)

The same fault in the ASAN build of main (server started directly with
LD_PRELOAD="libasan.so.6 failpwrite.so", ASAN_OPTIONS=fast_unwind_on_malloc=0):

==340889==ERROR: AddressSanitizer: heap-use-after-free on address 0x529000172800
READ of size 8 at 0x529000172800 thread T11
#0 end_io_cache mysys/mf_iocache.c:1871
#1 maria_repair_by_sort storage/maria/ma_check.c:4238
#2 ha_maria::repair(THD*, st_handler_check_param*, bool) storage/maria/ha_maria.cc:1729
#3 ha_maria::optimize(THD*, st_ha_check_opt*) storage/maria/ha_maria.cc:1596
#4 handler::ha_optimize(THD*, st_ha_check_opt*) sql/handler.cc:5974
#5 mysql_admin_table sql/sql_admin.cc:949
freed by thread T11 here:
#0 __interceptor_free
#1 my_free mysys/my_malloc.c:218
#2 maria_close storage/maria/ma_close.c:269
#3 maria_repair_by_sort storage/maria/ma_check.c:4119
#4 ha_maria::repair(THD*, st_handler_check_param*, bool) storage/maria/ha_maria.cc:1729
previously allocated by thread T11 here:
#0 __interceptor_malloc
#1 my_malloc mysys/my_malloc.c:93
#2 my_multi_malloc mysys/mulalloc.c:59
#3 maria_clone_internal storage/maria/ma_open.c:112
#4 maria_open storage/maria/ma_open.c:1196
#5 create_new_data_handle storage/maria/ma_check.c:6689
#6 maria_repair_by_sort storage/maria/ma_check.c:3869
SUMMARY: AddressSanitizer: heap-use-after-free mysys/mf_iocache.c:1871 in end_io_cache

Expected: OPTIMIZE/REPAIR TABLE returns an error (and leaves the table marked
crashed, as it does when the first write fails, see below); the server keeps
running.

Cause, as far as I can tell (line numbers at 327437ade1d6):

maria_repair_by_sort(), after the rows have been written:

https://github.com/MariaDB/server/blob/327437ade1d6dc6e4e57a6f1723b4abe4ac6c4b4/storage/maria/ma_check.c#L4116-L4123

if (sort_info.new_info != sort_info.info)
{
MARIA_STATE_INFO save_state= sort_info.new_info->s->state;
if (maria_close(sort_info.new_info))

{ _ma_check_print_error(param, "Got error %d on close", my_errno); goto err; }

maria_close() frees the handle unconditionally and only then returns the
error:

https://github.com/MariaDB/server/blob/327437ade1d6dc6e4e57a6f1723b4abe4ac6c4b4/storage/maria/ma_close.c#L269-L275

The error label still uses sort_info.new_info:

https://github.com/MariaDB/server/blob/327437ade1d6dc6e4e57a6f1723b4abe4ac6c4b4/storage/maria/ma_check.c#L4236-L4240
end_io_cache(&sort_info.new_info->rec_cache); <- use after free (ASAN)
https://github.com/MariaDB/server/blob/327437ade1d6dc6e4e57a6f1723b4abe4ac6c4b4/storage/maria/ma_check.c#L4248-L4252
unuse_data_file_descriptor(sort_info.new_info);
maria_close(sort_info.new_info); <- second free (glibc abort)

new_file is also still >= 0 at that point although maria_close() has already
closed the data file of new_info, so the error path probably closes that file
descriptor a second time (L4253-L4257); I have not verified that separately.

maria_repair() has the same pattern (maria_close() failure at L2938-L2942,
then end_io_cache() at L2994 and maria_close() at L3011 on the same pointer):
https://github.com/MariaDB/server/blob/327437ade1d6dc6e4e57a6f1723b4abe4ac6c4b4/storage/maria/ma_check.c#L2935-L2942

A possible fix: when maria_close(sort_info.new_info) fails, set
sort_info.new_info= info (or NULL) and new_file= -1 before "goto err", in both
functions.

Variant: the first write to the .TMD fails

If the first pwrite() to t2.TMD fails instead, the server does not crash:
OPTIMIZE TABLE reports "Got error 5 when trying to repair table" / "Error
writing file './test/t2.TMD' (Errcode: 5)" / "Operation failed", every later
statement on t2 fails with error 144 "Table is crashed and last repair
failed", and the table is left marked crashed (CHECK TABLE: "Found 2016 keys
of 2011" with the script above and FAIL_WHEN=1)
because maria_repair_by_sort() has already dropped the indexes in place and
calls maria_mark_crashed_on_repair() on error. The automatic repair does not
touch it because of the "last repair failed" flag; REPAIR TABLE from SQL
restores it with all rows. That is the documented MyISAM-style behaviour and
is mentioned here only to show that a failed write is otherwise handled
without a crash; the crash needs the error to surface at maria_close().

Related: MDEV-23824 (SIGSEGV in end_io_cache in maria_repair with a NULL
handle after an out-of-memory error; fixed in 10.3.26) is a different error
path. I did not find a report of this one.