We were running a very large insert (500M rows) from an innodb table to a tokudb table.
The insert failed and the mariadb crashed. Which came 1st is still unknown at this point.
The tokudb table has 700 partitions with 4 indexes.
After the failures anything regarding TokuDB metadata was returning that error: No such file or directory. Whether we were looking at this table or another.
show table status...
Select * from TokuDB_fractal_tree_info;
Using sysdig, we found out that no mater what table you would be looking at, tokudb opens all tokudb related files. And in this case, the file it was looking for, existed but had a different name.
The file it was looking for was:
But the file was named:
We shutdown the database, and renamed the file, and all worked after that.
We tried many thing to get around the problem, one was to drop the table, which wasn't possible as it was giving us the same error.