The first 10.6 release we used was 10.6.5. After upgrading to 10.6.5, whenever we perform live VM migrations we sometimes see that InnoDB will at some point no longer respond. We also see this behaviour in 10.6.8.
We perform a live VM migration on XCP-ng (xenserver). We have done this for about ~300 VMs, and about ~40 had issues. We would see the issues ranging from about a few minutes after the migration up to ~2 weeks after the migration (usually on less busy servers it would manifest later).
Queries remain stuck in the states Updating, Sending Data, Statistics, Filling Schema table, Commit and just never complete.
Running the command "Show engine innodb status" will hang indefinitely and never give any output.
A restart of mysql also will not work. we have to kill -9 in order to restart mysql. After that it works again.
We have not noticed this on MariaDB 10.5 and 10.4.
We have noticed this on single instances as well as instances running galera.
I did make a gcore of one of the instances that has issues (it is almost 5GB). Perhaps I can do anything with that, but I'm not sure what.
Any ideas on what is wrong here?