[MDEV-13689] [Draft] 10.1.10 : InnoDB: Failing assertion: addr.page == FIL_NULL || addr.boffset >= FIL_PAGE_DATA Created: 2017-08-31 Updated: 2023-12-19 Resolved: 2021-04-26 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Storage Engine - InnoDB |
| Affects Version/s: | 10.1.10, 10.2.7 |
| Fix Version/s: | N/A |
| Type: | Bug | Priority: | Minor |
| Reporter: | Elena Stepanova | Assignee: | Marko Mäkelä |
| Resolution: | Cannot Reproduce | Votes: | 3 |
| Labels: | None | ||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Description |
|
IMPORTANT NOTE: This was observed on 10.1.10, while the current latest release is 10.1.26. Unless it also happens on recent versions, there is nothing to be done here – many things have changed since 10.1.10, so the problem might be long gone. However, I want to have it filed, to make it searchable in case it's ever encountered again, in an old or a new version.
Preceding errors in the log:
|
| Comments |
| Comment by TAO ZHOU [ 2017-10-23 ] | |||||||||||||||||||||||||||||||||||
|
I am running 10.2.7 and now getting this same error. | |||||||||||||||||||||||||||||||||||
| Comment by Johnny Oskarsson [ 2020-02-09 ] | |||||||||||||||||||||||||||||||||||
|
I just got this in 10.3.22 as well. Also, how on earth is this a "Minor" bug? It crashes on startup for me, making MariaDB and all that depends on it (in my case: Nextcloud) completely unusable! Any ideas on how to recover from this? I tried the InnoDB recovery modes without success. | |||||||||||||||||||||||||||||||||||
| Comment by Elena Stepanova [ 2020-02-09 ] | |||||||||||||||||||||||||||||||||||
|
joskar, it was "Minor" on the reason explained at the very beginning of the description: it was only observed on very old versions and there was no indication it still existed. If you encountered it on 10.3.22, it will make a difference. Please attach the error log, stack traces from the coredump if you have it, and server config file(s). Thanks. | |||||||||||||||||||||||||||||||||||
| Comment by Elena Stepanova [ 2020-02-09 ] | |||||||||||||||||||||||||||||||||||
|
marko, do you have any advice on "how to recover from it" part? | |||||||||||||||||||||||||||||||||||
| Comment by Johnny Oskarsson [ 2020-02-09 ] | |||||||||||||||||||||||||||||||||||
|
I managed to recover from it just now, I finally got mysqld --innodb-force-recovery=5 working. I did a mysqldump --all-tables >mysql.backup, killed mysqld, removed everything in /var/lib/mysql, ran mysql_install_db and started mysqld anew and then did a mysql -u root <mysql.backup. Now it seems to work again, but I don't know if this was the best way to go about it. But I was a bit worried when I did not get any recovery working, but this was mostly because I interpreted the "server system variable" in the recovery documentation as an environment variable. Attached is an error log (of "one crash cycle"). The config files in /etc/mysql are the default Debian 10 configs, which are pretty much empty. Previously today I did some changes to the Nextcloud database (I was Enabling MySQL 4-byte support), in case that could be related. Sorry about the tone in my previous post, my frustration levels has been a bit too high today. | |||||||||||||||||||||||||||||||||||
| Comment by Elena Stepanova [ 2020-02-09 ] | |||||||||||||||||||||||||||||||||||
|
joskar, thanks. Yes, such problems are frustrating for us too, as there isn't much we can do when it happens once in two years and cannot be reproduced. Maybe marko will come up with some idea, though. Meanwhile, do you happen to still have the error log from the very first such crash and, importantly, the previous server session before it? It might be important to see what happened there before the problems started, maybe there were errors or warnings (or even notes) which will shed some light on it. | |||||||||||||||||||||||||||||||||||
| Comment by Johnny Oskarsson [ 2020-02-12 ] | |||||||||||||||||||||||||||||||||||
|
elenst, here are the logs I had on that day until the first assertion, which is to my knowledge the first such crash I have seen. On that note, it is completely possible that my problems originates from malfunctioning hardware. Any help in narrowing this down would be appreciated. | |||||||||||||||||||||||||||||||||||
| Comment by Alice Sherepa [ 2020-11-03 ] | |||||||||||||||||||||||||||||||||||
|
Server crashed as in
| |||||||||||||||||||||||||||||||||||
| Comment by Julian Rilli [ 2020-11-07 ] | |||||||||||||||||||||||||||||||||||
|
Hey, the same thing happened to me yesterday (mariadb-server-10.3 10.3.25-0+deb10u1; Linux Debian 10; Nextcloud 18.0.10.2; PHP 7.3.23). But the repair didn't work, with `--innodb-force-recovery=5` the mysql service came up again, but then mysql tables were missing in the engine. Well, I finally deleted the data folder, recreated the mysql user and restored the backup from the day before. | |||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2020-12-17 ] | |||||||||||||||||||||||||||||||||||
|
alice, what is the history of data.7z | |||||||||||||||||||||||||||||||||||
| Comment by Alice Sherepa [ 2020-12-17 ] | |||||||||||||||||||||||||||||||||||
|
marko, yes, data.7z is exactly the datadir after hitting bug from | |||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2021-04-26 ] | |||||||||||||||||||||||||||||||||||
|
We have daily internal testing that involves killing and restarting the server, or running Mariabackup concurrently with a write workload, and restoring the data. If this type of errors would occur in any way regularly, we would detect it. Two known sources of this type of problems are innodb_scrub_log (deprecated and ignored in Bad things can also happen if the server is being run on an unreliable file system, such as some implementations of NFS. Also, in the case of a power outage, one may learn in the hard way that some storage devices are cheating with volatile caches or by reordering writes. The messages "log sequence number is in the future" basically mean that you have to treat all InnoDB data as seriously corrupted. |