RHEL 7 VM

Extract from the log (to make it searchable in JIRA):

160816  9:36:29 InnoDB: Initializing buffer pool, size = 128.0M

160816  9:36:29 InnoDB: Completed initialization of buffer pool

160816  9:36:29 InnoDB: highest supported file format is Barracuda.

InnoDB: Log scan progressed past the checkpoint lsn 18647449

160816  9:36:29  InnoDB: Database was not shut down normally!

InnoDB: Starting crash recovery.

InnoDB: Reading tablespace information from the .ibd files...

InnoDB: Restoring possible half-written data pages from the doublewrite

InnoDB: buffer...

InnoDB: Doing recovery: scanned up to log sequence number 18647459

160816  9:36:29  InnoDB: Waiting for the background threads to start

160816  9:36:29  InnoDB: Assertion failure in thread 140100604172032 in file trx0purge.c line 848

InnoDB: Failing assertion: purge_sys->purge_trx_no <= purge_sys->rseg->last_trx_no

InnoDB: We intentionally generate a memory trap.

InnoDB: Submit a detailed bug report to http://bugs.mysql.com.

InnoDB: If you get repeated assertion failures or crashes, even

InnoDB: immediately after the mysqld startup, there may be

InnoDB: corruption in the InnoDB tablespace. Please refer to

InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html

InnoDB: about forcing recovery.

160816  9:36:29 [ERROR] mysqld got signal 6 ;

This could be because you hit a bug. It is also possible that this binary

jolson,

If you are experiencing the problem repeatedly, would it be possible to upload your datadir to our ftp (ftp.askmonty.org/private)?
Please also attach or upload your cnf file(s).

After that, since you probably need to get your server up and running, try to run it with innodb_force_recovery=2 (and if it doesn't help, with higher values).
See http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html for more details.

Elena,

Can't seem to connect to the ftp site (fireFTP).

jolson,

Regarding ftp, can you even open it? ftp.askmonty.org/private. Maybe there was a temporary glitch when you tried; but if it persists, maybe some firewall settings or whatever?

Regarding the table, what is the table you are referring to, and what do you mean by MyISAM instead of InnoDB? You created the table as InnoDB, it used to be InnoDB, but now it's MyISAM?
I can't imagine how it could be possible, but if it's so, we will be even more interested in your datadir.

But how did you discover it, if the server doesn't start?

Turns out that the table is myisam for full-text indexing. The site opens but I get the following error...553 Could not create file.
: /private/mariadb.txt

Also get this error in fireFTP debug window...

Debug: Not enough arguments [nsILoadContextInfoFactory.fromLoadContext]
baseProtocol.prototype.parseListData@chrome://fireftp/content/js/connection/baseProtocol.js:1032:11
ftpMozilla.prototype.readControl@chrome://fireftp/content/js/connection/controlSocket.js:661:27
ftpMozilla.prototype.connect/dataListener.onDataAvailable@chrome://fireftp/content/js/connection/controlSocket.js:77:11

Regarding FTP, your file mariadb.txt got uploaded fine. You are trying to override it, hence the problem.
Please name which other files (or at least how many), if any, you've uploaded. For future, it is always a good idea to add the JIRA issue number (in this case MDEV-10567) to file names, it will help to avoid conflicts with other uploads and will allow to recognize your files more easily.

I have no idea about fireFTP errors, sorry.

MyISAM tables have nothing to do with the InnoDB error that prevents your server from starting. As said before, you'll need to try innodb_force_recovery (but make a datadir backup first).

Just uploaded MDEV-10567-amaisd.zip. Data and log files all zipped up.

I have repaired the PageSearchIndex table

System is still crashing.

If by PageSearchIndex you mean the InnoDB table that you had corrupted, as I said before, the InnoDB assertion failure that you are having on startup has literally nothing to do with MyISAM tables, whether they are corrupted or not. Once again, as mentioned before more than once, you need to try innodb_force_recovery.

Yes well... I've tried that several times to no avail. Just to humor you I just ran it again. Here's the error message...

Job for mariadb.service failed because a fatal signal was delivered to the control process. See "systemctl status mariadb.service" and "journalctl -xe" for details.

Aug 19 08:17:16 web-ext.amaisd.org systemd[1]: mariadb.service stop-sigterm timed out. Killing.
Aug 19 08:17:16 web-ext.amaisd.org systemd[1]: mariadb.service: main process exited, code=killed, status=9/KILL
Aug 19 08:17:17 web-ext.amaisd.org systemd[1]: Failed to start MariaDB database server.
– Subject: Unit mariadb.service has failed
– Defined-By: systemd
– Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
–
-- Unit mariadb.service has failed.
–
-- The result is failed.

Apparently you and your snotty attitude have no answers. Guess I'll have to look somewhere else.

I'm sorry if you find the attitude snotty, but it's impossible to help when suggestions appear to be ignored. If you had mentioned earlier that you'd already tried the recovery option and it didn't help, we would have progressed further already and didn't have to circle around the same topic, which is a waste of time for everyone involved.

For the message above, it's just final systemd complaint, to see what's really wrong with the recovery attempt, we'll need the error log.

We are going to look into the data you provided earlier, but it will take time.
That said, if you know where else you can find help, by all means, you should try it, it can be beneficial for everyone.

Is the copy of the error log I uploaded sufficient or do you need the latest copy?

My apologies for not mentioning that I had already tried your suggestion. Hope your week has been better than mine.

Yes, it would be very interesting to see the latest copy of the error log, as it might say why innodb_force_recovery didn't work.

I've uploaded the log file.

After experimenting further - it is only the concrete DB that is malfunctioning. This is a backend for a Concrete5 CMS and when it crashes, the other DBs continue to function.

After yet more tinkering - disregard the foregoing - the instance actually isn't running.

I wanted to spend more time looking into your logs before answering, but just for a quick update, at the first glance I didn't see there any sign of picking up innodb_force_recovery option, so maybe the server somehow missed it. How did you provide it, via the config file, or on the command line? If it was the config file, was it certainly the one that the server uses?

Elena,

I believe I actually solved this. I deleted the databases, removed MariaDB from the server, re-installed and rebuilt the tables and it hase been stable all morning so far. Looks as if it must have been a bad piece of code stored in one of my tables.

Please comment if the problem re-surfaces.