Signal 6 crash corrupts ibd files

2017-02-27 01:25:07 7f50466df700  InnoDB: Assertion failure in thread 139982755723008 in file buf0lru.cc line 2245

InnoDB: We intentionally generate a memory trap.

InnoDB: Submit a detailed bug report to http://bugs.mysql.com.

InnoDB: If you get repeated assertion failures or crashes, even

InnoDB: immediately after the mysqld startup, there may be

InnoDB: corruption in the InnoDB tablespace. Please refer to

InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html

InnoDB: about forcing recovery.

/usr/sbin/mysqld(my_print_stacktrace+0x2b)[0xb924ab]

/usr/sbin/mysqld(handle_fatal_signal+0x3b6)[0x739136]

/lib64/libpthread.so.0[0x323120f7e0]

/lib64/libc.so.6(gsignal+0x35)[0x3230e325e5]

/lib64/libc.so.6(abort+0x175)[0x3230e33dc5]

/usr/sbin/mysqld[0x9c86c9]

/usr/sbin/mysqld[0x8db7d5]

/usr/sbin/mysqld[0x9280e8]

/usr/sbin/mysqld[0x8835ff]

/usr/sbin/mysqld(_Z8closefrmP5TABLEb+0x3a)[0x67c43a]

/usr/sbin/mysqld(_Z18intern_close_tableP5TABLE+0x59)[0x59a609]

/usr/sbin/mysqld(_Z8tc_purgeb+0x285)[0x6e88f5]

/usr/sbin/mysqld(_Z19close_cached_tablesP3THDP10TABLE_LISTbm+0x8f)[0x5a317f]

/usr/sbin/mysqld(_Z20reload_acl_and_cacheP3THDyP10TABLE_LISTPi+0x3ce)[0x6c023e]

/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x4889)[0x5e3899]

/usr/sbin/mysqld[0x5e5a97]

/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1c6c)[0x5e7e6c]

/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x453)[0x6a6f13]

/usr/sbin/mysqld(handle_one_connection+0x42)[0x6a6fe2]

/lib64/libpthread.so.0[0x3231207aa1]

/lib64/libc.so.6(clone+0x6d)[0x3230ee8aad]

170227 01:25:09 mysqld_safe Number of processes running now: 0

170227 01:25:09 mysqld_safe mysqld restarted

170227  1:25:09 [Note] /usr/sbin/mysqld (mysqld 10.0.29-MariaDB) starting as process 23465 ...

170227  1:25:09 [Note] InnoDB: Using mutexes to ref count buffer pool pages

170227  1:25:09 [Note] InnoDB: The InnoDB memory heap is disabled

170227  1:25:09 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins

170227  1:25:09 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier

170227  1:25:09 [Note] InnoDB: Compressed tables use zlib 1.2.3

170227  1:25:09 [Note] InnoDB: Using Linux native AIO

170227  1:25:10 [Note] InnoDB: Using CPU crc32 instructions

170227  1:25:10 [Note] InnoDB: Initializing buffer pool, size = 7.0G

170227  1:25:10 [Note] InnoDB: Completed initialization of buffer pool

170227  1:25:10 [Note] InnoDB: Highest supported file format is Barracuda.

170227  1:25:10 [Note] InnoDB: Log scan progressed past the checkpoint lsn 6877178025086

170227  1:25:10 [Note] InnoDB: Database was not shutdown normally!

170227  1:25:10 [Note] InnoDB: Starting crash recovery.

170227  1:25:10 [Note] InnoDB: Reading tablespace information from the .ibd files...

170227  1:28:01 [ERROR] InnoDB: Space id in fsp header 1851878432,but in the page header 544434535

170227  1:28:01 [ERROR] InnoDB: innodb-page-size mismatch in tablespace ./optometrydb/page_manager_handlers.ibd (table optometrydb/page_manager_handlers)

170227  1:28:01 [Note] InnoDB: Page size:1024 Pages to analyze:64

170227  1:28:01 [Note] InnoDB: Page size: 1024, Possible space_id count:0

170227  1:28:01 [Note] InnoDB: Page size:2048 Pages to analyze:64

170227  1:28:01 [Note] InnoDB: Page size: 2048, Possible space_id count:0

170227  1:28:01 [Note] InnoDB: Page size:4096 Pages to analyze:32

170227  1:28:01 [Note] InnoDB: Page size: 4096, Possible space_id count:0

170227  1:28:01 [Note] InnoDB: Page size:8192 Pages to analyze:16

170227  1:28:01 [Note] InnoDB: Page size: 8192, Possible space_id count:0

170227  1:28:01 [Note] InnoDB: Page size:16384 Pages to analyze:8

170227  1:28:01 [Note] InnoDB: VALID: space:1859997 page_no:1 page_size:16384

170227  1:28:01 [Note] InnoDB: VALID: space:1859997 page_no:2 page_size:16384

170227  1:28:01 [Note] InnoDB: VALID: space:1859997 page_no:3 page_size:16384

170227  1:28:01 [Note] InnoDB: VALID: space:1859997 page_no:4 page_size:16384

170227  1:28:01 [Note] InnoDB: VALID: space:1859997 page_no:5 page_size:16384

170227  1:28:01 [Note] InnoDB: Page size: 16384, Possible space_id count:1

170227  1:28:01 [Note] InnoDB: space_id:1859997, Number of pages matched: 5/5 (16384)

170227  1:28:01 [Note] InnoDB: Chosen space:1859997

170227  1:28:01 [Note] InnoDB: Restoring page 0 of tablespace 1859997

170227  1:28:01 [Warning] InnoDB: Doublewrite does not have page_no=0 of space: 1859997

2017-02-27 01:28:01 7f7346996820  InnoDB: Operating system error number 2 in a file operation.

InnoDB: The error means the system cannot find the path specified.

InnoDB: If you are installing InnoDB, remember that you must create

InnoDB: directories yourself, InnoDB does not create them.

InnoDB: Error: could not open single-table tablespace file ./optometrydb/page_manager_handlers.ibd

InnoDB: We do not continue the crash recovery, because the table may become

InnoDB: corrupt if we cannot apply the log records in the InnoDB log to it.

InnoDB: To fix the problem and start mysqld:

InnoDB: 1) If there is a permission problem in the file and mysqld cannot

InnoDB: open the file, you should modify the permissions.

InnoDB: 2) If the table is not needed, or you can restore it from a backup,

InnoDB: then you can remove the .ibd file, and InnoDB will do a normal

InnoDB: crash recovery and ignore that table.

InnoDB: 3) If the file system or the disk is broken, and you cannot remove

InnoDB: the .ibd file, you can set innodb_force_recovery > 0 in my.cnf

InnoDB: and force InnoDB to continue crash recovery here.

170227 01:28:01 mysqld_safe mysqld from pid file /var/lib/mysqldata/mysql/DATABASES/whsqlprd05.pid ended