Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 10.3.12
    • Fix Version/s: 10.2.28, 10.3.19, 10.4.9
    • Component/s: Server
    • Labels:
      None
    • Environment:
      Live Data Factory running on Windows 2008 R1 32bit

      Description

      Experienced a crash of our primary MariaDB DB this evening at 23:15 , please can you help?

      from the MariaDB log file the crash details are:-

      190729 23:15:10 [ERROR] mysqld got exception 0xc000000d ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed, 
      something is definitely wrong and this may fail.
       
      Server version: 10.3.12-MariaDB-log
      key_buffer_size=8388608
      read_buffer_size=36864
      max_used_connections=266
      max_threads=65537
      thread_count=270
      It is possible that mysqld could use up to 
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2381676 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      ntdll.dll!EtwNotificationUnregister()
      mysqld.exe!timer_callback()[threadpool_win.cc:419]
      ntdll.dll!RtlIsCriticalSectionLockedByThread()
      ntdll.dll!RtlSizeHeap()
      kernel32.dll!BaseThreadInitThunk()
      ntdll.dll!RtlInitializeExceptionChain()
      ntdll.dll!RtlInitializeExceptionChain()
      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.
      2019-07-29 23:18:21 0 [Note] InnoDB: Adjusting innodb_buffer_pool_instances from 8 to 1 since innodb_buffer_pool_size is less than 1024 MiB
      2019-07-29 23:18:21 0 [Note] InnoDB: Mutexes and rw_locks use Windows interlocked functions
      2019-07-29 23:18:21 0 [Note] InnoDB: Uses event mutexes
      2019-07-29 23:18:21 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2019-07-29 23:18:21 0 [Note] InnoDB: Number of pools: 1
      2019-07-29 23:18:21 0 [Note] InnoDB: Using SSE2 crc32 instructions
      2019-07-29 23:18:21 0 [Note] InnoDB: Initializing buffer pool, total size = 384M, instances = 1, chunk size = 128M
      2019-07-29 23:18:21 0 [Note] InnoDB: Completed initialization of buffer pool
      2019-07-29 23:18:21 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=150962067982
      2019-07-29 23:18:22 0 [Note] InnoDB: Transaction 294057132 was in the XA prepared state.
      2019-07-29 23:18:22 0 [Note] InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 0 row operations to undo
      2019-07-29 23:18:22 0 [Note] InnoDB: Trx id counter is 294057133
      2019-07-29 23:18:22 0 [Note] InnoDB: Starting final batch to recover 18 pages from redo log.
      2019-07-29 23:18:22 0 [Note] InnoDB: Last binlog file 'd:\MariaDB\master\logbin\Master-bin.000615', position 146245658
      2019-07-29 23:18:26 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
      2019-07-29 23:18:26 0 [Note] InnoDB: Starting in background the rollback of recovered transactions
      2019-07-29 23:18:26 0 [Note] InnoDB: Rollback of non-prepared transactions completed
      2019-07-29 23:18:26 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
      2019-07-29 23:18:26 0 [Note] InnoDB: Creating shared tablespace for temporary tables
      2019-07-29 23:18:26 0 [Note] InnoDB: Setting file '.\ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
      2019-07-29 23:18:26 0 [Note] InnoDB: File '.\ibtmp1' size is now 12 MB.
      2019-07-29 23:18:26 0 [Note] InnoDB: 10.3.12 started; log sequence number 150962117583; transaction id 294057135
      2019-07-29 23:18:26 0 [Note] InnoDB: Loading buffer pool(s) from D:\MariaDB\data\ib_buffer_pool
      2019-07-29 23:18:26 0 [Note] Plugin 'FEEDBACK' is disabled.
      2019-07-29 23:18:26 0 [Note] Recovering after a crash using d:/MariaDB/master/logbin/Master-bin
      2019-07-29 23:18:35 0 [Note] InnoDB: Buffer pool(s) load completed at 190729 23:18:35
      2019-07-29 23:18:44 0 [Note] Starting crash recovery...
      2019-07-29 23:18:44 0 [Note] InnoDB: Starting recovery for XA transactions...
      2019-07-29 23:18:44 0 [Note] InnoDB: Transaction 294057132 in prepared state after recovery
      2019-07-29 23:18:44 0 [Note] InnoDB: Transaction contains changes to 1 rows
      2019-07-29 23:18:44 0 [Note] InnoDB: 1 transactions in prepared state after recovery
      2019-07-29 23:18:44 0 [Note] Found 1 prepared transaction(s) in InnoDB
      2019-07-29 23:18:44 0 [Note] Crash recovery finished.
      2019-07-29 23:18:44 0 [Note] Server socket created on IP: '::'.
      2019-07-29 23:18:44 0 [Warning] 'user' entry 'root@jpdpt3' ignored in --skip-name-resolve mode.
      2019-07-29 23:18:44 0 [Warning] 'proxies_priv' entry '@% root@jpdpt3' ignored in --skip-name-resolve mode.
      2019-07-29 23:18:44 0 [Note] Reading of all Master_info entries succeded
      2019-07-29 23:18:44 0 [Note] Added new Master_info '' to hash table
      2019-07-29 23:18:44 0 [Note] D:\MariaDB\bin\mysqld.exe: ready for connections.
      Version: '10.3.12-MariaDB-log'  socket: ''  port: 3306  mariadb.org binary distribution
      2019-07-29 23:19:23 81 [Note] Start binlog_dump to slave_server(2), pos(Master-bin.000615, 146246161)
      2019-07-29 23:19:52 90 [Warning] Access denied for user 'root'@'127.0.0.1' (using password: YES)
      2019-07-29 23:24:11 0 [Note] D:\MariaDB\bin\mysqld.exe (initiated by: unknown): Normal shutdown
      2019-07-29 23:24:11 0 [Note] Event Scheduler: Purging the queue. 0 events
      2019-07-29 23:24:11 0 [Note] InnoDB: FTS optimize thread exiting.
      2019-07-29 23:24:11 0 [Note] InnoDB: Starting shutdown...
      2019-07-29 23:24:11 0 [Note] InnoDB: Dumping buffer pool(s) to D:\MariaDB\data\ib_buffer_pool
      2019-07-29 23:24:11 0 [Note] InnoDB: Instance 0, restricted to 6144 pages due to innodb_buf_pool_dump_pct=25
      2019-07-29 23:24:11 0 [Note] InnoDB: Buffer pool(s) dump completed at 190729 23:24:11
      2019-07-29 23:24:12 0 [Note] InnoDB: Shutdown completed; log sequence number 150964259475; transaction id 294063316
      2019-07-29 23:24:12 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
      2019-07-29 23:24:12 0 [Note] D:\MariaDB\bin\mysqld.exe: Shutdown complete
       
      2019-07-29 23:24:16 0 [Note] InnoDB: Adjusting innodb_buffer_pool_instances from 8 to 1 since innodb_buffer_pool_size is less than 1024 MiB
      2019-07-29 23:24:16 0 [Note] InnoDB: Mutexes and rw_locks use Windows interlocked functions
      2019-07-29 23:24:16 0 [Note] InnoDB: Uses event mutexes
      2019-07-29 23:24:16 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2019-07-29 23:24:16 0 [Note] InnoDB: Number of pools: 1
      2019-07-29 23:24:16 0 [Note] InnoDB: Using SSE2 crc32 instructions
      2019-07-29 23:24:16 0 [Note] InnoDB: Initializing buffer pool, total size = 384M, instances = 1, chunk size = 128M
      2019-07-29 23:24:16 0 [Note] InnoDB: Completed initialization of buffer pool
      2019-07-29 23:24:16 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
      2019-07-29 23:24:16 0 [Note] InnoDB: Creating shared tablespace for temporary tables
      2019-07-29 23:24:16 0 [Note] InnoDB: Setting file '.\ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
      2019-07-29 23:24:16 0 [Note] InnoDB: File '.\ibtmp1' size is now 12 MB.
      2019-07-29 23:24:16 0 [Note] InnoDB: 10.3.12 started; log sequence number 150964259475; transaction id 294063318
      2019-07-29 23:24:16 0 [Note] InnoDB: Loading buffer pool(s) from D:\MariaDB\data\ib_buffer_pool
      2019-07-29 23:24:16 0 [Note] Plugin 'FEEDBACK' is disabled.
      2019-07-29 23:24:16 0 [Note] Server socket created on IP: '::'.
      2019-07-29 23:24:16 0 [Warning] 'user' entry 'root@jpdpt3' ignored in --skip-name-resolve mode.
      2019-07-29 23:24:16 0 [Warning] 'proxies_priv' entry '@% root@jpdpt3' ignored in --skip-name-resolve mode.
      2019-07-29 23:24:16 0 [Note] Reading of all Master_info entries succeded
      2019-07-29 23:24:16 0 [Note] Added new Master_info '' to hash table
      2019-07-29 23:24:16 0 [Note] D:\MariaDB\bin\mysqld.exe: ready for connections.
      Version: '10.3.12-MariaDB-log'  socket: ''  port: 3306  mariadb.org binary distribution
      2019-07-29 23:24:31 0 [Note] InnoDB: Buffer pool(s) load completed at 190729 23:24:31
      2019-07-29 23:25:12 65 [Note] Start binlog_dump to slave_server(2), pos(Master-bin.000616, 1808629)
      2019-07-29 23:37:53 251 [Warning] Access denied for user 'zzdomainadmin'@'127.0.0.1' (using password: NO)
      

      at 23:15:11 the following appeared in the application event log on the server

      Faulting application mysqld.exe, version 10.3.12.0, time stamp 0x5c2f7f3f, faulting module ntdll.dll, version 6.0.6003.20562, time stamp 0x5cfa798d, exception code 0xc000000d, fault offset 0x00087471, process id 0x124, application start time 0x01d545acd61aec10.
      

      There appears to be no hardware issue (it is running on a Virtual Machine under vmware ESXi), and all other VMs and other processes are ok; no ballooning memory; it had plenty of memory available etc...

      The DB does run in Binlog mode and has a standby DB

      I have done a mysqlcheck --all-databases and all appears to be ok

      Can you urgently help please?

      Is this a known bug and is it fixed in a later version of MariaDB and if so which version please?

      Sorry for all the questions, I am panicking a great deal.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                wlad Vladislav Vaintroub
                Reporter:
                slaterss Stuart Slater
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: