Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-8575

MySQL refuse to start after a crash

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Blocker
    • Resolution: Incomplete
    • 10.1.6
    • N/A
    • Encryption
    • Ubuntu 12.04 + MariaDB 10.1.6 (from the repository)

    Description

      150807 14:58:27 mysqld_safe Starting mysqld daemon with databases from /data/mysql/data
      2015-08-07 14:58:28 140191626835840 [Note] /usr/sbin/mysqld (mysqld 10.1.6-MariaDB-1~precise-log) starting as process 7174 ...
      2015-08-07 14:58:28 140191626835840 [Note] Using encryption key id 1 for temporary files
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: Using mutexes to ref count buffer pool pages
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: The InnoDB memory heap is disabled
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: Memory barrier is not used
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: Compressed tables use zlib 1.2.3.4
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: Using Linux native AIO
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: Not using CPU crc32 instructions
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: Initializing buffer pool, size = 1.0G
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: Completed initialization of buffer pool
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: Highest supported file format is Barracuda.
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: Log scan progressed past the checkpoint lsn 21384312922
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: Database was not shutdown normally!
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: Starting crash recovery.
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: Reading tablespace information from the .ibd files...
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: Restoring possible half-written data pages
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: from the doublewrite buffer...
      InnoDB: Doing recovery: scanned up to log sequence number 21384329285
      2015-08-07 14:58:28 140191626835840 [Note] InnoDB: Starting an apply batch of log records to the database...
      InnoDB: Progress in percent: 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
      InnoDB: Apply batch completed
      InnoDB: Last MySQL binlog file position 0 3589410, file name /data/mysql/binlog/mariadb-bin.000003
      2015-08-07 14:58:30 140191626835840 [Note] InnoDB: 128 rollback segment(s) are active.
      2015-08-07 14:58:30 140191626835840 [Note] InnoDB: Waiting for purge to start
      2015-08-07 14:58:30 140191626835840 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.24-72.2 started; log sequence number 21384329285
      2015-08-07 14:58:30 7f80e81e3780  InnoDB: Assertion failure in thread 140191626835840 in file os0sync.cc line 419
      InnoDB: Failing assertion: event
      InnoDB: We intentionally generate a memory trap.
      InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
      InnoDB: If you get repeated assertion failures or crashes, even
      InnoDB: immediately after the mysqld startup, there may be
      InnoDB: corruption in the InnoDB tablespace. Please refer to
      InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
      InnoDB: about forcing recovery.
      150807 14:58:30 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see http://kb.askmonty.org/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed,
      something is definitely wrong and this may fail.
       
      Server version: 10.1.6-MariaDB-1~precise-log
      key_buffer_size=134217728
      read_buffer_size=131072
      max_used_connections=0
      max_threads=1002
      thread_count=0
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 10540258 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x0x0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x0 thread_stack 0x48000
      /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0x7f80e8c35e2b]
      /usr/sbin/mysqld(handle_fatal_signal+0x455)[0x7f80e87cdbd5]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0)[0x7f80e7dd9cb0]
      /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35)[0x7f80e617b0d5]
      /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b)[0x7f80e617e83b]
      /usr/sbin/mysqld(+0x756247)[0x7f80e8962247]
      /usr/sbin/mysqld(+0x89e448)[0x7f80e8aaa448]
      /usr/sbin/mysqld(+0x83ea5b)[0x7f80e8a4aa5b]
      /usr/sbin/mysqld(+0x847c44)[0x7f80e8a53c44]
      /usr/sbin/mysqld(+0x848e7d)[0x7f80e8a54e7d]
      /usr/sbin/mysqld(+0x84a013)[0x7f80e8a56013]
      /usr/sbin/mysqld(+0x84b0f3)[0x7f80e8a570f3]
      /usr/sbin/mysqld(+0x729f27)[0x7f80e8935f27]
      /usr/sbin/mysqld(+0x732c98)[0x7f80e893ec98]
      /usr/sbin/mysqld(+0x7ce657)[0x7f80e89da657]
      /usr/sbin/mysqld(+0x6f66d5)[0x7f80e89026d5]
      /usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x68)[0x7f80e87d0178]
      /usr/sbin/mysqld(+0x46c8c5)[0x7f80e86788c5]
      /usr/sbin/mysqld(_Z11plugin_initPiPPci+0x8e2)[0x7f80e8679822]
      /usr/sbin/mysqld(+0x3a63e7)[0x7f80e85b23e7]
      /usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x1d10)[0x7f80e85b4c10]
      /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7f80e616676d]
      /usr/sbin/mysqld(+0x39d099)[0x7f80e85a9099]
      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.
      150807 14:58:30 mysqld_safe mysqld from pid file /data/mysql/data/aurelien-VGN-CR42Z-R.pid ended

      The goal was to see impact on perf with and without encryption.

      my.cnf

      [client]
       
      # default-character-set = utf8
       
      [mysqld]
       
      performance_schema = on
      event_scheduler = on
      character-set-server = utf8
      open_files_limit =  65535
       
       
      character-set-client-handshake = FALSE
      collation-server = utf8_general_ci
       
      log_error = /data/mysql/log/error.log
       
      bind-address        = 0.0.0.0
      external-locking    = off
      skip-name-resolve
      datadir             = /data/mysql/data
       
      #make a crc32 of ip server
      server-id=476467
       
      replicate-ignore-db=mysql
      replicate-ignore-db=information_schema
      replicate-ignore-db=performance_schema
      skip-slave-start
       
      log-slave-updates
       
      #innodb
      innodb_buffer_pool_size = 1G
      innodb_flush_method     = O_DIRECT
       
       
       #for master
      log_bin                 = /data/mysql/binlog/mariadb-bin
      log_bin_index           = /data/mysql/binlog/mariadb-bin.index
       
      max_binlog_size         = 1G
      max_binlog_size         = 1G
      expire_logs_days        = 15
      binlog-ignore-db    = information_schema
      binlog-ignore-db    = mysql
      binlog-ignore-db    = performance_schema
       
      binlog_format = ROW
       
      #log
      slow_query_log_file     = /data/mysql/log/mariadb-slow.log
      long_query_time = 1
       
       
      # * Fine Tuning
      #
      max_connections         = 1000
      connect_timeout         = 60
      wait_timeout            = 600
      max_allowed_packet      = 16M
      thread_cache_size       = 128
      sort_buffer_size        = 10M
      bulk_insert_buffer_size = 16M
      tmp_table_size          = 64M
      max_heap_table_size     = 64M
       
       
      innodb_file_per_table
      innodb_autoinc_lock_mode = 2
      innodb_flush_log_at_trx_commit = 2
       
       
      [mysql]
      default-character-set   = utf8
       
      [mariadb-10.1]
      plugin-load-add=file_key_management.so
      file-key-management
      file-key-management-filename='key.enc'
      file_key_management_filekey='secretPassword2'
       
      innodb-encrypt-tables
      innodb-encrypt-log
      innodb-encryption-threads=2
      encrypt-tmp-disk-tables=1

      I made some test with sysbench and server crashed few time after (and server was inactif), because I could finish my test and I see that the encryption decrease perf of 15%.

      last sysbench :

      sysbench --test=/usr/share/doc/sysbench/tests/db/oltp.lua --num-threads=32 --mysql-host=localhost --mysql-user=root --db-driver=mysql --mysql-password=password --mysql-db=test --max-time=60 --max-requests=0 run

      (with table encrypted)

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              Aurelien_LEQUOY Aurélien LEQUOY
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.