Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-10167

MariaDB crashes with Xen PVH - mysqld got signal 11

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Not a Bug
    • 10.0.23, 10.0.25, 10.0(EOL)
    • N/A
    • Storage Engine - InnoDB
    • None

    Description

      Hello,
      after upgrading our Gentoo Linux systems to latest stable versions, one of our MariaDB instance started to crash. The crashes seems to be random at times like:

      160527 9:34:05 [ERROR] mysqld got signal 11 ;
      160527 13:52:37 [ERROR] mysqld got signal 11 ;
      160530 20:46:51 [ERROR] mysqld got signal 11 ;
      160602 5:32:48 [ERROR] mysqld got signal 11 ;

      We have a master-slave replication. We first upgraded the slave to 10.0.25, had it running for 2 weeks without problems. Then we did a master failover, it worked for a week and then it started to crash. First it crashed on MariaDB 10.0.25, then again on 10.0.25. We downgraded to MariaDB 10.0.23 (that version worked before the system update), but it crashed again. Then we switched back to the old master instance (failover) with MariaDB 10.0.23, but it crashed again.

      The MariaDB instances are running virtualized as Xen DomU (in pvh mode) on Linux Kernel 4.1.24 with 230GB RAM / 24 vcpus. MariaDB was compiled with the following Gentoo USE flags: "extraengine jemalloc openssl pam server".

      The database is mostly used for selects (1000-3000 qps).

      Attachments

        1. 01-crash-mariadb-10.0.25-1.log
          41 kB
        2. 02-crash-mariadb-10.0.25-2.log
          1 kB
        3. 03-crash-mariadb-10.0.23-1.log
          20 kB
        4. 04-crash-mariadb-10.0.23-2.log
          23 kB
        5. my.cnf
          2 kB
        6. mysqld.err
          23 kB

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-10020 InnoDB NOT IN Query Crash When One Item Is NULL

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            View 9 older comments
            jplindst Jan Lindström (Inactive) added a comment -

            Some of the crashes are long semaphore waits, but not all. Too bad that you did not provide full unedited error logs for all crash cases. Not familiar with Xen PVH or PV mode.

            jplindst Jan Lindström (Inactive) added a comment - Some of the crashes are long semaphore waits, but not all. Too bad that you did not provide full unedited error logs for all crash cases. Not familiar with Xen PVH or PV mode.
            hydrapolic Tomáš Mózes added a comment -

            @jplindst, please see the attached mysqld.err.

            hydrapolic Tomáš Mózes added a comment - @jplindst, please see the attached mysqld.err.
            jplindst Jan Lindström (Inactive) added a comment -

            I did, it does not contain the long semaphore wait error messages seen on other error logs and from startup to crash there is several hours and no error messages.

            jplindst Jan Lindström (Inactive) added a comment - I did, it does not contain the long semaphore wait error messages seen on other error logs and from startup to crash there is several hours and no error messages.
            hydrapolic Tomáš Mózes added a comment -

            @jplindst, I first attached the crash logs, then the surrounding logs. The information there is complete, just not in one place (sorry for that), so mysqld.err contains the full logs except the crash logs which are in separate logs.

            hydrapolic Tomáš Mózes added a comment - @jplindst, I first attached the crash logs, then the surrounding logs. The information there is complete, just not in one place (sorry for that), so mysqld.err contains the full logs except the crash logs which are in separate logs.
            hydrapolic Tomáš Mózes added a comment -

            This bug can be closed.

            This was caused by Xen PVH mode. After switching back to PV, the crashes are gone.

            Thanks for all the help.

            hydrapolic Tomáš Mózes added a comment - This bug can be closed. This was caused by Xen PVH mode. After switching back to PV, the crashes are gone. Thanks for all the help.

            People

              jplindst Jan Lindström (Inactive)
              hydrapolic Tomáš Mózes
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.