Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-14493

Mariadb nodes freeze with 1927 internal error

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Incomplete
    • 10.2.10
    • N/A
    • Galera, Server
    • Ubuntu 16, 10.2.10+maria~xenial amd64

    Description

      When the system is loaded (only happens in production), I get the following log with a database freeze, upon restarting the mariadb process, I sometimes need to restore a backup due to partial data corruption that occurs (as far as the application - drupal is concerned).

      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Warning] WSREP: BF applier failed to open_and_lock_tables: 1927, fatal: 0 wsrep = (exec_mode: 1 conflict_state: 5 seqno: 903792)
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [ERROR] Slave SQL: Error executing row event: 'Connection was killed', Internal MariaDB error code: 1927
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Warning] WSREP: RBR event 3 Write_rows_v1 apply warning: 1927, 903792
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Note] WSREP: applier aborted: 903792
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Warning] WSREP: Failed to apply app buffer: seqno: 903792, status: 1
      Nov 23 21:42:55 websrv4 mysqld[1725]: #011 at galera/src/trx_handle.cpp:apply():351
      Nov 23 21:42:55 websrv4 mysqld[1725]: Retrying 2th time
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Warning] WSREP: BF applier failed to open_and_lock_tables: 1927, fatal: 0 wsrep = (exec_mode: 1 conflict_state: 5 seqno: 903792)
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [ERROR] Slave SQL: Error executing row event: 'Connection was killed', Internal MariaDB error code: 1927
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Warning] WSREP: RBR event 3 Write_rows_v1 apply warning: 1927, 903792
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Note] WSREP: applier aborted: 903792
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Warning] WSREP: Failed to apply app buffer: seqno: 903792, status: 1
      Nov 23 21:42:55 websrv4 mysqld[1725]: #011 at galera/src/trx_handle.cpp:apply():351
      Nov 23 21:42:55 websrv4 mysqld[1725]: Retrying 3th time
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Warning] WSREP: BF applier failed to open_and_lock_tables: 1927, fatal: 0 wsrep = (exec_mode: 1 conflict_state: 5 seqno: 903792)
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [ERROR] Slave SQL: Error executing row event: 'Connection was killed', Internal MariaDB error code: 1927
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Warning] WSREP: RBR event 3 Write_rows_v1 apply warning: 1927, 903792
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Note] WSREP: applier aborted: 903792
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Warning] WSREP: Failed to apply app buffer: seqno: 903792, status: 1
      Nov 23 21:42:55 websrv4 mysqld[1725]: #011 at galera/src/trx_handle.cpp:apply():351
      Nov 23 21:42:55 websrv4 mysqld[1725]: Retrying 4th time
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Warning] WSREP: BF applier failed to open_and_lock_tables: 1927, fatal: 0 wsrep = (exec_mode: 1 conflict_state: 5 seqno: 903792)
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [ERROR] Slave SQL: Error executing row event: 'Connection was killed', Internal MariaDB error code: 1927
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Warning] WSREP: RBR event 3 Write_rows_v1 apply warning: 1927, 903792
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Note] WSREP: applier aborted: 903792
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Warning] WSREP: failed to replay trx: source: 6ba51620-d084-11e7-9d2d-d7d0542d8744 version: 3 local: 1 state: REPLAYING flags: 1 conn_id: 475 trx_id: 1929982 seqnos (l: 3092, g: 903792, s: 903752, d: 903791, ts: 851004290800)
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [Warning] WSREP: Failed to apply trx 903792 4 times
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [ERROR] WSREP: trx_replay failed for: 6, schema: xxdbnamexx, query: `<94>B<B4>#035V
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081879742208 [ERROR] Aborting
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140082909173504 [Warning] Aborted connection 473 to db: 'xxxx' user: 'xxx' host: 'xxx' (Got an error reading communication packets)
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081878832896 [Warning] Aborted connection 478 to db: 'xxxx' user: 'xxx' host: 'xxx' (Got an error reading communication packets)
      Nov 23 21:42:55 websrv4 mysqld[1725]: 2017-11-23 21:42:55 140081773995776 [Warning] Aborted connection 507 to db: 'xxxx' user: 'xxxx' host: 'xxxx' (Deadlock found when trying to get lock; try restarting transacti)
      Nov 23 21:42:57 websrv4 mysqld[1725]: 2017-11-23 21:42:57 140082910992128 [Warning] Aborted connection 462 to db: 'xxxx' user: 'xxxx' host: 'xxxx' (Deadlock found when trying to get lock; try restarting transacti)
      Nov 23 21:42:57 websrv4 mysqld[1725]: 2017-11-23 21:42:57 140082910082816 [Warning] Aborted connection 461 to db: 'xxxx' user: 'xxxx' host: 'xxxx' (Deadlock found when trying to get lock; try restarting transacti)
      Nov 23 21:43:13 websrv4 mysqld[1725]: =====================================

      Attachments

        Activity

          People

            Unassigned Unassigned
            alon Alon O
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.