Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-21758

Events switched randomly to SLAVESIDE_DISABLED

Details

    Description

      After upgrade from 10.3.18 to 10.3.22, events in all databases probably randomly switched it's status to 'SLAVESIDE_DISABLED' (statuses before this happened were ENABLED and some events were DISABLED). I'm not using master-slave configuration at all, it is single server.

      Attachments

        Issue Links

          Activity

            azurit azurit created issue -

            Thanks for the report.

            The regression was introduced by this commit:

            commit 0fd5b11eb05be8e8e996b26d845aae3b863448d3
            Author: Jan Lindström <jan.lindstrom@mariadb.com>
            Date:   Fri Sep 6 14:54:22 2019 +0300
             
                MDEV-20511: Galera replication of events is not consistent
                
                After SST from master node (the one where event is ENABLED) - you will end up with the event enabled on two nodes, hence it's now being executed twice. It can be solved by comparing event's originator with server_id. if not equal, then change its status to 'SLAVESIDE_DISABLED'
            

            The change makes all events which were created by a server with a different server_id to get disabled. Moreover, they remain disabled even if the server is later started with the initial server_id again.

            create event ev on schedule every 1 minute do set @a= 1;
            select name, originator, status from mysql.event;
             
            --let $server_id= `SELECT @@global.server_id`
             
            --echo #
            --echo # Restarting server with server_id=100
            --echo #
            --let $restart_parameters= --server-id=100
            --source include/restart_mysqld.inc
             
            select @@global.server_id;
            select name, originator, status from mysql.event;
            --eval set global server_id= $server_id
             
            --echo #
            --echo # Restarting server with the original server_id=$server_id
            --echo #
            --let $restart_parameters=
            --source include/restart_mysqld.inc
             
            select @@global.server_id;
            select name, originator, status from mysql.event;
             
            # Cleanup
            drop event ev;
            

            10.2 8382f106

            select name, originator, status from mysql.event;
            name	originator	status
            ev	1	ENABLED
            #
            # Restarting server with server_id=100
            #
            select @@global.server_id;
            @@global.server_id
            100
            select name, originator, status from mysql.event;
            name	originator	status
            ev	1	SLAVESIDE_DISABLED
            set global server_id= 1;
            #
            # Restarting server with the original server_id=1
            #
            select @@global.server_id;
            @@global.server_id
            1
            select name, originator, status from mysql.event;
            name	originator	status
            ev	1	SLAVESIDE_DISABLED
            

            elenst Elena Stepanova added a comment - Thanks for the report. The regression was introduced by this commit: commit 0fd5b11eb05be8e8e996b26d845aae3b863448d3 Author: Jan Lindström <jan.lindstrom@mariadb.com> Date: Fri Sep 6 14:54:22 2019 +0300   MDEV-20511: Galera replication of events is not consistent After SST from master node (the one where event is ENABLED) - you will end up with the event enabled on two nodes, hence it's now being executed twice. It can be solved by comparing event's originator with server_id. if not equal, then change its status to 'SLAVESIDE_DISABLED' The change makes all events which were created by a server with a different server_id to get disabled. Moreover, they remain disabled even if the server is later started with the initial server_id again. create event ev on schedule every 1 minute do set @a= 1; select name , originator, status from mysql.event;   --let $server_id= `SELECT @@global.server_id`   --echo # --echo # Restarting server with server_id=100 --echo # --let $restart_parameters= --server-id=100 --source include/restart_mysqld.inc   select @@ global .server_id; select name , originator, status from mysql.event; --eval set global server_id= $server_id   --echo # --echo # Restarting server with the original server_id=$server_id --echo # --let $restart_parameters= --source include/restart_mysqld.inc   select @@ global .server_id; select name , originator, status from mysql.event;   # Cleanup drop event ev; 10.2 8382f106 select name , originator, status from mysql.event; name originator status ev 1 ENABLED # # Restarting server with server_id=100 # select @@ global .server_id; @@ global .server_id 100 select name , originator, status from mysql.event; name originator status ev 1 SLAVESIDE_DISABLED set global server_id= 1; # # Restarting server with the original server_id=1 # select @@ global .server_id; @@ global .server_id 1 select name , originator, status from mysql.event; name originator status ev 1 SLAVESIDE_DISABLED
            elenst Elena Stepanova made changes -
            Field Original Value New Value
            elenst Elena Stepanova made changes -
            Component/s Events [ 10102 ]
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s 10.4 [ 22408 ]
            Affects Version/s 10.2 [ 14601 ]
            Affects Version/s 10.3 [ 22126 ]
            Affects Version/s 10.4 [ 22408 ]
            Assignee Jan Lindström [ jplindst ]
            Labels regression
            Priority Major [ 3 ] Blocker [ 1 ]
            azurit azurit made changes -
            Description After upgrade from 10.3.18 to 10.3.22, events in all databases probably randomly switched it's status to 'SLAVESIDE_DISABLED' (statuses before this happened were ENABLED and some events were DISABLED). I'm not using server-slave configuration at all, it is single server. After upgrade from 10.3.18 to 10.3.22, events in all databases probably randomly switched it's status to 'SLAVESIDE_DISABLED' (statuses before this happened were ENABLED and some events were DISABLED). I'm not using master-slave configuration at all, it is single server.
            jplindst Jan Lindström (Inactive) made changes -
            Status Open [ 1 ] In Progress [ 3 ]

            If SST is done from a galera node that is also acting as MASTER newly synced node in galera eco-system will also copy-over the event state enabling duplicate event in galera eco-system. Identification uses naturally server_id so not sure what you can do here to (1) avoid enabling events that should not be enabled and (2) allow events that should be enabled to be enabled.

            jplindst Jan Lindström (Inactive) added a comment - If SST is done from a galera node that is also acting as MASTER newly synced node in galera eco-system will also copy-over the event state enabling duplicate event in galera eco-system. Identification uses naturally server_id so not sure what you can do here to (1) avoid enabling events that should not be enabled and (2) allow events that should be enabled to be enabled.
            jplindst Jan Lindström (Inactive) made changes -
            issue.field.resolutiondate 2020-03-09 11:58:56.0 2020-03-09 11:58:56.288
            jplindst Jan Lindström (Inactive) made changes -
            Fix Version/s 10.2.32 [ 24221 ]
            Fix Version/s 10.3.23 [ 24222 ]
            Fix Version/s 10.4.13 [ 24223 ]
            Fix Version/s 10.5.2 [ 24030 ]
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s 10.4 [ 22408 ]
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Closed [ 6 ]

            greenman, jacob.moorman,

            We should probably add some sort of a note or a warning for users recommending them to check their events (at least if there was a chance that server_id of their server has changed at least once). As I understand, the fix makes sure that the events don't get disabled anymore, but those which did will remain disabled, so people would need to update them manually to re-enable.

            elenst Elena Stepanova added a comment - greenman , jacob.moorman , We should probably add some sort of a note or a warning for users recommending them to check their events (at least if there was a chance that server_id of their server has changed at least once). As I understand, the fix makes sure that the events don't get disabled anymore, but those which did will remain disabled, so people would need to update them manually to re-enable.
            greenman Ian Gilfillan made changes -
            greenman Ian Gilfillan added a comment -

            Thanks elenst, will document - MDEV-21896

            greenman Ian Gilfillan added a comment - Thanks elenst , will document - MDEV-21896
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 104032 ] MariaDB v4 [ 157342 ]

            People

              jplindst Jan Lindström (Inactive)
              azurit azurit
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.