[MXS-2847] Replication does not stop after deleting the replication filter Created: 2020-01-24  Updated: 2020-01-27  Resolved: 2020-01-27

Status: Closed
Project: MariaDB MaxScale
Component/s: N/A
Affects Version/s: None
Fix Version/s: N/A

Type: Bug Priority: Blocker
Reporter: Petko Vasilev (Inactive) Assignee: Unassigned
Resolution: Not a Bug Votes: 0
Labels: externalDependency


 Description   

This relates to out htap deployment that uses maxscale's replication filter to replicate between a innodb and a columnstore table.

After enabling the replication, it seems to work correctly.

After disabling the replication it seems that it is not completely disabled.
It seems that it is still active, but without replacing the source database with the target database. This means that inserts in the source innodb table get duplicated.

steps to disable replication

  • unlink filter from service in maxscale
  • delete filter in maxscale
  • do "stop slave; start slave;" on the server to restart replication


 Comments   
Comment by Petko Vasilev (Inactive) [ 2020-01-24 ]

copying the logs from the related bug in DBAAS
==========================
Just some logs for posterity
create the tables, set replication and check that it's working:

MariaDB [(none)]> create database innodb_test;
Query OK, 1 row affected (0.004 sec)
 
MariaDB [(none)]> CREATE TABLE innodb_test.test1 (id INT, customer varchar(45)) ENGINE = INNODB;
Query OK, 0 rows affected (0.019 sec)
 
MariaDB [(none)]> create database columnstore_test;
Query OK, 1 row affected (0.002 sec)
 
MariaDB [(none)]> CREATE TABLE columnstore_test.test1 (id INTEGER, customer VARCHAR(45)) ENGINE=ColumnStore;
Query OK, 0 rows affected (0.263 sec)
 
MariaDB [(none)]> select set_replications('test1','innodb_test','columnstore_test');
+------------------------------------------------------------+
| set_replications('test1','innodb_test','columnstore_test') |
+------------------------------------------------------------+
| Success.                                                   |
+------------------------------------------------------------+
1 row in set (0.019 sec)
 
MariaDB [(none)]> INSERT INTO innodb_test.test1(id, customer) VALUES (5, 'whatever');
Query OK, 1 row affected (0.007 sec)
 
MariaDB [(none)]> select * from innodb_test.test1;
+------+----------+
| id   | customer |
+------+----------+
|    5 | whatever |
+------+----------+
1 row in set (0.000 sec)
 
MariaDB [(none)]> select * from columnstore_test.test1;
+------+----------+
| id   | customer |
+------+----------+
|    5 | whatever |
+------+----------+
1 row in set (0.069 sec)

The filter at this point

root@htap-sandbox-mdb-mxs-0:/# curl --user skysql_admin:test http://127.0.0.1:8989/v1/filters/replication_filter
{
    "links": {
        "self": "http://127.0.0.1:8989/v1/filters/replication_filter"
    },
    "data": {
        "id": "replication_filter",
        "type": "filters",
        "relationships": {
            "services": {
                "links": {
                    "self": "http://127.0.0.1:8989/v1/services/"
                },
                "data": [
                    {
                        "id": "Replication-Service",
                        "type": "services"
                    }
                ]
            }
        },
        "attributes": {
            "module": "binlogfilter",
            "parameters": {
                "match": "test1",
                "exclude": null,
                "rewrite_src": "innodb_test",
                "rewrite_dest": "columnstore_test"
            }
        },
        "links": {
            "self": "http://127.0.0.1:8989/v1/filters/replication_filter"
        }
    }
}

Then delete the filter and insert another row
Notice that the row has been duplicated

MariaDB [(none)]> select set_replications('','innodb_test','columnstore_test');
+-------------------------------------------------------+
| set_replications('','innodb_test','columnstore_test') |
+-------------------------------------------------------+
| Success.                                              |
+-------------------------------------------------------+
1 row in set (0.017 sec)
 
MariaDB [(none)]> INSERT INTO innodb_test.test1(id, customer) VALUES (5, 'whatever');
Query OK, 1 row affected (0.006 sec)
 
MariaDB [(none)]> select * from innodb_test.test1;
+------+----------+
| id   | customer |
+------+----------+
|    5 | whatever |
|    5 | whatever |
|    5 | whatever |
+------+----------+
3 rows in set (0.000 sec)
 
MariaDB [(none)]> select * from columnstore_test.test1;
+------+----------+
| id   | customer |
+------+----------+
|    5 | whatever |
+------+----------+
1 row in set (0.010 sec)

for reference, at this point the filter has been unlinked from the service and deleted

root@htap-sandbox-mdb-mxs-0:/# curl --user skysql_admin:test http://127.0.0.1:8989/v1/filters/replication_filter
root@htap-sandbox-mdb-mxs-0:/# curl --user skysql_admin:test http://127.0.0.1:8989/v1/services/Replication-Service
{
    "links": {
        "self": "http://127.0.0.1:8989/v1/services/Replication-Service"
    },
    "data": {
        "id": "Replication-Service",
        "type": "services",
        "attributes": {
            "router": "readconnroute",
            "state": "Started",
            "router_diagnostics": {
                "connections": 5,
                "current_connections": 1,
                "queries": 60
            },
            "started": "Fri Jan 24 15:24:41 2020",
            "total_connections": 5,
            "connections": 1,
            "parameters": {
                "router_options": null,
                "user": "skysql_replication",
                "password": "P?0|fB8M|TL7M-KmsfsQv.wjENVCX9Of",
                "enable_root_user": false,
                "max_retry_interval": 3600,
                "max_connections": 0,
                "connection_timeout": 0,
                "net_write_timeout": 0,
                "auth_all_servers": false,
                "strip_db_esc": true,
                "localhost_match_wildcard_host": true,
                "version_string": null,
                "weightby": null,
                "log_auth_warnings": true,
                "retry_on_failure": true,
                "session_track_trx_state": false,
                "retain_last_statements": -1,
                "session_trace": false,
                "cluster": null
            },
            "listeners": [
                {
                    "id": "Replication-Listener",
                    "type": "listeners",
                    "attributes": {
                        "state": "Running",
                        "parameters": {
                            "protocol": "MariaDBClient",
                            "port": 3309,
                            "socket": null,
                            "authenticator_options": "",
                            "address": "0.0.0.0",
                            "authenticator": null,
                            "ssl": "false",
                            "ssl_cert": null,
                            "ssl_key": null,
                            "ssl_ca_cert": null,
                            "ssl_version": "MAX",
                            "ssl_cert_verify_depth": 9,
                            "ssl_verify_peer_certificate": true
                        },
                        "authenticator_diagnostics": [
                            {
                                "user": "skysql_replication",
                                "host": "%"
                            },
                            {
                                "user": "skysql_replication",
                                "host": "localhost"
                            },
                            {
                                "user": "mysql",
                                "host": "localhost"
                            },
                            {
                                "user": "skysql_admin",
                                "host": "127.0.0.1"
                            },
                            {
                                "user": "skysql_admin",
                                "host": "localhost"
                            },
                            {
                                "user": "skysql_admin",
                                "host": "10.%"
                            },
                            {
                                "user": "skysql_monitoring",
                                "host": "10.%"
                            },
                            {
                                "user": "skysql_monitoring",
                                "host": "127.0.0.1"
                            },
                            {
                                "user": "skysql_cej",
                                "host": "127.0.0.1"
                            },
                            {
                                "user": "skysql_cej",
                                "host": "localhost"
                            },
                            {
                                "user": "skysql_maxscale",
                                "host": "10.%"
                            },
                            {
                                "user": "skysql_maxscale",
                                "host": "127.0.0.1"
                            },
                            {
                                "user": "skysql_replication",
                                "host": "10.%"
                            },
                            {
                                "user": "skysql_replication",
                                "host": "127.0.0.1"
                            },
                            {
                                "user": "skysql_maxscale",
                                "host": "10.%"
                            },
                            {
                                "user": "skysql_maxscale",
                                "host": "127.0.0.1"
                            },
                            {
                                "user": "skysql_replication",
                                "host": "10.%"
                            },
                            {
                                "user": "skysql_replication",
                                "host": "127.0.0.1"
                            }
                        ]
                    }
                }
            ]
        },
        "relationships": {
            "servers": {
                "links": {
                    "self": "http://127.0.0.1:8989/v1/servers/"
                },
                "data": [
                    {
                        "id": "HTAP-Node",
                        "type": "servers"
                    }
                ]
            }
        },
        "links": {
            "self": "http://127.0.0.1:8989/v1/services/Replication-Service"
        }
    }
}

===========================
I was thinking that a possible workaround would be to put an invalid regex in the "match".
something like "$a^" should not match anything and prevent this

Comment by markus makela [ 2020-01-24 ]

This is not a bug and is expected behavior when no filters are defined. If no filtering is done, the replication is routed unmodified. The filtering should either exclude all event or the replication should be stopped.

Comment by markus makela [ 2020-01-27 ]

This is expected behavior: replication without any filters should replicate all data.

To prevent replication:

  • Invert the match pattern to an exclude-all pattern: exclude=.*
  • Stop the listener associated with the service. This causes network connections to timeout.
  • Destroy the listener associated with the service. This causes network connections to be rejected.
Generated at Thu Feb 08 04:17:07 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.