Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-32172

introspect server's replication settings from SQL stored routines

Details

    Description

      Background

      Historically, the only command available for showing complete information on replication status in MySQL and MariaDB has been the SHOW REPLICA STATUS command (formerly known as SHOW SLAVE STATUS). See MySQL docs and MariaDB docs.

      This command has a major downside: its output cannot be captured and used in a SQL SELECT query, subquery, or SQL stored routine. All of the following result in syntax errors:

      • SHOW REPLICA STATUS LIKE 'Last_Errno
      • SHOW SLAVE STATUS INTO Last_Errno, Master_Log_File
      • SELECT Last_Errno, Master_Log_File FROM (SHOW SLAVE STATUS)
      • SET @array = (SHOW SLAVE STATUS)

      The frequently-cited workaround is to execute the SHOW SLAVE STATUS command from some external program via a connector/driver; at that layer, the output of SHOW REPLICA STATUS can be read and manipulated in a way that's equivalent to the output of a SELECT query.

      Problems this causes

      However, because of this gap between the SQL syntax and the information sought, it was impossible to write a SQL stored routine which introspects the replication status of the MySQL/MariaDB server on which it's running.

      Numerous blog posts and StackOverflow questions over the course of decades attest to frustration and confusion over this situation, e.g.:

      Incomplete fixes in MySQL 5.7 and MariaDB 10.5

      Starting in MySQL 5.7 and MariaDB 10.5, some of the information conveyed in the output of SHOW SLAVE STATUS became available via SELECT queries on the PERFORMANCE_SCHEMA.REPLICATION* views.

      However, even with the addition of the relevant performance-schema views, not all of the information available in the output of SHOW SLAVE STATUS is available via normal SELECT queries.

      The MariaDB documentation does not explicitly note this, but the MySQL documentation does:

      several SHOW SLAVE STATUS columns are not preserved in the Performance Schema replication tables:

      [list of such columns]

      A 2018 blog post by @wisborg helpfully shows the gaps in the form of a table mapping SHOW SLAVE STATUS output fields to performance-schema tables and columns:

      Potential solutions

      1. Improve the syntactic flexibility of SHOW SLAVE STATUS, so that it's possible to do e.g. SELECT field FROM (SHOW SLAVE STATUS).
      2. Ensure that all of the internal replication-related metadata that is exposed in SHOW SLAVE STATUS is also available in performance_schema views.

      (1) depends on extending a non-standard and inconsistent syntax syntax. I strongly prefer (2).

      Attachments

        Issue Links

          Activity

            dlenski Daniel Lenski (Inactive) created issue -
            dlenski Daniel Lenski (Inactive) made changes -
            Field Original Value New Value
            Description h2. Background
            Historically, the only command available for showing complete
            information on replication status in MySQL and MariaDB has been the
            {{SHOW REPLICA STATUS}} command (formerly known as {{SHOW SLAVE STATUS}}). See [MySQL docs|https://dev.mysql.com/doc/refman/5.7/en/show-slave-status.html] and [MariaDB docs|https://mariadb.com/kb/en/show-replica-status].

            This command has a major downside: its output cannot be captured and
            used in a SQL {{SELECT}} query, subquery, or SQL stored routine.
            All of the following result in syntax errors:
            - {{SHOW REPLICA STATUS LIKE 'Last_Errno}}
            - {{SHOW SLAVE INTO Last_Errno, Master_Log_File}}
            - {{SELECT Last_Errno, Master_Log_File FROM (SHOW SLAVE STATUS)}}
            - {{SET @array = (SHOW SLAVE STATUS)}}

            The frequently-cited workaround is to execute the {{SHOW SLAVE STATUS}}
            command from some external program via a connector/driver; at that
            layer, the output of {{SHOW REPLICA STATUS}} _can_ be read and manipulated
            in a way that's equivalent to the output of a {{SELECT}} query.

            h2. Problems this causes
            However, because of this gap between the SQL syntax and the
            information sough, it was {color:red}impossible to write a SQL stored
            routine which introspects the replication status of the MySQL/MariaDB
            server on which it's running{color}.

            Numerous blog posts and StackOverflow questions over the course of
            decades attest to frustration and confusion over this situation, e.g.:

            * "Can we capture only Slave_IO_Running in SHOW SLAVE STATUS in MySQL" (2011)
              https://dba.stackexchange.com/questions/12554
            * "What is the SELECT statement equivalent of SHOW ALL SLAVES STATUS?" (2021)
              https://dba.stackexchange.com/questions/287263

            h3. Incomplete fixes in MySQL 5.7 and MariaDB 10.5

            Starting in MySQL 5.7 and MariaDB 10.5, *some of* the information conveyed
            in the output of {{SHOW SLAVE STATUS}} became [available via {{SELECT}} queries
            on the {{PERFORMANCE_SCHEMA.REPLICATION*}} views|https://mariadb.com/kb/en/list-of-performance-schema-tables].

            However, even *with*
            the addition of the relevant performance-schema views, {color:red}not all of the
            information available in the output of {{SHOW SLAVE STATUS}} is available via
            normal {{SELECT}} queries.{color}

            The MariaDB documentation does not explicitly note this, but [the MySQL
            documentation does|https://dev.mysql.com/doc/refman/5.7/en/performance-schema-replication-tables.html#idm46228724639264]:

            {quote}several SHOW SLAVE STATUS columns are not preserved in the
            Performance Schema replication tables:

            \[list of such columns\]{quote}

            A [2018 blog post by @wisborg|https://mysql.wisborg.dk/2018/10/05/replication-monitoring-with-the-performance-schema/#old-versus-new] helpfully shows the gaps in the form of a
            table mapping {{SHOW SLAVE STATUS}} output fields to performance-schema
            tables and columns:

            !Screenshot from 2023-09-14 13-42-15.png!

            h2. Potential solutions

            # Improve the syntactic flexibility of {{SHOW SLAVE STATUS}}, so that it's possible to do e.g. {{SELECT field FROM (SHOW SLAVE STATUS)}}.
            # Ensure that *all* of the internal replication-related metadata that is exposed in {{SHOW SLAVE STATUS}} is _also available_ in {{performance_schema}} views.

            (1) depends on extending a non-standard and inconsistent syntax syntax. I strongly prefer (2).
            h2. Background
            Historically, the only command available for showing complete information on replication status in MySQL and MariaDB has been the {{SHOW REPLICA STATUS}} command (formerly known as {{SHOW SLAVE STATUS}}). See [MySQL docs|https://dev.mysql.com/doc/refman/5.7/en/show-slave-status.html] and [MariaDB docs|https://mariadb.com/kb/en/show-replica-status].

            This command has a major downside: its output cannot be captured and used in a SQL {{SELECT}} query, subquery, or SQL stored routine. All of the following result in syntax errors:
            - {{SHOW REPLICA STATUS LIKE 'Last_Errno}}
            - {{SHOW SLAVE INTO Last_Errno, Master_Log_File}}
            - {{SELECT Last_Errno, Master_Log_File FROM (SHOW SLAVE STATUS)}}
            - {{SET @array = (SHOW SLAVE STATUS)}}

            The frequently-cited workaround is to execute the {{SHOW SLAVE STATUS}} command from some external program via a connector/driver; at that layer, the output of {{SHOW REPLICA STATUS}} _can_ be read and manipulated in a way that's equivalent to the output of a {{SELECT}} query.

            h2. Problems this causes
            However, because of this gap between the SQL syntax and the information sought, it was {color:red}impossible to write a SQL stored routine which introspects the replication status of the MySQL/MariaDB server on which it's running{color}.

            Numerous blog posts and StackOverflow questions over the course of decades attest to frustration and confusion over this situation, e.g.:

            * "Can we capture only Slave_IO_Running in SHOW SLAVE STATUS in MySQL" (2011)
              https://dba.stackexchange.com/questions/12554
            * "What is the SELECT statement equivalent of SHOW ALL SLAVES STATUS?" (2021)
              https://dba.stackexchange.com/questions/287263

            h3. Incomplete fixes in MySQL 5.7 and MariaDB 10.5

            Starting in MySQL 5.7 and MariaDB 10.5, *some of* the information conveyed in the output of {{SHOW SLAVE STATUS}} became [available via {{SELECT}} queries on the {{PERFORMANCE_SCHEMA.REPLICATION*}} views|https://mariadb.com/kb/en/list-of-performance-schema-tables].

            However, even *with* the addition of the relevant performance-schema views, {color:orange}not all of the information available in the output of {{SHOW SLAVE STATUS}} is available via normal {{SELECT}} queries.{color}

            The MariaDB documentation does not explicitly note this, but [the MySQL documentation does|https://dev.mysql.com/doc/refman/5.7/en/performance-schema-replication-tables.html#idm46228724639264]:

            {quote}several SHOW SLAVE STATUS columns are not preserved in the Performance Schema replication tables:

            \[list of such columns\]{quote}

            A [2018 blog post by @wisborg|https://mysql.wisborg.dk/2018/10/05/replication-monitoring-with-the-performance-schema/#old-versus-new] helpfully shows the gaps in the form of a table mapping {{SHOW SLAVE STATUS}} output fields to performance-schema tables and columns:

            !Screenshot from 2023-09-14 13-42-15.png|thumb!

            h2. Potential solutions

            # Improve the syntactic flexibility of {{SHOW SLAVE STATUS}}, so that it's possible to do e.g. {{SELECT field FROM (SHOW SLAVE STATUS)}}.
            # Ensure that *all* of the internal replication-related metadata that is exposed in {{SHOW SLAVE STATUS}} is _also available_ in {{performance_schema}} views.

            (1) depends on extending a non-standard and inconsistent syntax syntax. I strongly prefer (2).
            dlenski Daniel Lenski (Inactive) made changes -
            Description h2. Background
            Historically, the only command available for showing complete information on replication status in MySQL and MariaDB has been the {{SHOW REPLICA STATUS}} command (formerly known as {{SHOW SLAVE STATUS}}). See [MySQL docs|https://dev.mysql.com/doc/refman/5.7/en/show-slave-status.html] and [MariaDB docs|https://mariadb.com/kb/en/show-replica-status].

            This command has a major downside: its output cannot be captured and used in a SQL {{SELECT}} query, subquery, or SQL stored routine. All of the following result in syntax errors:
            - {{SHOW REPLICA STATUS LIKE 'Last_Errno}}
            - {{SHOW SLAVE INTO Last_Errno, Master_Log_File}}
            - {{SELECT Last_Errno, Master_Log_File FROM (SHOW SLAVE STATUS)}}
            - {{SET @array = (SHOW SLAVE STATUS)}}

            The frequently-cited workaround is to execute the {{SHOW SLAVE STATUS}} command from some external program via a connector/driver; at that layer, the output of {{SHOW REPLICA STATUS}} _can_ be read and manipulated in a way that's equivalent to the output of a {{SELECT}} query.

            h2. Problems this causes
            However, because of this gap between the SQL syntax and the information sought, it was {color:red}impossible to write a SQL stored routine which introspects the replication status of the MySQL/MariaDB server on which it's running{color}.

            Numerous blog posts and StackOverflow questions over the course of decades attest to frustration and confusion over this situation, e.g.:

            * "Can we capture only Slave_IO_Running in SHOW SLAVE STATUS in MySQL" (2011)
              https://dba.stackexchange.com/questions/12554
            * "What is the SELECT statement equivalent of SHOW ALL SLAVES STATUS?" (2021)
              https://dba.stackexchange.com/questions/287263

            h3. Incomplete fixes in MySQL 5.7 and MariaDB 10.5

            Starting in MySQL 5.7 and MariaDB 10.5, *some of* the information conveyed in the output of {{SHOW SLAVE STATUS}} became [available via {{SELECT}} queries on the {{PERFORMANCE_SCHEMA.REPLICATION*}} views|https://mariadb.com/kb/en/list-of-performance-schema-tables].

            However, even *with* the addition of the relevant performance-schema views, {color:orange}not all of the information available in the output of {{SHOW SLAVE STATUS}} is available via normal {{SELECT}} queries.{color}

            The MariaDB documentation does not explicitly note this, but [the MySQL documentation does|https://dev.mysql.com/doc/refman/5.7/en/performance-schema-replication-tables.html#idm46228724639264]:

            {quote}several SHOW SLAVE STATUS columns are not preserved in the Performance Schema replication tables:

            \[list of such columns\]{quote}

            A [2018 blog post by @wisborg|https://mysql.wisborg.dk/2018/10/05/replication-monitoring-with-the-performance-schema/#old-versus-new] helpfully shows the gaps in the form of a table mapping {{SHOW SLAVE STATUS}} output fields to performance-schema tables and columns:

            !Screenshot from 2023-09-14 13-42-15.png|thumb!

            h2. Potential solutions

            # Improve the syntactic flexibility of {{SHOW SLAVE STATUS}}, so that it's possible to do e.g. {{SELECT field FROM (SHOW SLAVE STATUS)}}.
            # Ensure that *all* of the internal replication-related metadata that is exposed in {{SHOW SLAVE STATUS}} is _also available_ in {{performance_schema}} views.

            (1) depends on extending a non-standard and inconsistent syntax syntax. I strongly prefer (2).
            h2. Background
            Historically, the only command available for showing complete information on replication status in MySQL and MariaDB has been the {{SHOW REPLICA STATUS}} command (formerly known as {{SHOW SLAVE STATUS}}). See [MySQL docs|https://dev.mysql.com/doc/refman/5.7/en/show-slave-status.html] and [MariaDB docs|https://mariadb.com/kb/en/show-replica-status].

            This command has a major downside: its output cannot be captured and used in a SQL {{SELECT}} query, subquery, or SQL stored routine. All of the following result in syntax errors:
            - {{SHOW REPLICA STATUS LIKE 'Last_Errno}}
            - {{SHOW SLAVE STATUS INTO Last_Errno, Master_Log_File}}
            - {{SELECT Last_Errno, Master_Log_File FROM (SHOW SLAVE STATUS)}}
            - {{SET @array = (SHOW SLAVE STATUS)}}

            The frequently-cited workaround is to execute the {{SHOW SLAVE STATUS}} command from some external program via a connector/driver; at that layer, the output of {{SHOW REPLICA STATUS}} _can_ be read and manipulated in a way that's equivalent to the output of a {{SELECT}} query.

            h2. Problems this causes
            However, because of this gap between the SQL syntax and the information sought, it was {color:red}impossible to write a SQL stored routine which introspects the replication status of the MySQL/MariaDB server on which it's running{color}.

            Numerous blog posts and StackOverflow questions over the course of decades attest to frustration and confusion over this situation, e.g.:

            * "Can we capture only Slave_IO_Running in SHOW SLAVE STATUS in MySQL" (2011)
              https://dba.stackexchange.com/questions/12554
            * "What is the SELECT statement equivalent of SHOW ALL SLAVES STATUS?" (2021)
              https://dba.stackexchange.com/questions/287263

            h3. Incomplete fixes in MySQL 5.7 and MariaDB 10.5

            Starting in MySQL 5.7 and MariaDB 10.5, *some of* the information conveyed in the output of {{SHOW SLAVE STATUS}} became [available via {{SELECT}} queries on the {{PERFORMANCE_SCHEMA.REPLICATION*}} views|https://mariadb.com/kb/en/list-of-performance-schema-tables].

            However, even *with* the addition of the relevant performance-schema views, {color:orange}not all of the information available in the output of {{SHOW SLAVE STATUS}} is available via normal {{SELECT}} queries.{color}

            The MariaDB documentation does not explicitly note this, but [the MySQL documentation does|https://dev.mysql.com/doc/refman/5.7/en/performance-schema-replication-tables.html#idm46228724639264]:

            {quote}several SHOW SLAVE STATUS columns are not preserved in the Performance Schema replication tables:

            \[list of such columns\]{quote}

            A [2018 blog post by @wisborg|https://mysql.wisborg.dk/2018/10/05/replication-monitoring-with-the-performance-schema/#old-versus-new] helpfully shows the gaps in the form of a table mapping {{SHOW SLAVE STATUS}} output fields to performance-schema tables and columns:

            !Screenshot from 2023-09-14 13-42-15.png|thumb!

            h2. Potential solutions

            # Improve the syntactic flexibility of {{SHOW SLAVE STATUS}}, so that it's possible to do e.g. {{SELECT field FROM (SHOW SLAVE STATUS)}}.
            # Ensure that *all* of the internal replication-related metadata that is exposed in {{SHOW SLAVE STATUS}} is _also available_ in {{performance_schema}} views.

            (1) depends on extending a non-standard and inconsistent syntax syntax. I strongly prefer (2).
            dlenski Daniel Lenski (Inactive) made changes -
            serg Sergei Golubchik made changes -
            Assignee Andrei Elkin [ elkin ]
            serg Sergei Golubchik made changes -
            Issue Type Bug [ 1 ] Task [ 3 ]
            serg Sergei Golubchik made changes -
            Summary SQL stored routines cannot fully introspect server's replication settings introspect server's replication settings from SQL stored routines
            serg Sergei Golubchik made changes -
            Fix Version/s 11.4 [ 29301 ]
            serg Sergei Golubchik made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            serg Sergei Golubchik made changes -
            Assignee Andrei Elkin [ elkin ] Sergei Golubchik [ serg ]
            serg Sergei Golubchik made changes -
            Assignee Sergei Golubchik [ serg ] Andrei Elkin [ elkin ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            serg Sergei Golubchik made changes -
            bengrieser Ben Grieser made changes -
            serg Sergei Golubchik made changes -
            Fix Version/s 11.5 [ 29506 ]
            Fix Version/s 11.4 [ 29301 ]
            serg Sergei Golubchik made changes -
            Fix Version/s 11.6 [ 29515 ]
            Fix Version/s 11.5 [ 29506 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Fix Version/s 11.7 [ 29815 ]
            Fix Version/s 11.6 [ 29515 ]
            mariadb-jira-automation Jira Automation (IT) made changes -
            Zendesk Related Tickets 202085

            dlenski hi!

            As a part of MDEV-33856, information_schema.slave_status was added to show all fields that exist in SHOW ALL SLAVES STATUS. Internally, these outputs are populated using the same logic, so they will stay in-sync with future additions. Can I close this ticket, or is there still desired functionality missing?

            bnestere Brandon Nesterenko added a comment - dlenski hi! As a part of MDEV-33856 , information_schema.slave_status was added to show all fields that exist in SHOW ALL SLAVES STATUS. Internally, these outputs are populated using the same logic, so they will stay in-sync with future additions. Can I close this ticket, or is there still desired functionality missing?
            bnestere Brandon Nesterenko made changes -
            Assignee Andrei Elkin [ elkin ] Brandon Nesterenko [ JIRAUSER48702 ]
            bnestere Brandon Nesterenko made changes -
            Status In Review [ 10002 ] Stalled [ 10000 ]
            bnestere Brandon Nesterenko made changes -
            Status Stalled [ 10000 ] Needs Feedback [ 10501 ]
            bnestere Brandon Nesterenko made changes -
            bnestere Brandon Nesterenko made changes -
            bnestere Brandon Nesterenko made changes -
            Fix Version/s N/A [ 14700 ]
            Fix Version/s 11.7 [ 29815 ]
            Resolution Incomplete [ 4 ]
            Status Needs Feedback [ 10501 ] Closed [ 6 ]
            bnestere Brandon Nesterenko made changes -
            Resolution Incomplete [ 4 ]
            Status Closed [ 6 ] Stalled [ 10000 ]

            As the reporter is inactive, and MDEV-33856 appears to provide the desired functionality, I'm closing this as a duplicate.

            bnestere Brandon Nesterenko added a comment - As the reporter is inactive, and MDEV-33856 appears to provide the desired functionality, I'm closing this as a duplicate.
            bnestere Brandon Nesterenko made changes -
            Component/s Replication [ 10100 ]
            Fix Version/s 11.6.0 [ 29839 ]
            Fix Version/s N/A [ 14700 ]
            Resolution Duplicate [ 3 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            bnestere Brandon Nesterenko made changes -
            bnestere Brandon Nesterenko made changes -

            People

              bnestere Brandon Nesterenko
              dlenski Daniel Lenski (Inactive)
              Votes:
              2 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.