Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-21732

libmariadb3 symbols changed in stable release (10.3), breaks ABI compatiblity

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Not a Bug
    • 10.3(EOL)
    • N/A
    • libmariadb
    • None

    Description

      In Debian we use a libmariadb3.symbols file to track if and what symbols change in each release. The history for 10.3 has had now several symbol changes in last 6 months:

      Examples:
      https://salsa.debian.org/mariadb-team/mariadb-10.3/commit/f5804f0ac87b2301499ba8dd262d33fdbcfb44d7
      https://salsa.debian.org/mariadb-team/mariadb-10.3/commit/122fbbb85f1b401f6d292d399fd6197055d81898

      Full history: https://salsa.debian.org/mariadb-team/mariadb-10.3/commits/buster/debian/libmariadb3.symbols

      Question 1) What is the policy on this?

      Is this intentional or a bug? Were there decisions to ship new libmariadb3 versions in a stable release so that it is not fully backwards compatible or did it just happen? WIll you continue to do this?

      Question 2) How to mitigate issues?

      Should we just hope that the symbols changed are not consumed by anybody? So far I've seen only one complaint: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=951210

      Should libmariadb3 be split out from the server package so that it would in the future be released independently with it's own version numbers (3.0.x, 3.1.x etc)?

      Attachments

        Issue Links

          Activity

            I think that the first change is fine, new symbols were added, they have a newer version that follows semantic versioning rules.

            The second is strange. georg, could you please comment on that?

            serg Sergei Golubchik added a comment - I think that the first change is fine, new symbols were added, they have a newer version that follows semantic versioning rules. The second is strange. georg , could you please comment on that?
            georg Georg Richter added a comment -

            2nd issue is caused by fix for CONC-437. For some reasons (likely a previous copy and paste error), the nysql_get_timeout symbols were defined for _18 and _3.

            georg Georg Richter added a comment - 2nd issue is caused by fix for CONC-437 . For some reasons (likely a previous copy and paste error), the nysql_get_timeout symbols were defined for _18 and _3.

            otto, so, if I understand georg correctly, adding new symbols with a new library version is fine (as long as semver rules are being followed) and removing symbols is generally not fine, but in this case the symbols were added as a mistake — they shouldn't have been there in the first place and no application could be using them, because they were never in the libmysqlclient_18 library, so removal was safe.

            serg Sergei Golubchik added a comment - otto , so, if I understand georg correctly, adding new symbols with a new library version is fine (as long as semver rules are being followed) and removing symbols is generally not fine, but in this case the symbols were added as a mistake — they shouldn't have been there in the first place and no application could be using them, because they were never in the libmysqlclient_18 library, so removal was safe.

            In other words, neither change breaks ABI

            serg Sergei Golubchik added a comment - In other words, neither change breaks ABI

            For at least one user it did break the ABI: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=951210
            But that is just one.

            Related to this, I opened a PR against mariadb-10.2 to ship a libmariadb3.symbols file upstream, so that these symbol changes are easier to track and will be clearly noticed already during upstream development and not post-release when it is too late to: https://github.com/MariaDB/server/pull/1449

            Also, I noticed that currently the latest releases of MariaDB 10.2, 10.3, 10.4 and 10.5 all ship the MariaDB Connector C version 3.1.7:

            $ grep 'SET(CPACK_PACKAGE_VERSION_' libmariadb/CMakeLists.txt
            SET(CPACK_PACKAGE_VERSION_MAJOR 3)
            SET(CPACK_PACKAGE_VERSION_MINOR 1)
            SET(CPACK_PACKAGE_VERSION_PATCH 7)
            

            This is however not reflected in the Debian packaging. The libmariadb3 package is shipped with version 10.2.31, 10.3.22, 10.4.12 and 10.5.1 respectively using the server version (not 3.1.7).

            This means that at some point in the 10.2 stable release cycle the library version was bumped from 3.0.x to 3.1.x. Is this safe to do in a stable release?

            At what point does the MariaDB Connector C versions need to start to diverge (if ever)? Should we be prepared to ship the library with package name libmariadb31 or libmariadb32 at some point?

            Or is there just an assumption that we have the same version everywhere and it does not matter since the ABI is always backwards compatible?

            There is the server ABI also to consider. In upstream packaging the version is not separated in any way, but in downstream Debian packaging the server plugins are placed in the directory /usr/lib/<arch>/mariadb19/*.so, eg:

            mariadb-plugin-rocksdb.install:usr/lib/*/mariadb19/plugin/ha_rocksdb.so
            mariadb-plugin-spider.install:usr/lib/*/mariadb19/plugin/ha_spider.so
            mariadb-plugin-tokudb.install:usr/lib/*/mariadb19/plugin/ha_tokudb.so
            

            Use of mariadb18 started in https://salsa.debian.org/mariadb-team/mariadb-10.1/commit/5aecef81f774e245f60e8c3f70f44190f7473537 and was bumped to maraidb19 in https://salsa.debian.org/mariadb-team/mariadb-10.3/commit/b4a5e7b2e2ab6bff42edfc9317cfa1c61980dea3

            The versions in package names and paths exists to facilitate a migration from old to new, so that for some period of time both versions can co-exists at least on some level while all binaries that used a certain ABI version are recompiled for the new version. I don't have a clear picture how all of this will work in the future, so I decided to spell out what the situation is and maybe you have some comments..

            otto Otto Kekäläinen added a comment - For at least one user it did break the ABI: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=951210 But that is just one. Related to this, I opened a PR against mariadb-10.2 to ship a libmariadb3.symbols file upstream, so that these symbol changes are easier to track and will be clearly noticed already during upstream development and not post-release when it is too late to: https://github.com/MariaDB/server/pull/1449 Also, I noticed that currently the latest releases of MariaDB 10.2, 10.3, 10.4 and 10.5 all ship the MariaDB Connector C version 3.1.7: $ grep 'SET(CPACK_PACKAGE_VERSION_' libmariadb/CMakeLists.txt SET(CPACK_PACKAGE_VERSION_MAJOR 3) SET(CPACK_PACKAGE_VERSION_MINOR 1) SET(CPACK_PACKAGE_VERSION_PATCH 7) This is however not reflected in the Debian packaging. The libmariadb3 package is shipped with version 10.2.31, 10.3.22, 10.4.12 and 10.5.1 respectively using the server version ( not 3.1.7). This means that at some point in the 10.2 stable release cycle the library version was bumped from 3.0.x to 3.1.x. Is this safe to do in a stable release? At what point does the MariaDB Connector C versions need to start to diverge (if ever)? Should we be prepared to ship the library with package name libmariadb31 or libmariadb32 at some point? Or is there just an assumption that we have the same version everywhere and it does not matter since the ABI is always backwards compatible? There is the server ABI also to consider. In upstream packaging the version is not separated in any way, but in downstream Debian packaging the server plugins are placed in the directory /usr/lib/<arch>/mariadb19/*.so, eg: mariadb-plugin-rocksdb.install:usr/lib /*/mariadb19/plugin/ha_rocksdb.so mariadb-plugin-spider.install:usr/lib/*/ mariadb19/plugin/ha_spider.so mariadb-plugin-tokudb.install:usr/lib/*/mariadb19/plugin/ha_tokudb.so Use of mariadb18 started in https://salsa.debian.org/mariadb-team/mariadb-10.1/commit/5aecef81f774e245f60e8c3f70f44190f7473537 and was bumped to maraidb19 in https://salsa.debian.org/mariadb-team/mariadb-10.3/commit/b4a5e7b2e2ab6bff42edfc9317cfa1c61980dea3 The versions in package names and paths exists to facilitate a migration from old to new, so that for some period of time both versions can co-exists at least on some level while all binaries that used a certain ABI version are recompiled for the new version. I don't have a clear picture how all of this will work in the future, so I decided to spell out what the situation is and maybe you have some comments..
            otto Otto Kekäläinen added a comment - Related: https://jira.mariadb.org/browse/MDEV-14921

            To summarize status:

            • MariaDB Connector C follow semantic versioning: https://semver.org/
            • New symbols are added in minor versions, e.g. 3.0 -> 3.1 added some
            • Old symbols are removed only in major versions, e.g. next time when 4.0 is published
            • Since all 3.x.x are always backwards compatible (unless there is a mistake, like reported in this issue) it should be safe to release always the latest client library in MariaDB Server releases. Today we have e.g. MariaDB 10.1 that ships MariaDB Connector C 3.1.7, but some day we might have say server 10.3 with client libs 3.4.5 and it should just work.
            • I filed https://github.com/MariaDB/server/pull/1449 which will help ensure symbols changes are always done correctly (in 3.x only added)
            • Package name (and paths) will stay libmariadb3 until we one day get libmariadb4.
            • Packages libmariadb3 and libmariadb4 need to be co-installable so that consuming software have time to migrate/rebuild

            I suggest this issue would be closed when:

            otto Otto Kekäläinen added a comment - To summarize status: MariaDB Connector C follow semantic versioning: https://semver.org/ New symbols are added in minor versions, e.g. 3.0 -> 3.1 added some Old symbols are removed only in major versions, e.g. next time when 4.0 is published Since all 3.x.x are always backwards compatible (unless there is a mistake, like reported in this issue) it should be safe to release always the latest client library in MariaDB Server releases. Today we have e.g. MariaDB 10.1 that ships MariaDB Connector C 3.1.7, but some day we might have say server 10.3 with client libs 3.4.5 and it should just work. I filed https://github.com/MariaDB/server/pull/1449 which will help ensure symbols changes are always done correctly (in 3.x only added) Package name (and paths) will stay libmariadb3 until we one day get libmariadb4. Packages libmariadb3 and libmariadb4 need to be co-installable so that consuming software have time to migrate/rebuild I suggest this issue would be closed when: https://github.com/MariaDB/server/pull/1449 is merged AND a server version that ships https://jira.mariadb.org/browse/CONC-437 has been released so the "mistake" is fixed for server packaging

            People

              georg Georg Richter
              otto Otto Kekäläinen
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.