Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-6284

Merge downstream Debian/Ubuntu packaging into upstream MariaDB

Details

    • 10.2.2-2

    Description

      == Motivation ==

      Merge downstream .deb-packaging into upstream MariaDB to unify, streamline and ease future release packaging. This will also fix some packaging errors in current MariaDB.org deb-packages.

      == Patch description ==

      List of notable changes:

      • Completely new Debian packaging standard (3.0 quilt) including rules file, patch conventions and .files -> .install file listing conventions
      • The will no longer be a debian/dist subfolder and the packaging of Debian an Ubuntu will be unified (= Debian will ship with AppArmor files included)
      • Support for Debian releases before Wheezy (7) and Ubuntu releases before Precise (12.04) will be dropped

      The upstream MariaDB contents of debian/* will not be fully identical to the downstream Debian/Ubuntu one.

      List of intended differences between Debian master and MariaDB master regarding debian/ contents:

      • control file: maintainer, uploaders, VCS and browser links
      • omitted packages (not allowed in Debian): libmariadbclient18, libmysqlclient18, libmariadbclient-dev, mysql-common
      • omitted plugins (problematic in Debian): Handler socket, TokuDB, Mroonga, Cassandra
      • keep autobake.sh in mariadb.org repo

      To close this bug please:

      1) Merge selected parts of debian/* from https://anonscm.debian.org/gitweb/?p=pkg-mysql/mariadb-10.0.git to https://github.com/MariaDB/server via pull requests arriving from branch ok-debpkg at https://github.com/ottok/mariadb

      2) Backport form MariaDB development head to 10.1, 10.0 and maybe even 5.5 commits that fix individual issues.

      == Quality assurance ==

      The new debian/* contents represents the state of the art of Debian packaging. The mariadb-5.5 and mariadb-10.0 has passed all Debian quality assurance and landed in the official Debian repositories. More details at https://wiki.debian.org/Teams/MySQL/MariaDB

      These same packages have also gone downstream to Ubuntu, has passed Ubuntu QA and are included since the Ubuntu 14.04 release.

      Buildbot test for ok-debpkg branch: https://buildbot.askmonty.org/buildbot/grid?branch=ok-debpkg&category=main

      == Roll-out planning ==

      This merge should be safe to release with 10.1. All new installs and upgrades using the MariaDB.org repo or mixing it with official Debian or Ubuntu repository contents is expected to work.

      Below are the different scenarios how releasing the new packging at MariaDB.org will affect current installs/upgrades:

      A) If users currently have a MariaDB.org repo enabled, those packages will have a + in their version number. Debian package manager will always consider e.g. 10.1.6+maria as more recent than 10.1.6 from the native repositories. Therefore people having MariaDB.org repositories will always get primarly the packages from MariaDB.org repositories. It is very unlikely that a Debian repository would ever have a more recent release than in MariaDB.org (e.g. 10.1.7 would override 10.1.6+maria, but that is not likely to take place as MariaDB.org always releases first).

      B) The new packaging is designed and tested that it allows seamless upgrades from old deb packaging generation to new generation. Whether the new packaging is from MariaDB.org or distro repositories does not matter.

      C) Backwards works only partially, but is a rare case (e.g. current Ubuntu users trying to upgrade from new style debs 10.0 to MariaDB.org repo old style debs 10.0 will see issues, e.g. MDEV-5977). As soon as this merge is done and MariaDB.org publishes packages using the new packaging this corner case is fixed.

      D) If users remove the MariaDB.org from their install, they will automatically have their 10.1.6+maria updated to Debian/Ubuntu native 5.5.39 once it is released in Debian.

      E) Like previously, upgrading from MySQL 5.5 to MariaDB 5.5 works seamlessly and backwards also. Updating from MariaDB 5.5 to 10.0 works seamlessly but backwards will have issues (downgrade flag, InnoDB log size mismatch etc) the package manager will not solve automatically. If users dump and import data manually, and in between completely purge and install the packages, then users can always migrate to whatever version.

      Attachments

        1. ottok.diff
          552 kB
        2. screenshot-1.png
          screenshot-1.png
          60 kB
        3. screenshot-2.png
          screenshot-2.png
          111 kB

        Issue Links

          Activity

            otto Otto Kekäläinen created issue -
            serg Sergei Golubchik made changes -
            Field Original Value New Value
            Assignee Sergei Golubchik [ serg ]
            serg Sergei Golubchik made changes -
            Affects Version/s 5.5.38 [ 15400 ]
            serg Sergei Golubchik made changes -
            Fix Version/s 5.5.39 [ 15800 ]
            serg Sergei Golubchik made changes -
            Fix Version/s 5.5.38 [ 15400 ]
            Fix Version/s 5.5.39 [ 15800 ]
            serg Sergei Golubchik made changes -
            Fix Version/s 5.5.39 [ 15800 ]
            Fix Version/s 5.5.38 [ 15400 ]
            serg Sergei Golubchik made changes -
            Workflow defaullt [ 41621 ] MariaDB v2 [ 43220 ]
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -

            Ubuntu 14.04 packages (MariaDB 5.5.37) contain libmysqld., but both libmysqld.so and libmysqld.a are in 'libmariadbd-dev' package while in RPM (for example CentOS 6) libmysqld.a is in MariaDB-devel package and libmysqld.so* are in MariaDB-server package.
            As a result everything that is build under Ubuntu 14.04 depends on 'libmariadbd-dev' (on development package!)
            Are we going to merge this packaging ito MariaDB?

            tturenko Timofey Turenko added a comment - Ubuntu 14.04 packages (MariaDB 5.5.37) contain libmysqld. , but both libmysqld.so and libmysqld.a are in 'libmariadbd-dev' package while in RPM (for example CentOS 6) libmysqld.a is in MariaDB-devel package and libmysqld.so* are in MariaDB-server package. As a result everything that is build under Ubuntu 14.04 depends on 'libmariadbd-dev' (on development package!) Are we going to merge this packaging ito MariaDB?
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            Fix Version/s 10.0 [ 16000 ]

            About "Fix Version: 10.0" - please note that the merge request is for the 5.5 series only. I'll do a separate merge request for 10.0 and open a new JIRA issue for that, as the contents in debian/* for 10.0 is different enough to justify a separate issue. But before I push the 10.0 branch I'd like to see 5.5 merged first so we can learn from possible issues that arise in 5.5 work and fix not repeat them for 10.0.

            otto Otto Kekäläinen added a comment - About "Fix Version: 10.0" - please note that the merge request is for the 5.5 series only. I'll do a separate merge request for 10.0 and open a new JIRA issue for that, as the contents in debian/* for 10.0 is different enough to justify a separate issue. But before I push the 10.0 branch I'd like to see 5.5 merged first so we can learn from possible issues that arise in 5.5 work and fix not repeat them for 10.0.
            serg Sergei Golubchik made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            Status In Progress [ 3 ] In Review [ 10002 ]
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            Assignee Sergei Golubchik [ serg ] Otto Kekäläinen [ otto ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            otto Otto Kekäläinen made changes -
            otto Otto Kekäläinen made changes -
            otto Otto Kekäläinen made changes -
            otto Otto Kekäläinen made changes -
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            Assignee Otto Kekäläinen [ otto ] Sergei Golubchik [ serg ]
            serg Sergei Golubchik made changes -
            Status Stalled [ 10000 ] In Progress [ 3 ]
            serg Sergei Golubchik made changes -
            Assignee Sergei Golubchik [ serg ] Otto Kekäläinen [ otto ]

            serg This hasn't seen much progress because I've been busy with constantly updating and improving the Debian/Ubuntu packaging. Also the turnaround for request/review has been slow. What if we would meet and have a 1-2 day sprint to get this merge reviewed and pushed in one go?

            What if we also changed the strategy, and not merge the latest packaging into 10.0 and 5.5 but rather only on 10.1 and keep downstream and upstream Debian packaging in sync from 10.1 onwards?

            otto Otto Kekäläinen added a comment - serg This hasn't seen much progress because I've been busy with constantly updating and improving the Debian/Ubuntu packaging. Also the turnaround for request/review has been slow. What if we would meet and have a 1-2 day sprint to get this merge reviewed and pushed in one go? What if we also changed the strategy, and not merge the latest packaging into 10.0 and 5.5 but rather only on 10.1 and keep downstream and upstream Debian packaging in sync from 10.1 onwards?
            otto Otto Kekäläinen made changes -
            ratzpo Rasmus Johansson (Inactive) made changes -
            Workflow MariaDB v2 [ 43220 ] MariaDB v3 [ 64909 ]
            otto Otto Kekäläinen made changes -
            otto Otto Kekäläinen made changes -
            otto Otto Kekäläinen made changes -
            Summary Merge downstream MariaDB 5.5 Debian/Ubuntu packaging into maria/5.5 Merge downstream Debian/Ubuntu packaging into upstream MariaDB
            otto Otto Kekäläinen made changes -
            Affects Version/s 5.5.37 [ 15000 ]
            otto Otto Kekäläinen made changes -
            Component/s Packaging [ 10700 ]
            Component/s Platform Debian [ 10136 ]
            otto Otto Kekäläinen made changes -
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.1 [ 16100 ]
            Fix Version/s 10.0 [ 16000 ]
            Fix Version/s 5.5 [ 15800 ]
            otto Otto Kekäläinen made changes -
            Description == Motivation ==

            Merge downstream .deb-packaging into upstream MariaDB to unify, streamline and ease future 5.5.xx release packaging. This will also fix some packaging errors in current MariaDB.org deb-packages.

            == Patch description ==

            To close this bug please merge at https://code.launchpad.net/~otto/maria/maria-fix-debpkg-5.5 in to maria/5.5 branch.

            Merge request is open at https://code.launchpad.net/~otto/maria/maria-fix-debpkg-5.5/+merge/221576 (please remember to update status and set revision number once merged).

            The patch renews most of debian/* contents with new packaging that follows latest Debian policies, uses new dh rules file, new patch format, etc. Old autobake.sh is still there and some other minor cruft that might need to be processed later (e.g. the contents of dist/).

            The upstream MariaDB contents of debian/* will not be fully identical to the downstream Debian/Ubuntu one. Upstream MariaDB.org deb's will differ by *not* including the libmariadbclient.so rename and it *will* include the oqgraph module which was dropped downstream due to build dependency issues.

            == Versioning ==

            This patch is intended for the 5.5 branch. Later I'll send separate patches for 10.0 and 10.1.

            == Quality assurance ==

            The new debian/* contents represents the state of the art of Debian packaging. The mariadb-5.5 has passed all Debian quality assurance and landed in the Debian testing repositories. More details at https://wiki.debian.org/Teams/MySQL/MariaDBPlan

            These same packages have also gone downstream to Ubuntu, has passed Ubuntu QA and are included in the latest Ubuntu 14.04 release.

            The packaging source is mirrored at https://github.com/ottok/mariadb-5.5 which is build and successfully passes MariaDB.org Buildbot:
            http://buildbot.askmonty.org/buildbot/builders/debpkg-trusty
            http://buildbot.askmonty.org/buildbot/builders/debpkg-sid

            The contents of this patch differs slightly from the packaging source in git due to the changes described above, but I've tested that it builds on my own computer and considering the nature of the minor differences and overall quality it should be safe to merge this on the maria/5.5 branch.

            Still, just to make sure, I'll wait for this merge to be completed and verified by the full buildbot array of tests before I submit similar merge requests for 10.0 and 10.1 (trunk) branches.

            == Roll-out planning ==

            This merge should be safe to release with 5.5.38. All new installs and upgrades using the MariaDB.org repo or mixing it with official Debian or Ubuntu repository contents is expected to work.

            Below are the different scenarios how releasing the new packging at MariaDB.org will affect current installs/upgrades:

            A) If users currently have a MariaDB.org repo enabled, those packages will have a + in their version number. Debian package manager will always consider e.g. 5.5.38+maria as more recent than 5.5.38 from the native repositories. Therefore people having MariaDB.org repositories will always get primarly the packages from MariaDB.org repositories. It is very unlikely that a Debian repository would ever have a more recent release than in MariaDB.org (e.g. 5.5.39 would override 5.5.38+maria, but that is not likely to take place as MariaDB.org always releases first).

            B) The new packaging is designed and tested that it allows seamless upgrades from old deb packaging generation to new generation. Whether the new packaging is from MariaDB.org or distro repositories does not matter.

            C) Backwards works only partially, but is a rare case (e.g. current Ubuntu users trying to upgrade from new style debs 5.5.38 to MariaDB.org repo old style debs 5.5.38 will see issues, e.g. MDEV-5977). As soon as this merge is done and MariaDB.org publishes packages using the new packaging this corner case is fixed.

            D) If users remove the MariaDB.org from their install, they will automatically have their 5.5.38+maria updated to Debian/Ubuntu native 5.5.39 once it is released in Debian.

            E) Like previously, upgrading from MySQL 5.5 to MariaDB 5.5 works seamlessly and backwards also. Updating from MariaDB 5.5 to 10.0 works seamlessly but backwards will have issues (downgrade flag, InnoDB log size mismatch etc) the package manager will not solve automatically. If users dump and import data manually, and in between completely purge and install the packages, then users can always migrate to whatever version.
            == Motivation ==

            Merge downstream .deb-packaging into upstream MariaDB to unify, streamline and ease future release packaging. This will also fix some packaging errors in current MariaDB.org deb-packages.

            The patch renews most of debian/* contents with new packaging that follows latest Debian policies, uses new dh rules file, new patch format, etc. Old autobake.sh is still there and some other minor cruft that might need to be processed later (e.g. the contents of dist/).

            The upstream MariaDB contents of debian/* will not be fully identical to the downstream Debian/Ubuntu one. Upstream MariaDB.org deb's will differ by *not* including the libmariadbclient.so rename and it *will* include the oqgraph module which was dropped downstream due to build dependency issues.

            *TODO: write full list of intended differences between Debian master and MariaDB master regarding debian/* contents*

            == Patch description ==

            To close this bug please:

            1) Merge selected parts of debian/* from https://anonscm.debian.org/gitweb/?p=pkg-mysql/mariadb-10.0.git to https://github.com/MariaDB/server via pull requests arriving from branch https://github.com/ottok/mariadb

            2) Backport form MariaDB development head to 10.1, 10.0 and maybe even 5.5 commits that fix individual issues.

            == Quality assurance ==

            The new debian/* contents represents the state of the art of Debian packaging. The mariadb-5.5 and mariadb-10.0 has passed all Debian quality assurance and landed in the official Debian repositories. More details at https://wiki.debian.org/Teams/MySQL/MariaDB

            These same packages have also gone downstream to Ubuntu, has passed Ubuntu QA and are included since the Ubuntu 14.04 release.

            The packaging source is mirrored at https://github.com/ottok/mariadb-5.5 which is build and successfully passes MariaDB.org Buildbot:
            http://buildbot.askmonty.org/buildbot/builders/debpkg-trusty
            http://buildbot.askmonty.org/buildbot/builders/debpkg-sid

            == Roll-out planning ==

            This merge should be safe to release with 10.x. All new installs and upgrades using the MariaDB.org repo or mixing it with official Debian or Ubuntu repository contents is expected to work.

            Below are the different scenarios how releasing the new packging at MariaDB.org will affect current installs/upgrades:

            A) If users currently have a MariaDB.org repo enabled, those packages will have a + in their version number. Debian package manager will always consider e.g. 5.5.38+maria as more recent than 5.5.38 from the native repositories. Therefore people having MariaDB.org repositories will always get primarly the packages from MariaDB.org repositories. It is very unlikely that a Debian repository would ever have a more recent release than in MariaDB.org (e.g. 5.5.39 would override 5.5.38+maria, but that is not likely to take place as MariaDB.org always releases first).

            B) The new packaging is designed and tested that it allows seamless upgrades from old deb packaging generation to new generation. Whether the new packaging is from MariaDB.org or distro repositories does not matter.

            C) Backwards works only partially, but is a rare case (e.g. current Ubuntu users trying to upgrade from new style debs 5.5.38 to MariaDB.org repo old style debs 5.5.38 will see issues, e.g. MDEV-5977). As soon as this merge is done and MariaDB.org publishes packages using the new packaging this corner case is fixed.

            D) If users remove the MariaDB.org from their install, they will automatically have their 5.5.38+maria updated to Debian/Ubuntu native 5.5.39 once it is released in Debian.

            E) Like previously, upgrading from MySQL 5.5 to MariaDB 5.5 works seamlessly and backwards also. Updating from MariaDB 5.5 to 10.0 works seamlessly but backwards will have issues (downgrade flag, InnoDB log size mismatch etc) the package manager will not solve automatically. If users dump and import data manually, and in between completely purge and install the packages, then users can always migrate to whatever version.
            otto Otto Kekäläinen added a comment - serg Note that current https://github.com/ottok/mariadb-10.0/compare/upstream...master.patch is 45488 lines long..
            otto Otto Kekäläinen made changes -
            Description == Motivation ==

            Merge downstream .deb-packaging into upstream MariaDB to unify, streamline and ease future release packaging. This will also fix some packaging errors in current MariaDB.org deb-packages.

            The patch renews most of debian/* contents with new packaging that follows latest Debian policies, uses new dh rules file, new patch format, etc. Old autobake.sh is still there and some other minor cruft that might need to be processed later (e.g. the contents of dist/).

            The upstream MariaDB contents of debian/* will not be fully identical to the downstream Debian/Ubuntu one. Upstream MariaDB.org deb's will differ by *not* including the libmariadbclient.so rename and it *will* include the oqgraph module which was dropped downstream due to build dependency issues.

            *TODO: write full list of intended differences between Debian master and MariaDB master regarding debian/* contents*

            == Patch description ==

            To close this bug please:

            1) Merge selected parts of debian/* from https://anonscm.debian.org/gitweb/?p=pkg-mysql/mariadb-10.0.git to https://github.com/MariaDB/server via pull requests arriving from branch https://github.com/ottok/mariadb

            2) Backport form MariaDB development head to 10.1, 10.0 and maybe even 5.5 commits that fix individual issues.

            == Quality assurance ==

            The new debian/* contents represents the state of the art of Debian packaging. The mariadb-5.5 and mariadb-10.0 has passed all Debian quality assurance and landed in the official Debian repositories. More details at https://wiki.debian.org/Teams/MySQL/MariaDB

            These same packages have also gone downstream to Ubuntu, has passed Ubuntu QA and are included since the Ubuntu 14.04 release.

            The packaging source is mirrored at https://github.com/ottok/mariadb-5.5 which is build and successfully passes MariaDB.org Buildbot:
            http://buildbot.askmonty.org/buildbot/builders/debpkg-trusty
            http://buildbot.askmonty.org/buildbot/builders/debpkg-sid

            == Roll-out planning ==

            This merge should be safe to release with 10.x. All new installs and upgrades using the MariaDB.org repo or mixing it with official Debian or Ubuntu repository contents is expected to work.

            Below are the different scenarios how releasing the new packging at MariaDB.org will affect current installs/upgrades:

            A) If users currently have a MariaDB.org repo enabled, those packages will have a + in their version number. Debian package manager will always consider e.g. 5.5.38+maria as more recent than 5.5.38 from the native repositories. Therefore people having MariaDB.org repositories will always get primarly the packages from MariaDB.org repositories. It is very unlikely that a Debian repository would ever have a more recent release than in MariaDB.org (e.g. 5.5.39 would override 5.5.38+maria, but that is not likely to take place as MariaDB.org always releases first).

            B) The new packaging is designed and tested that it allows seamless upgrades from old deb packaging generation to new generation. Whether the new packaging is from MariaDB.org or distro repositories does not matter.

            C) Backwards works only partially, but is a rare case (e.g. current Ubuntu users trying to upgrade from new style debs 5.5.38 to MariaDB.org repo old style debs 5.5.38 will see issues, e.g. MDEV-5977). As soon as this merge is done and MariaDB.org publishes packages using the new packaging this corner case is fixed.

            D) If users remove the MariaDB.org from their install, they will automatically have their 5.5.38+maria updated to Debian/Ubuntu native 5.5.39 once it is released in Debian.

            E) Like previously, upgrading from MySQL 5.5 to MariaDB 5.5 works seamlessly and backwards also. Updating from MariaDB 5.5 to 10.0 works seamlessly but backwards will have issues (downgrade flag, InnoDB log size mismatch etc) the package manager will not solve automatically. If users dump and import data manually, and in between completely purge and install the packages, then users can always migrate to whatever version.
            == Motivation ==

            Merge downstream .deb-packaging into upstream MariaDB to unify, streamline and ease future release packaging. This will also fix some packaging errors in current MariaDB.org deb-packages.

            == Patch description ==

            List of notable changes:
            * Completely new Debian packaging standard (3.0 quilt) including rules file, patch conventions and .files -> .install file listing conventions
            * The will no longer be a debian/dist subfolder and the packaging of Debian an Ubuntu will be unified (= Debian will ship with AppArmor files included)
            * Support for Debian releases before Wheezy (7) and Ubuntu releases before Precise (12.04) will be dropped

            The upstream MariaDB contents of debian/* will not be fully identical to the downstream Debian/Ubuntu one.

            List of intended differences between Debian master and MariaDB master regarding debian/ contents:
            * control file: maintainer, uploaders, VCS and browser links
            * omitted packages (not allowed in Debian): libmariadbclient18, libmysqlclient18, libmariadbclient-dev, mysql-common
            * omitted plugins (problematic in Debian): Handler socket, TokuDB, Mroonga, Cassandra
            * keep autobake.sh in mariadb.org repo

            To close this bug please:

            1) Merge selected parts of debian/* from https://anonscm.debian.org/gitweb/?p=pkg-mysql/mariadb-10.0.git to https://github.com/MariaDB/server via pull requests arriving from branch ok-debpkg at https://github.com/ottok/mariadb

            2) Backport form MariaDB development head to 10.1, 10.0 and maybe even 5.5 commits that fix individual issues.

            == Quality assurance ==

            The new debian/* contents represents the state of the art of Debian packaging. The mariadb-5.5 and mariadb-10.0 has passed all Debian quality assurance and landed in the official Debian repositories. More details at https://wiki.debian.org/Teams/MySQL/MariaDB

            These same packages have also gone downstream to Ubuntu, has passed Ubuntu QA and are included since the Ubuntu 14.04 release.

            Buildbot test for ok-debpkg branch: https://buildbot.askmonty.org/buildbot/grid?branch=ok-debpkg&category=main

            == Roll-out planning ==

            This merge should be safe to release with 10.1. All new installs and upgrades using the MariaDB.org repo or mixing it with official Debian or Ubuntu repository contents is expected to work.

            Below are the different scenarios how releasing the new packging at MariaDB.org will affect current installs/upgrades:

            A) If users currently have a MariaDB.org repo enabled, those packages will have a + in their version number. Debian package manager will always consider e.g. 10.1.6+maria as more recent than 10.1.6 from the native repositories. Therefore people having MariaDB.org repositories will always get primarly the packages from MariaDB.org repositories. It is very unlikely that a Debian repository would ever have a more recent release than in MariaDB.org (e.g. 10.1.7 would override 10.1.6+maria, but that is not likely to take place as MariaDB.org always releases first).

            B) The new packaging is designed and tested that it allows seamless upgrades from old deb packaging generation to new generation. Whether the new packaging is from MariaDB.org or distro repositories does not matter.

            C) Backwards works only partially, but is a rare case (e.g. current Ubuntu users trying to upgrade from new style debs 10.0 to MariaDB.org repo old style debs 10.0 will see issues, e.g. MDEV-5977). As soon as this merge is done and MariaDB.org publishes packages using the new packaging this corner case is fixed.

            D) If users remove the MariaDB.org from their install, they will automatically have their 10.1.6+maria updated to Debian/Ubuntu native 5.5.39 once it is released in Debian.

            E) Like previously, upgrading from MySQL 5.5 to MariaDB 5.5 works seamlessly and backwards also. Updating from MariaDB 5.5 to 10.0 works seamlessly but backwards will have issues (downgrade flag, InnoDB log size mismatch etc) the package manager will not solve automatically. If users dump and import data manually, and in between completely purge and install the packages, then users can always migrate to whatever version.
            serg Sergei Golubchik made changes -
            elenst Elena Stepanova made changes -
            Issue Type Bug [ 1 ] Task [ 3 ]
            otto Otto Kekäläinen made changes -
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            Attachment ottok.diff [ 38904 ]

            Attached a complete review of the above: ottok.diff

            serg Sergei Golubchik added a comment - Attached a complete review of the above: ottok.diff
            otto Otto Kekäläinen made changes -

            Recent changes done to downstream packaging based on Serg's review:

            commit ada2c8c84271283eb0f604820d5994c9a057784e
            Author: Otto Kekäläinen <otto@seravo.fi>
            Date:   Tue Jul 21 22:40:16 2015 +0300
             
                Recover mysql-upgrade dir/link handlig wrongly removed in f7caa041db
                
                The preinst is supposed to save possible mysql datadir link
                and the postinst is supposed to recover it.
                
                Also unify namig of the variables that reference the data
                and log directories.
             
            commit d190985e7ba7621c99f877a9451943c6e2ae5d54
            Author: Otto Kekäläinen <otto@seravo.fi>
            Date:   Tue Jul 21 22:02:54 2015 +0300
             
                mysqld config: rename key_buffer -> key_buffer_size
                
                Short name version is deprecated. Use full name instead.
                https://mariadb.com/kb/en/mariadb/myisam-system-variables/#key_buffer_size
             
            commit d8fb0f21e7e835336b37d9d3533d7276fe7a99b3
            Author: Otto Kekäläinen <otto@seravo.fi>
            Date:   Tue Jul 21 21:12:18 2015 +0300
             
                Clean up old cruft from rules file after review by Sergei Golubchik
                
                 * Don't set or use MYSQL_BUILD_CC as they were hard-coded anyway
                 * Remove unnecessary comment about how make translates -j option to --jobserver
                 * Assembler functions are long gone and references to them should be removed
                 * C and C++ should have same optimization levels, that is -O3
                 * BIG-JOINS are not used anywhere, remove them
                 * Make compilation comment nicer (visible via "show variables like 'version%';")
                 * Make -DDEB contain actual distro name, might be of later use and anyway better
                   than just simple '1' (true) value
                 * Not needed to have TAOCRYPT_OPT in CFLAGS, taocrypt is C++
                 * Libwrap is disabled by default
                 * INFO_BIN and INFO_SRC files are not used anymore, no need to install them.
                 * Use INSTALL_MYSQLTESTDIR instead of manual mv line
             
            commit a9bbb0c5dac55aefcad7131dc29a8b2a809cc3ed
            Author: Otto Kekäläinen <otto@seravo.fi>
            Date:   Tue Jul 21 18:23:25 2015 +0300
             
                Unified config file layout with upstream .cnf layout
                
                Now the packages mariadb-common, mariadb-client and mariadb-server each
                install their own cnf file. A system will not have extra cnf files around.
                
                Also the .cnf files list all possible stanzas that can be used to
                compartmentalize settings for different binaries.
                
                Role model was upstream
                https://github.com/MariaDB/server/tree/83ba48b7c670f6dba465325cafd808c91f551544/support-files/rpm
             
            commit 59c96c74e098767b2e01c0fa6b2e70b8bce592cc
            Author: Otto Kekäläinen <otto@seravo.fi>
            Date:   Tue Jul 21 18:00:27 2015 +0300
             
                Simplyfy overkill utf8mb4 settings
                
                From https://dev.mysql.com/doc/refman/5.6/en/charset-connection.html
                
                  A SET NAMES 'charset_name' statement is equivalent to these three statements:
                
                  SET character_set_client = charset_name;
                  SET character_set_results = charset_name;
                  SET character_set_connection = charset_name;
                
                Therefore it is probably overkill to have 'SET NAMES'.
                
                Also using both - and _ in variable names in useless as they translate
                to the same variable.
             
            commit 2f71f6834d93d9591cc187d6cce6e3dd7268bd33
            Author: Otto Kekäläinen <otto@seravo.fi>
            Date:   Tue Jul 21 17:46:24 2015 +0300
             
                Update upstream info in d/copyright
             
            commit df02e679e29dabd1926f477c70d984cc168fc159
            Author: Otto Kekäläinen <otto@seravo.fi>
            Date:   Tue Jul 21 17:27:29 2015 +0300
             
                The test suite must depend on the exact MariaDB version as tests are version specific.
                
                Depending on latest or any newer version is wrong, as tests might not be compatible
                with some never program versions.
             
            commit e3fe9539f010270aac5aa6e70a2c0e561c749df0
            Author: Otto Kekäläinen <otto@seravo.fi>
            Date:   Tue Jul 21 17:16:11 2015 +0300
             
                Wrap and sort
             
            commit ed276550168e9933f9e804bcf5f10da0cf4fe648
            Author: Otto Kekäläinen <otto@seravo.fi>
            Date:   Tue Jul 21 16:24:30 2015 +0300
             
                Created package libmariadbd and ship libmysqld.so.18 in it.
             
            commit 0ed651b0cd8bafed1d43be339d499cc0214e54e5
            Author: Otto Kekäläinen <otto@seravo.fi>
            Date:   Tue Jul 21 15:39:47 2015 +0300
             
                Add libdbd-mysql-perl as Depends instead of Suggests as libdbi-perl needs it.
                
                The package libdbi-perl is a dependency and it is useless unless libdbd-mysql-perl
                is also installed. Thus both should be in Depends. These lines are now also
                identical to those in the mysql-5.6 control file.
             
            commit be79bfaf31b9839205cb34f6bb1244c2f9b36ef5
            Author: Otto Kekäläinen <otto@seravo.fi>
            Date:   Tue Jul 21 08:39:45 2015 +0300
             
                Updated d/changelog
             

            (full log at https://github.com/ottok/mariadb-10.0/commits/master)

            Preliminary branch for upstream merge is at https://github.com/MariaDB/server/compare/10.1...ottok:ok-debpkg

            otto Otto Kekäläinen added a comment - Recent changes done to downstream packaging based on Serg's review: commit ada2c8c84271283eb0f604820d5994c9a057784e Author: Otto Kekäläinen <otto@seravo.fi> Date: Tue Jul 21 22:40:16 2015 +0300   Recover mysql-upgrade dir/link handlig wrongly removed in f7caa041db The preinst is supposed to save possible mysql datadir link and the postinst is supposed to recover it. Also unify namig of the variables that reference the data and log directories.   commit d190985e7ba7621c99f877a9451943c6e2ae5d54 Author: Otto Kekäläinen <otto@seravo.fi> Date: Tue Jul 21 22:02:54 2015 +0300   mysqld config: rename key_buffer -> key_buffer_size Short name version is deprecated. Use full name instead. https://mariadb.com/kb/en/mariadb/myisam-system-variables/#key_buffer_size   commit d8fb0f21e7e835336b37d9d3533d7276fe7a99b3 Author: Otto Kekäläinen <otto@seravo.fi> Date: Tue Jul 21 21:12:18 2015 +0300   Clean up old cruft from rules file after review by Sergei Golubchik * Don't set or use MYSQL_BUILD_CC as they were hard-coded anyway * Remove unnecessary comment about how make translates -j option to --jobserver * Assembler functions are long gone and references to them should be removed * C and C++ should have same optimization levels, that is -O3 * BIG-JOINS are not used anywhere, remove them * Make compilation comment nicer (visible via "show variables like 'version%';") * Make -DDEB contain actual distro name, might be of later use and anyway better than just simple '1' (true) value * Not needed to have TAOCRYPT_OPT in CFLAGS, taocrypt is C++ * Libwrap is disabled by default * INFO_BIN and INFO_SRC files are not used anymore, no need to install them. * Use INSTALL_MYSQLTESTDIR instead of manual mv line   commit a9bbb0c5dac55aefcad7131dc29a8b2a809cc3ed Author: Otto Kekäläinen <otto@seravo.fi> Date: Tue Jul 21 18:23:25 2015 +0300   Unified config file layout with upstream .cnf layout Now the packages mariadb-common, mariadb-client and mariadb-server each install their own cnf file. A system will not have extra cnf files around. Also the .cnf files list all possible stanzas that can be used to compartmentalize settings for different binaries. Role model was upstream https://github.com/MariaDB/server/tree/83ba48b7c670f6dba465325cafd808c91f551544/support-files/rpm   commit 59c96c74e098767b2e01c0fa6b2e70b8bce592cc Author: Otto Kekäläinen <otto@seravo.fi> Date: Tue Jul 21 18:00:27 2015 +0300   Simplyfy overkill utf8mb4 settings From https://dev.mysql.com/doc/refman/5.6/en/charset-connection.html A SET NAMES 'charset_name' statement is equivalent to these three statements: SET character_set_client = charset_name; SET character_set_results = charset_name; SET character_set_connection = charset_name; Therefore it is probably overkill to have 'SET NAMES'. Also using both - and _ in variable names in useless as they translate to the same variable.   commit 2f71f6834d93d9591cc187d6cce6e3dd7268bd33 Author: Otto Kekäläinen <otto@seravo.fi> Date: Tue Jul 21 17:46:24 2015 +0300   Update upstream info in d/copyright   commit df02e679e29dabd1926f477c70d984cc168fc159 Author: Otto Kekäläinen <otto@seravo.fi> Date: Tue Jul 21 17:27:29 2015 +0300   The test suite must depend on the exact MariaDB version as tests are version specific. Depending on latest or any newer version is wrong, as tests might not be compatible with some never program versions.   commit e3fe9539f010270aac5aa6e70a2c0e561c749df0 Author: Otto Kekäläinen <otto@seravo.fi> Date: Tue Jul 21 17:16:11 2015 +0300   Wrap and sort   commit ed276550168e9933f9e804bcf5f10da0cf4fe648 Author: Otto Kekäläinen <otto@seravo.fi> Date: Tue Jul 21 16:24:30 2015 +0300   Created package libmariadbd and ship libmysqld.so.18 in it.   commit 0ed651b0cd8bafed1d43be339d499cc0214e54e5 Author: Otto Kekäläinen <otto@seravo.fi> Date: Tue Jul 21 15:39:47 2015 +0300   Add libdbd-mysql-perl as Depends instead of Suggests as libdbi-perl needs it. The package libdbi-perl is a dependency and it is useless unless libdbd-mysql-perl is also installed. Thus both should be in Depends. These lines are now also identical to those in the mysql-5.6 control file.   commit be79bfaf31b9839205cb34f6bb1244c2f9b36ef5 Author: Otto Kekäläinen <otto@seravo.fi> Date: Tue Jul 21 08:39:45 2015 +0300   Updated d/changelog   (full log at https://github.com/ottok/mariadb-10.0/commits/master ) Preliminary branch for upstream merge is at https://github.com/MariaDB/server/compare/10.1...ottok:ok-debpkg
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            otto Otto Kekäläinen made changes -
            otto Otto Kekäläinen made changes -
            otto Otto Kekäläinen added a comment - - edited

            serg Pull request filed at https://github.com/MariaDB/server/pull/97

            It fixes at least MDEV-8667 and MDEV-6326 right now, and the new control file will prevent lots of future problems.

            As the control file logic is now almost in sync with that of the official Debian version, this pull request will prevent problems people might have who try to upgrade from 10.0 in Debian/Ubuntu to 10.1 from mariadb.org repos.

            Once this PR is accepted, I'll continue on the merge for other parts.

            otto Otto Kekäläinen added a comment - - edited serg Pull request filed at https://github.com/MariaDB/server/pull/97 It fixes at least MDEV-8667 and MDEV-6326 right now, and the new control file will prevent lots of future problems. As the control file logic is now almost in sync with that of the official Debian version, this pull request will prevent problems people might have who try to upgrade from 10.0 in Debian/Ubuntu to 10.1 from mariadb.org repos. Once this PR is accepted, I'll continue on the merge for other parts.
            serg Sergei Golubchik made changes -
            Assignee Otto Kekäläinen [ otto ] Sergei Golubchik [ serg ]
            serg Sergei Golubchik made changes -
            Status In Progress [ 3 ] In Review [ 10002 ]
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            Fix Version/s 10.2 [ 14601 ]
            serg Sergei Golubchik made changes -
            Priority Major [ 3 ] Critical [ 2 ]

            merged

            serg Sergei Golubchik added a comment - merged
            serg Sergei Golubchik made changes -
            Assignee Sergei Golubchik [ serg ] Otto Kekäläinen [ otto ]
            Status In Review [ 10002 ] Stalled [ 10000 ]

            sergratzpo I think I should continue to do another round of backporting fixes to 10.1 to close multiple issues linked above so that 10.1 would ship with less broken packaging. However the current 10.1 master head fails in multiple tests and I don't have a clean starting point: https://buildbot.askmonty.org/buildbot/grid?branch=ok-debpkg-10.1&category=main

            None of those fails are due to Debian packaging. I'll wait until 10.1 master is all green before I start working on my branch.

            otto Otto Kekäläinen added a comment - serg ratzpo I think I should continue to do another round of backporting fixes to 10.1 to close multiple issues linked above so that 10.1 would ship with less broken packaging. However the current 10.1 master head fails in multiple tests and I don't have a clean starting point: https://buildbot.askmonty.org/buildbot/grid?branch=ok-debpkg-10.1&category=main None of those fails are due to Debian packaging. I'll wait until 10.1 master is all green before I start working on my branch.
            danblack Daniel Black made changes -
            otto Otto Kekäläinen made changes -
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.1 [ 16100 ]
            Labels debian packaging debian foundation packaging
            otto Otto Kekäläinen made changes -
            Status Stalled [ 10000 ] In Progress [ 3 ]
            danblack Daniel Black added a comment - https://buildbot.askmonty.org/buildbot/grid?branch=ok-debpkg-10.1&category=main green again.
            ratzpo Rasmus Johansson (Inactive) made changes -
            Sprint 10.2.2-2 [ 92 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Rank Ranked lower
            ratzpo Rasmus Johansson (Inactive) made changes -
            Sprint 10.2.2-2 [ 92 ] 10.2.2-2, 10.2.2-3 [ 92, 94 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Sprint 10.2.2-2, 10.2.2-3 [ 92, 94 ] 10.2.2-2 [ 92 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Rank Ranked higher

            I've been working on the next step for 5 days now, should soon be done. WIP at https://github.com/ottok/mariadb/commits/ok-debpkg-10.2

            otto Otto Kekäläinen added a comment - I've been working on the next step for 5 days now, should soon be done. WIP at https://github.com/ottok/mariadb/commits/ok-debpkg-10.2

            PR available at https://github.com/MariaDB/server/pull/251

            There is still quite a lot to do to close this issue and all the issues linked to it, but this is a big milestone in that process.

            otto Otto Kekäläinen added a comment - PR available at https://github.com/MariaDB/server/pull/251 There is still quite a lot to do to close this issue and all the issues linked to it, but this is a big milestone in that process.
            elenst Elena Stepanova made changes -
            Labels debian foundation packaging 10.2-ga debian foundation packaging

            Changing fixVersion. Everything that was supposed to be done for 10.2 has been done. The rest will go into 10.3.

            ratzpo Rasmus Johansson (Inactive) added a comment - Changing fixVersion. Everything that was supposed to be done for 10.2 has been done. The rest will go into 10.3.
            ratzpo Rasmus Johansson (Inactive) made changes -
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s 10.2 [ 14601 ]
            ratzpo Rasmus Johansson (Inactive) made changes -
            Labels 10.2-ga debian foundation packaging debian foundation packaging
            serg Sergei Golubchik made changes -
            Priority Critical [ 2 ] Major [ 3 ]

            Work continues now on 10.3 branch by me and oerdnj

            otto Otto Kekäläinen added a comment - Work continues now on 10.3 branch by me and oerdnj
            danblack Daniel Black made changes -
            danblack Daniel Black made changes -
            serg Sergei Golubchik made changes -
            Fix Version/s 10.4 [ 22408 ]
            julien.fritsch Julien Fritsch made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            julien.fritsch Julien Fritsch made changes -
            Description == Motivation ==

            Merge downstream .deb-packaging into upstream MariaDB to unify, streamline and ease future release packaging. This will also fix some packaging errors in current MariaDB.org deb-packages.

            == Patch description ==

            List of notable changes:
            * Completely new Debian packaging standard (3.0 quilt) including rules file, patch conventions and .files -> .install file listing conventions
            * The will no longer be a debian/dist subfolder and the packaging of Debian an Ubuntu will be unified (= Debian will ship with AppArmor files included)
            * Support for Debian releases before Wheezy (7) and Ubuntu releases before Precise (12.04) will be dropped

            The upstream MariaDB contents of debian/* will not be fully identical to the downstream Debian/Ubuntu one.

            List of intended differences between Debian master and MariaDB master regarding debian/ contents:
            * control file: maintainer, uploaders, VCS and browser links
            * omitted packages (not allowed in Debian): libmariadbclient18, libmysqlclient18, libmariadbclient-dev, mysql-common
            * omitted plugins (problematic in Debian): Handler socket, TokuDB, Mroonga, Cassandra
            * keep autobake.sh in mariadb.org repo

            To close this bug please:

            1) Merge selected parts of debian/* from https://anonscm.debian.org/gitweb/?p=pkg-mysql/mariadb-10.0.git to https://github.com/MariaDB/server via pull requests arriving from branch ok-debpkg at https://github.com/ottok/mariadb

            2) Backport form MariaDB development head to 10.1, 10.0 and maybe even 5.5 commits that fix individual issues.

            == Quality assurance ==

            The new debian/* contents represents the state of the art of Debian packaging. The mariadb-5.5 and mariadb-10.0 has passed all Debian quality assurance and landed in the official Debian repositories. More details at https://wiki.debian.org/Teams/MySQL/MariaDB

            These same packages have also gone downstream to Ubuntu, has passed Ubuntu QA and are included since the Ubuntu 14.04 release.

            Buildbot test for ok-debpkg branch: https://buildbot.askmonty.org/buildbot/grid?branch=ok-debpkg&category=main

            == Roll-out planning ==

            This merge should be safe to release with 10.1. All new installs and upgrades using the MariaDB.org repo or mixing it with official Debian or Ubuntu repository contents is expected to work.

            Below are the different scenarios how releasing the new packging at MariaDB.org will affect current installs/upgrades:

            A) If users currently have a MariaDB.org repo enabled, those packages will have a + in their version number. Debian package manager will always consider e.g. 10.1.6+maria as more recent than 10.1.6 from the native repositories. Therefore people having MariaDB.org repositories will always get primarly the packages from MariaDB.org repositories. It is very unlikely that a Debian repository would ever have a more recent release than in MariaDB.org (e.g. 10.1.7 would override 10.1.6+maria, but that is not likely to take place as MariaDB.org always releases first).

            B) The new packaging is designed and tested that it allows seamless upgrades from old deb packaging generation to new generation. Whether the new packaging is from MariaDB.org or distro repositories does not matter.

            C) Backwards works only partially, but is a rare case (e.g. current Ubuntu users trying to upgrade from new style debs 10.0 to MariaDB.org repo old style debs 10.0 will see issues, e.g. MDEV-5977). As soon as this merge is done and MariaDB.org publishes packages using the new packaging this corner case is fixed.

            D) If users remove the MariaDB.org from their install, they will automatically have their 10.1.6+maria updated to Debian/Ubuntu native 5.5.39 once it is released in Debian.

            E) Like previously, upgrading from MySQL 5.5 to MariaDB 5.5 works seamlessly and backwards also. Updating from MariaDB 5.5 to 10.0 works seamlessly but backwards will have issues (downgrade flag, InnoDB log size mismatch etc) the package manager will not solve automatically. If users dump and import data manually, and in between completely purge and install the packages, then users can always migrate to whatever version.
            .c== Motivation ==

            Merge downstream .deb-packaging into upstream MariaDB to unify, streamline and ease future release packaging. This will also fix some packaging errors in current MariaDB.org deb-packages.

            == Patch description ==

            List of notable changes:
            * Completely new Debian packaging standard (3.0 quilt) including rules file, patch conventions and .files -> .install file listing conventions
            * The will no longer be a debian/dist subfolder and the packaging of Debian an Ubuntu will be unified (= Debian will ship with AppArmor files included)
            * Support for Debian releases before Wheezy (7) and Ubuntu releases before Precise (12.04) will be dropped

            The upstream MariaDB contents of debian/* will not be fully identical to the downstream Debian/Ubuntu one.

            List of intended differences between Debian master and MariaDB master regarding debian/ contents:
            * control file: maintainer, uploaders, VCS and browser links
            * omitted packages (not allowed in Debian): libmariadbclient18, libmysqlclient18, libmariadbclient-dev, mysql-common
            * omitted plugins (problematic in Debian): Handler socket, TokuDB, Mroonga, Cassandra
            * keep autobake.sh in mariadb.org repo

            To close this bug please:

            1) Merge selected parts of debian/* from https://anonscm.debian.org/gitweb/?p=pkg-mysql/mariadb-10.0.git to https://github.com/MariaDB/server via pull requests arriving from branch ok-debpkg at https://github.com/ottok/mariadb

            2) Backport form MariaDB development head to 10.1, 10.0 and maybe even 5.5 commits that fix individual issues.

            == Quality assurance ==

            The new debian/* contents represents the state of the art of Debian packaging. The mariadb-5.5 and mariadb-10.0 has passed all Debian quality assurance and landed in the official Debian repositories. More details at https://wiki.debian.org/Teams/MySQL/MariaDB

            These same packages have also gone downstream to Ubuntu, has passed Ubuntu QA and are included since the Ubuntu 14.04 release.

            Buildbot test for ok-debpkg branch: https://buildbot.askmonty.org/buildbot/grid?branch=ok-debpkg&category=main

            == Roll-out planning ==

            This merge should be safe to release with 10.1. All new installs and upgrades using the MariaDB.org repo or mixing it with official Debian or Ubuntu repository contents is expected to work.

            Below are the different scenarios how releasing the new packging at MariaDB.org will affect current installs/upgrades:

            A) If users currently have a MariaDB.org repo enabled, those packages will have a + in their version number. Debian package manager will always consider e.g. 10.1.6+maria as more recent than 10.1.6 from the native repositories. Therefore people having MariaDB.org repositories will always get primarly the packages from MariaDB.org repositories. It is very unlikely that a Debian repository would ever have a more recent release than in MariaDB.org (e.g. 10.1.7 would override 10.1.6+maria, but that is not likely to take place as MariaDB.org always releases first).

            B) The new packaging is designed and tested that it allows seamless upgrades from old deb packaging generation to new generation. Whether the new packaging is from MariaDB.org or distro repositories does not matter.

            C) Backwards works only partially, but is a rare case (e.g. current Ubuntu users trying to upgrade from new style debs 10.0 to MariaDB.org repo old style debs 10.0 will see issues, e.g. MDEV-5977). As soon as this merge is done and MariaDB.org publishes packages using the new packaging this corner case is fixed.

            D) If users remove the MariaDB.org from their install, they will automatically have their 10.1.6+maria updated to Debian/Ubuntu native 5.5.39 once it is released in Debian.

            E) Like previously, upgrading from MySQL 5.5 to MariaDB 5.5 works seamlessly and backwards also. Updating from MariaDB 5.5 to 10.0 works seamlessly but backwards will have issues (downgrade flag, InnoDB log size mismatch etc) the package manager will not solve automatically. If users dump and import data manually, and in between completely purge and install the packages, then users can always migrate to whatever version.
            julien.fritsch Julien Fritsch made changes -
            Description .c== Motivation ==

            Merge downstream .deb-packaging into upstream MariaDB to unify, streamline and ease future release packaging. This will also fix some packaging errors in current MariaDB.org deb-packages.

            == Patch description ==

            List of notable changes:
            * Completely new Debian packaging standard (3.0 quilt) including rules file, patch conventions and .files -> .install file listing conventions
            * The will no longer be a debian/dist subfolder and the packaging of Debian an Ubuntu will be unified (= Debian will ship with AppArmor files included)
            * Support for Debian releases before Wheezy (7) and Ubuntu releases before Precise (12.04) will be dropped

            The upstream MariaDB contents of debian/* will not be fully identical to the downstream Debian/Ubuntu one.

            List of intended differences between Debian master and MariaDB master regarding debian/ contents:
            * control file: maintainer, uploaders, VCS and browser links
            * omitted packages (not allowed in Debian): libmariadbclient18, libmysqlclient18, libmariadbclient-dev, mysql-common
            * omitted plugins (problematic in Debian): Handler socket, TokuDB, Mroonga, Cassandra
            * keep autobake.sh in mariadb.org repo

            To close this bug please:

            1) Merge selected parts of debian/* from https://anonscm.debian.org/gitweb/?p=pkg-mysql/mariadb-10.0.git to https://github.com/MariaDB/server via pull requests arriving from branch ok-debpkg at https://github.com/ottok/mariadb

            2) Backport form MariaDB development head to 10.1, 10.0 and maybe even 5.5 commits that fix individual issues.

            == Quality assurance ==

            The new debian/* contents represents the state of the art of Debian packaging. The mariadb-5.5 and mariadb-10.0 has passed all Debian quality assurance and landed in the official Debian repositories. More details at https://wiki.debian.org/Teams/MySQL/MariaDB

            These same packages have also gone downstream to Ubuntu, has passed Ubuntu QA and are included since the Ubuntu 14.04 release.

            Buildbot test for ok-debpkg branch: https://buildbot.askmonty.org/buildbot/grid?branch=ok-debpkg&category=main

            == Roll-out planning ==

            This merge should be safe to release with 10.1. All new installs and upgrades using the MariaDB.org repo or mixing it with official Debian or Ubuntu repository contents is expected to work.

            Below are the different scenarios how releasing the new packging at MariaDB.org will affect current installs/upgrades:

            A) If users currently have a MariaDB.org repo enabled, those packages will have a + in their version number. Debian package manager will always consider e.g. 10.1.6+maria as more recent than 10.1.6 from the native repositories. Therefore people having MariaDB.org repositories will always get primarly the packages from MariaDB.org repositories. It is very unlikely that a Debian repository would ever have a more recent release than in MariaDB.org (e.g. 10.1.7 would override 10.1.6+maria, but that is not likely to take place as MariaDB.org always releases first).

            B) The new packaging is designed and tested that it allows seamless upgrades from old deb packaging generation to new generation. Whether the new packaging is from MariaDB.org or distro repositories does not matter.

            C) Backwards works only partially, but is a rare case (e.g. current Ubuntu users trying to upgrade from new style debs 10.0 to MariaDB.org repo old style debs 10.0 will see issues, e.g. MDEV-5977). As soon as this merge is done and MariaDB.org publishes packages using the new packaging this corner case is fixed.

            D) If users remove the MariaDB.org from their install, they will automatically have their 10.1.6+maria updated to Debian/Ubuntu native 5.5.39 once it is released in Debian.

            E) Like previously, upgrading from MySQL 5.5 to MariaDB 5.5 works seamlessly and backwards also. Updating from MariaDB 5.5 to 10.0 works seamlessly but backwards will have issues (downgrade flag, InnoDB log size mismatch etc) the package manager will not solve automatically. If users dump and import data manually, and in between completely purge and install the packages, then users can always migrate to whatever version.
            == Motivation ==

            Merge downstream .deb-packaging into upstream MariaDB to unify, streamline and ease future release packaging. This will also fix some packaging errors in current MariaDB.org deb-packages.

            == Patch description ==

            List of notable changes:
            * Completely new Debian packaging standard (3.0 quilt) including rules file, patch conventions and .files -> .install file listing conventions
            * The will no longer be a debian/dist subfolder and the packaging of Debian an Ubuntu will be unified (= Debian will ship with AppArmor files included)
            * Support for Debian releases before Wheezy (7) and Ubuntu releases before Precise (12.04) will be dropped

            The upstream MariaDB contents of debian/* will not be fully identical to the downstream Debian/Ubuntu one.

            List of intended differences between Debian master and MariaDB master regarding debian/ contents:
            * control file: maintainer, uploaders, VCS and browser links
            * omitted packages (not allowed in Debian): libmariadbclient18, libmysqlclient18, libmariadbclient-dev, mysql-common
            * omitted plugins (problematic in Debian): Handler socket, TokuDB, Mroonga, Cassandra
            * keep autobake.sh in mariadb.org repo

            To close this bug please:

            1) Merge selected parts of debian/* from https://anonscm.debian.org/gitweb/?p=pkg-mysql/mariadb-10.0.git to https://github.com/MariaDB/server via pull requests arriving from branch ok-debpkg at https://github.com/ottok/mariadb

            2) Backport form MariaDB development head to 10.1, 10.0 and maybe even 5.5 commits that fix individual issues.

            == Quality assurance ==

            The new debian/* contents represents the state of the art of Debian packaging. The mariadb-5.5 and mariadb-10.0 has passed all Debian quality assurance and landed in the official Debian repositories. More details at https://wiki.debian.org/Teams/MySQL/MariaDB

            These same packages have also gone downstream to Ubuntu, has passed Ubuntu QA and are included since the Ubuntu 14.04 release.

            Buildbot test for ok-debpkg branch: https://buildbot.askmonty.org/buildbot/grid?branch=ok-debpkg&category=main

            == Roll-out planning ==

            This merge should be safe to release with 10.1. All new installs and upgrades using the MariaDB.org repo or mixing it with official Debian or Ubuntu repository contents is expected to work.

            Below are the different scenarios how releasing the new packging at MariaDB.org will affect current installs/upgrades:

            A) If users currently have a MariaDB.org repo enabled, those packages will have a + in their version number. Debian package manager will always consider e.g. 10.1.6+maria as more recent than 10.1.6 from the native repositories. Therefore people having MariaDB.org repositories will always get primarly the packages from MariaDB.org repositories. It is very unlikely that a Debian repository would ever have a more recent release than in MariaDB.org (e.g. 10.1.7 would override 10.1.6+maria, but that is not likely to take place as MariaDB.org always releases first).

            B) The new packaging is designed and tested that it allows seamless upgrades from old deb packaging generation to new generation. Whether the new packaging is from MariaDB.org or distro repositories does not matter.

            C) Backwards works only partially, but is a rare case (e.g. current Ubuntu users trying to upgrade from new style debs 10.0 to MariaDB.org repo old style debs 10.0 will see issues, e.g. MDEV-5977). As soon as this merge is done and MariaDB.org publishes packages using the new packaging this corner case is fixed.

            D) If users remove the MariaDB.org from their install, they will automatically have their 10.1.6+maria updated to Debian/Ubuntu native 5.5.39 once it is released in Debian.

            E) Like previously, upgrading from MySQL 5.5 to MariaDB 5.5 works seamlessly and backwards also. Updating from MariaDB 5.5 to 10.0 works seamlessly but backwards will have issues (downgrade flag, InnoDB log size mismatch etc) the package manager will not solve automatically. If users dump and import data manually, and in between completely purge and install the packages, then users can always migrate to whatever version.
            otto Otto Kekäläinen made changes -
            otto Otto Kekäläinen made changes -
            otto Otto Kekäläinen made changes -
            otto Otto Kekäläinen made changes -
            otto Otto Kekäläinen made changes -
            otto Otto Kekäläinen made changes -
            otto Otto Kekäläinen made changes -
            julien.fritsch Julien Fritsch made changes -
            Priority Critical [ 2 ] Major [ 3 ]

            otto, may I ask you if it's normal that this issue that is not closed is blocking so many issues that are closed?

            julien.fritsch Julien Fritsch added a comment - otto , may I ask you if it's normal that this issue that is not closed is blocking so many issues that are closed?

            julien.fritsch Yes, this issue is still open. We need to fix e.g. https://jira.mariadb.org/browse/CONC-304 and https://jira.mariadb.org/browse/CONC-456 and then we are closer to closing this one. I expect this issue to be closed during the 10.5 cycle (assuming I get some help with a couple of things that should be changed upstream).

            otto Otto Kekäläinen added a comment - julien.fritsch Yes, this issue is still open. We need to fix e.g. https://jira.mariadb.org/browse/CONC-304 and https://jira.mariadb.org/browse/CONC-456 and then we are closer to closing this one. I expect this issue to be closed during the 10.5 cycle (assuming I get some help with a couple of things that should be changed upstream).

            otto my question is more my question is more: how can these issues that have a link "blocking by" this one, can be closed it this one is not closed already?

            julien.fritsch Julien Fritsch added a comment - otto my question is more my question is more: how can these issues that have a link "blocking by" this one, can be closed it this one is not closed already?

            Issues marked "blocked by" are things that need to be fixed upstream before we can finalize the merge.

            Issues marked "blocks" are bugs that are already fixed in Debian downstream and will automatically be fixed upstream at MariaDB.org when this issue is finalized.

            otto Otto Kekäläinen added a comment - Issues marked "blocked by" are things that need to be fixed upstream before we can finalize the merge. Issues marked "blocks" are bugs that are already fixed in Debian downstream and will automatically be fixed upstream at MariaDB.org when this issue is finalized.

            Hi otto would the "upstream" help then, maybe here to explain that indeed, this issue?

            julien.fritsch Julien Fritsch added a comment - Hi otto would the "upstream" help then, maybe here to explain that indeed, this issue?

            I am both upstream and downstream. Currently things holding back this are my PRs pending to be reviewed/accepted and the issues that are still open that need help from somebody else. Most pressingly https://jira.mariadb.org/browse/CONC-304 and https://jira.mariadb.org/browse/CONC-456 as stated above. I'll try to book 2h of time with Georg to get those CONC done, they are fairly simple path and filename changes, but just need to be done in a way that is backwards compatible.

            otto Otto Kekäläinen added a comment - I am both upstream and downstream. Currently things holding back this are my PRs pending to be reviewed/accepted and the issues that are still open that need help from somebody else. Most pressingly https://jira.mariadb.org/browse/CONC-304 and https://jira.mariadb.org/browse/CONC-456 as stated above. I'll try to book 2h of time with Georg to get those CONC done, they are fairly simple path and filename changes, but just need to be done in a way that is backwards compatible.

            OK, clear thank you

            julien.fritsch Julien Fritsch added a comment - OK, clear thank you

            Side notice: seems mtr depends on patch and libtool-bin, need to add that as a run-time dependency:

            nano +2175 mysql-test-run.pl
             
            sub executable_setup () {
             
              $exe_patch='patch' if `patch -v`;
             
              #
              # Check if libtool is available in this distribution/clone
              # we need it when valgrinding or debugging non installed binary
              # Otherwise valgrind will valgrind the libtool wrapper or bash
              # and gdb will not find the real executable to debug
              #
              if ( -x "../libtool")
              {
                $exe_libtool= "../libtool";
                if ($opt_valgrind or $glob_debugger or $opt_strace)
                {
                  mtr_report("Using \"$exe_libtool\" when running valgrind, strace or debugger");
                }
              }
            
            

            Also, for all tests to run, they should be run as user 'mysql' with access to pam_auth_tool, and again with another user that is not in user.mysql by default. Also some tests only run when mtr is invoked with --big-test, there is IPv6 available, there is a Cassandra back-end running etc. I wonder how big tests should be run to validate that there are no testable regressions in this MDEV once all patches are done (it is going to be huge).

            otto Otto Kekäläinen added a comment - Side notice: seems mtr depends on patch and libtool-bin, need to add that as a run-time dependency: nano +2175 mysql-test-run.pl   sub executable_setup () {   $exe_patch='patch' if `patch -v`;   # # Check if libtool is available in this distribution/clone # we need it when valgrinding or debugging non installed binary # Otherwise valgrind will valgrind the libtool wrapper or bash # and gdb will not find the real executable to debug # if ( -x "../libtool") { $exe_libtool= "../libtool"; if ($opt_valgrind or $glob_debugger or $opt_strace) { mtr_report("Using \"$exe_libtool\" when running valgrind, strace or debugger"); } } Also, for all tests to run, they should be run as user 'mysql' with access to pam_auth_tool, and again with another user that is not in user.mysql by default. Also some tests only run when mtr is invoked with --big-test, there is IPv6 available, there is a Cassandra back-end running etc. I wonder how big tests should be run to validate that there are no testable regressions in this MDEV once all patches are done (it is going to be huge).
            otto Otto Kekäläinen made changes -
            Attachment screenshot-1.png [ 51147 ]

            This MDEV can be closed when all of these are green:

            https://salsa.debian.org/mariadb-team/mariadb-server/pipelines/124140

            otto Otto Kekäläinen added a comment - This MDEV can be closed when all of these are green: https://salsa.debian.org/mariadb-team/mariadb-server/pipelines/124140

            patch can be "suggested" or "recommended", because tests won't fail if it's not present, they'll be automatically skipped

            serg Sergei Golubchik added a comment - patch can be "suggested" or "recommended", because tests won't fail if it's not present, they'll be automatically skipped
            otto Otto Kekäläinen made changes -
            otto Otto Kekäläinen made changes -
            Attachment screenshot-2.png [ 51814 ]
            otto Otto Kekäläinen added a comment - - edited

            > Side notice: seems mtr depends on patch and libtool-bin, need to add that as a run-time dependency:

            Tested libtool-bin, but it did not provide any extended traces, on the contrary the trace lost the filename paths:

            Tested by running mtr (without --force and without --parallel) and then issuing in another windows `killall -s 6 mariadbd` comparing baseline and situation after running `apt install libtool-bin`.

            `patch` is already a suggests, no need to research it.

            otto Otto Kekäläinen added a comment - - edited > Side notice: seems mtr depends on patch and libtool-bin, need to add that as a run-time dependency: Tested libtool-bin, but it did not provide any extended traces, on the contrary the trace lost the filename paths: Tested by running mtr (without --force and without --parallel) and then issuing in another windows `killall -s 6 mariadbd` comparing baseline and situation after running `apt install libtool-bin`. `patch` is already a suggests, no need to research it.
            otto Otto Kekäläinen made changes -
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s 10.4 [ 22408 ]

            All but https://jira.mariadb.org/browse/MDEV-5484 and merging of patches (https://jira.mariadb.org/browse/MDEV-14900) is done, but I deem this issue now closed enough.

            otto Otto Kekäläinen added a comment - All but https://jira.mariadb.org/browse/MDEV-5484 and merging of patches ( https://jira.mariadb.org/browse/MDEV-14900 ) is done, but I deem this issue now closed enough.
            otto Otto Kekäläinen made changes -
            Fix Version/s 10.5.4 [ 24264 ]
            Fix Version/s 10.5 [ 23123 ]
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Closed [ 6 ]
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 64909 ] MariaDB v4 [ 132346 ]

            People

              otto Otto Kekäläinen
              otto Otto Kekäläinen
              Votes:
              5 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.