Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-16828

dbgsym / debuginfo packages missing from mirrors for Ubuntu artful/bionic

Details

    Attachments

      Issue Links

        Activity

          The tool we use for creating our repositories, reprepro, doesn't support .ddeb packages. See: https://bugs.launchpad.net/ubuntu/+source/reprepro/+bug/799889

          It's been a while since I last looked into it (around the time of the 17.10 "artful" release), and the most recent message on the bug report says that there is a patch in the master branch to add support for .ddeb packages. So it's probably time to revist this and manually compile the latest source and try it out.

          dbart Daniel Bartholomew added a comment - The tool we use for creating our repositories, reprepro, doesn't support .ddeb packages. See: https://bugs.launchpad.net/ubuntu/+source/reprepro/+bug/799889 It's been a while since I last looked into it (around the time of the 17.10 "artful" release), and the most recent message on the bug report says that there is a patch in the master branch to add support for .ddeb packages. So it's probably time to revist this and manually compile the latest source and try it out.
          danblack Daniel Black added a comment -

          Thanks dbart

          danblack Daniel Black added a comment - Thanks dbart

          My tests with a locally compiled version of reprepro patched for multiversion and ddeb support have not yielded anything we can use. The binary throws all sorts of errors and refuses to include new packages.

          I will try again later, but for now we're still unable to have multi-version repositories for Debian/Ubuntu.

          dbart Daniel Bartholomew added a comment - My tests with a locally compiled version of reprepro patched for multiversion and ddeb support have not yielded anything we can use. The binary throws all sorts of errors and refuses to include new packages. I will try again later, but for now we're still unable to have multi-version repositories for Debian/Ubuntu.

          If we are unable to get a working version of `reprepro` then we need to investigate using a different tool to create/update our Debian/Ubuntu repositories.

          dbart Daniel Bartholomew added a comment - If we are unable to get a working version of `reprepro` then we need to investigate using a different tool to create/update our Debian/Ubuntu repositories.

          freight and aptly are two alternatives. It doesn't look like freight has been updated in a while, but maybe it just doesn't need to be, but it's also not in the Ubuntu repositories. Aptly seems better in both of those two areas. I'll give both a shot though, and I'm open to other suggestions.

          The main criteria for any reprepro replacement are:

          1. able to integrate the solution into the release process
          2. able to include multiple versions
          3. able to include .ddeb files
          4. won't impact users (ideally users should not even know that how we create/update our repositories has changed)
          dbart Daniel Bartholomew added a comment - freight and aptly are two alternatives. It doesn't look like freight has been updated in a while, but maybe it just doesn't need to be, but it's also not in the Ubuntu repositories. Aptly seems better in both of those two areas. I'll give both a shot though, and I'm open to other suggestions. The main criteria for any reprepro replacement are: able to integrate the solution into the release process able to include multiple versions able to include .ddeb files won't impact users (ideally users should not even know that how we create/update our repositories has changed)
          danblack Daniel Black added a comment -

          The only other things I can think of are cloud services like OBS and https://packagecloud.io/ and I don't know how well either will support requirement 4.

          aptly doesn't seem too bad, seems to support multiple version and dbgsym packages (https://github.com/aptly-dev/aptly/pull/499) and ddebs (https://github.com/aptly-dev/aptly/commit/b877e06a02e96d3cdcb9728f9e587d4132a76b79).

          Hope it goes well. Thanks for looking into it.

          danblack Daniel Black added a comment - The only other things I can think of are cloud services like OBS and https://packagecloud.io/ and I don't know how well either will support requirement 4. aptly doesn't seem too bad, seems to support multiple version and dbgsym packages ( https://github.com/aptly-dev/aptly/pull/499 ) and ddebs ( https://github.com/aptly-dev/aptly/commit/b877e06a02e96d3cdcb9728f9e587d4132a76b79 ). Hope it goes well. Thanks for looking into it.
          danblack Daniel Black added a comment -

          I see Debian(buster,sid,stretch) and Ubuntu(xenial) on mirrors.

          Lack of full distro support for Debian/Ubuntu is a notable gap in GeoffMontee wonderfully updated:
          https://mariadb.com/kb/en/library/how-to-produce-a-full-stack-trace-for-mysqld/#installing-debug-info-packages-on-linux

          danblack Daniel Black added a comment - I see Debian(buster,sid,stretch) and Ubuntu(xenial) on mirrors. Lack of full distro support for Debian/Ubuntu is a notable gap in GeoffMontee wonderfully updated: https://mariadb.com/kb/en/library/how-to-produce-a-full-stack-trace-for-mysqld/#installing-debug-info-packages-on-linux
          danblack Daniel Black added a comment -

          dbart, noticed you have amd64 debug symbols being generated for bionic, disco and eon,

          Is the mechanism used supportable to have ppc64le and arm64 getting dbugsym packages for the LTS 18.04 bionic?

          only looked at 10.4:
          http://mirror.netinch.com/pub/mariadb/repo/10.4/ubuntu/pool/main/m/mariadb-10.4/

          danblack Daniel Black added a comment - dbart , noticed you have amd64 debug symbols being generated for bionic, disco and eon, Is the mechanism used supportable to have ppc64le and arm64 getting dbugsym packages for the LTS 18.04 bionic? only looked at 10.4: http://mirror.netinch.com/pub/mariadb/repo/10.4/ubuntu/pool/main/m/mariadb-10.4/

          Thanks for pointing out they were missing. I see what the problem is. I'll make sure they are included in the next release.

          dbart Daniel Bartholomew added a comment - Thanks for pointing out they were missing. I see what the problem is. I'll make sure they are included in the next release.

          I've updated the Repo Configuration Tool at https://downloads.mariadb.org/mariadb/repositories/ with instructions on how to enable `-dbgsym` packages to show up in APT for bionic, eoan, and focal.

          On bionic there's an issue with some warnings that appear when the debug package component is configured and you run an `apt update`. Eoan and Focal don't show those warnings, so it might just be an issue with the version of apt in bionic and not the repository, and the warnings can be safely ignored. My testing hasn't uncovered any other issues, so I think I'm about ready to close this, unless there's something I'm forgetting.

          dbart Daniel Bartholomew added a comment - I've updated the Repo Configuration Tool at https://downloads.mariadb.org/mariadb/repositories/ with instructions on how to enable `-dbgsym` packages to show up in APT for bionic, eoan, and focal. On bionic there's an issue with some warnings that appear when the debug package component is configured and you run an `apt update`. Eoan and Focal don't show those warnings, so it might just be an issue with the version of apt in bionic and not the repository, and the warnings can be safely ignored. My testing hasn't uncovered any other issues, so I think I'm about ready to close this, unless there's something I'm forgetting.
          danblack Daniel Black added a comment -

          Thank you dbart I'll test next week. Can I assume focal support for aarch64/ppc64le is coming after its LTS release at the end of the month?

          danblack Daniel Black added a comment - Thank you dbart I'll test next week. Can I assume focal support for aarch64/ppc64le is coming after its LTS release at the end of the month?

          danblack Yes, I'll be spinning up builders for aarch64 and ppc64le once the release is out.

          dbart Daniel Bartholomew added a comment - danblack Yes, I'll be spinning up builders for aarch64 and ppc64le once the release is out.

          Basically, the fix for this MDEV was to use a patched version of reprepro that adds the multi-version capability and the ability to import Ubuntu's .ddeb packages. A README for how I compiled it is here: https://gist.github.com/dbart/0ba0f991e9b86d472e28e02998fce835

          dbart Daniel Bartholomew added a comment - Basically, the fix for this MDEV was to use a patched version of reprepro that adds the multi-version capability and the ability to import Ubuntu's .ddeb packages. A README for how I compiled it is here: https://gist.github.com/dbart/0ba0f991e9b86d472e28e02998fce835

          People

            dbart Daniel Bartholomew
            danblack Daniel Black
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.