If you run the debian/autobake-deb.sh multiple times on the same source directory it will fail with the error:
dh: error: debian/control has a duplicate entry for mariadb-plugin-columnstore
This happens because the columnstore section of the autobake-deb.sh script has a simple `cat` statement to add the mariadb-plugin-columnstore package config and it doesn't check if the config has already been added to debian/control, it just blindly adds an additional package config block to the end of debian/control every time it is run.
This won't ever show up in buildbot because we're always running debian/autobake-deb.sh on a fresh directory. But it becomes a pain when trying to debug package builds locally because you have to remember to remove the columnstore package config from the end of debian/control prior to re-running the script.
Running the script should be idempotent, meaning back-to-back runs should have the same result assuming no changes have been made to the source tree. The way to fix would be to either:
Add a check to the columnstore section of the script to not run the cat command unless the mariadb-plugin-columnstore package config is missing from debian/control
...or...
Add the mariadb-plugin-columnstore package config to debian/control so that it doesn't have to be added at compile time
The idempotent is provided by + '[' '!' -f debian/mariadb-plugin-columnstore.install ']'. Is that not being picked up? I guess that git checkout . is still leaving the file around. I'll look at doing something better still.
root@4405d53ce3e1:/source# ls -la debian/mariadb-plugin-columnstore.install
ls: cannot access 'debian/mariadb-plugin-columnstore.install': No such file or directory
I don't understand how you use "git checkout . is still leaving the file around" to explain that "subsequent runs of debian/autobake-deb.sh are not idempotent".
It seems that your PR#2104 has nothing to do with autobake-deb.sh not being idempotent
Sergei Golubchik
added a comment - I don't understand how you use "git checkout . is still leaving the file around" to explain that "subsequent runs of debian/autobake-deb.sh are not idempotent".
It seems that your PR#2104 has nothing to do with autobake-deb.sh not being idempotent
Wrong git command - git clean -dfx will remove debian/mariadb-plugin-columnstore.* files as they aren't checked but leave debian/control with the added package. I'm assuming this was dbart's patten. So if that was used between consecutive debian/autobake-deb.sh then multiple Package: mariadb-plugin-columnstore in debian/control instances will result.
So the PR purges an existing Package: mariadb-plugin-columnstore in debian/control before pulling the definition from the submodule and also unconditionally copies the other columnstore files. I saw this useful in case the submodule was updated between subsequent runs.
Daniel Black
added a comment - Wrong git command - git clean -dfx will remove debian/mariadb-plugin-columnstore.* files as they aren't checked but leave debian/control with the added package. I'm assuming this was dbart 's patten. So if that was used between consecutive debian/autobake-deb.sh then multiple Package: mariadb-plugin-columnstore in debian/control instances will result.
So the PR purges an existing Package: mariadb-plugin-columnstore in debian/control before pulling the definition from the submodule and also unconditionally copies the other columnstore files. I saw this useful in case the submodule was updated between subsequent runs.
People
Daniel Black
Daniel Bartholomew
Votes:
0Vote for this issue
Watchers:
3Start watching this issue
Dates
Created:
Updated:
Resolved:
Git Integration
Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.
{"report":{"fcp":1670.2999997138977,"ttfb":329.8999996185303,"pageVisibility":"visible","entityId":110318,"key":"jira.project.issue.view-issue","isInitial":true,"threshold":1000,"elementTimings":{},"userDeviceMemory":8,"userDeviceProcessors":64,"apdex":0.5,"journeyId":"4e5bc281-3024-4e1c-993e-64ba120af755","navigationType":0,"readyForUser":1767.5,"redirectCount":0,"resourceLoadedEnd":1908.2999997138977,"resourceLoadedStart":364.3999996185303,"resourceTiming":[{"duration":43.200000286102295,"initiatorType":"link","name":"https://jira.mariadb.org/s/2c21342762a6a02add1c328bed317ffd-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/css/_super/batch.css","startTime":364.3999996185303,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":364.3999996185303,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":407.59999990463257,"responseStart":0,"secureConnectionStart":0},{"duration":49.5,"initiatorType":"link","name":"https://jira.mariadb.org/s/7ebd35e77e471bc30ff0eba799ebc151-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/css/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.css?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&slack-enabled=true&whisper-enabled=true","startTime":365.19999980926514,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":365.19999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":414.69999980926514,"responseStart":0,"secureConnectionStart":0},{"duration":231.59999990463257,"initiatorType":"script","name":"https://jira.mariadb.org/s/0917945aaa57108d00c5076fea35e069-CDN/lu2cib/820016/12ta74/0a8bac35585be7fc6c9cc5a0464cd4cf/_/download/contextbatch/js/_super/batch.js?locale=en","startTime":366,"connectEnd":366,"connectStart":366,"domainLookupEnd":366,"domainLookupStart":366,"fetchStart":366,"redirectEnd":0,"redirectStart":0,"requestStart":506.09999990463257,"responseEnd":597.5999999046326,"responseStart":530.8999996185303,"secureConnectionStart":366},{"duration":471.69999980926514,"initiatorType":"script","name":"https://jira.mariadb.org/s/2d8175ec2fa4c816e8023260bd8c1786-CDN/lu2cib/820016/12ta74/494e4c556ecbb29f90a3d3b4f09cb99c/_/download/contextbatch/js/jira.browse.project,project.issue.navigator,jira.view.issue,jira.general,jira.global,atl.general,-_super/batch.js?agile_global_admin_condition=true&jag=true&jira.create.linked.issue=true&locale=en&slack-enabled=true&whisper-enabled=true","startTime":370.59999990463257,"connectEnd":370.59999990463257,"connectStart":370.59999990463257,"domainLookupEnd":370.59999990463257,"domainLookupStart":370.59999990463257,"fetchStart":370.59999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":506.2999997138977,"responseEnd":842.2999997138977,"responseStart":550,"secureConnectionStart":370.59999990463257},{"duration":172.80000019073486,"initiatorType":"script","name":"https://jira.mariadb.org/s/a9324d6758d385eb45c462685ad88f1d-CDN/lu2cib/820016/12ta74/c92c0caa9a024ae85b0ebdbed7fb4bd7/_/download/contextbatch/js/atl.global,-_super/batch.js?locale=en","startTime":374.2999997138977,"connectEnd":374.2999997138977,"connectStart":374.2999997138977,"domainLookupEnd":374.2999997138977,"domainLookupStart":374.2999997138977,"fetchStart":374.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":506.2999997138977,"responseEnd":547.0999999046326,"responseStart":546.5,"secureConnectionStart":374.2999997138977},{"duration":173.7999997138977,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-en/jira.webresources:calendar-en.js","startTime":374.5,"connectEnd":374.5,"connectStart":374.5,"domainLookupEnd":374.5,"domainLookupStart":374.5,"fetchStart":374.5,"redirectEnd":0,"redirectStart":0,"requestStart":506.3999996185303,"responseEnd":548.2999997138977,"responseStart":547.7999997138977,"secureConnectionStart":374.5},{"duration":173,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:calendar-localisation-moment/jira.webresources:calendar-localisation-moment.js","startTime":374.69999980926514,"connectEnd":374.69999980926514,"connectStart":374.69999980926514,"domainLookupEnd":374.69999980926514,"domainLookupStart":374.69999980926514,"fetchStart":374.69999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":506.69999980926514,"responseEnd":547.6999998092651,"responseStart":547.1999998092651,"secureConnectionStart":374.69999980926514},{"duration":89.2000002861023,"initiatorType":"link","name":"https://jira.mariadb.org/s/b04b06a02d1959df322d9cded3aeecc1-CDN/lu2cib/820016/12ta74/a2ff6aa845ffc9a1d22fe23d9ee791fc/_/download/contextbatch/css/jira.global.look-and-feel,-_super/batch.css","startTime":374.8999996185303,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":374.8999996185303,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":464.09999990463257,"responseStart":0,"secureConnectionStart":0},{"duration":170.40000009536743,"initiatorType":"script","name":"https://jira.mariadb.org/rest/api/1.0/shortcuts/820016/47140b6e0a9bc2e4913da06536125810/shortcuts.js?context=issuenavigation&context=issueaction","startTime":375.09999990463257,"connectEnd":468.7999997138977,"connectStart":468.7999997138977,"domainLookupEnd":468.7999997138977,"domainLookupStart":468.7999997138977,"fetchStart":375.09999990463257,"redirectEnd":0,"redirectStart":0,"requestStart":506.8999996185303,"responseEnd":545.5,"responseStart":537.0999999046326,"secureConnectionStart":468.7999997138977},{"duration":91.59999990463257,"initiatorType":"link","name":"https://jira.mariadb.org/s/3ac36323ba5e4eb0af2aa7ac7211b4bb-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/css/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.css?jira.create.linked.issue=true","startTime":375.2999997138977,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":375.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":466.8999996185303,"responseStart":0,"secureConnectionStart":0},{"duration":169.60000038146973,"initiatorType":"script","name":"https://jira.mariadb.org/s/5d5e8fe91fbc506585e83ea3b62ccc4b-CDN/lu2cib/820016/12ta74/d176f0986478cc64f24226b3d20c140d/_/download/contextbatch/js/com.atlassian.jira.projects.sidebar.init,-_super,-project.issue.navigator,-jira.view.issue/batch.js?jira.create.linked.issue=true&locale=en","startTime":375.3999996185303,"connectEnd":375.3999996185303,"connectStart":375.3999996185303,"domainLookupEnd":375.3999996185303,"domainLookupStart":375.3999996185303,"fetchStart":375.3999996185303,"redirectEnd":0,"redirectStart":0,"requestStart":507.7999997138977,"responseEnd":545,"responseStart":519.5999999046326,"secureConnectionStart":375.3999996185303},{"duration":1532,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-js/jira.webresources:bigpipe-js.js","startTime":376.2999997138977,"connectEnd":376.2999997138977,"connectStart":376.2999997138977,"domainLookupEnd":376.2999997138977,"domainLookupStart":376.2999997138977,"fetchStart":376.2999997138977,"redirectEnd":0,"redirectStart":0,"requestStart":880,"responseEnd":1908.2999997138977,"responseStart":1905,"secureConnectionStart":376.2999997138977},{"duration":1508.5999999046326,"initiatorType":"script","name":"https://jira.mariadb.org/s/d41d8cd98f00b204e9800998ecf8427e-CDN/lu2cib/820016/12ta74/1.0/_/download/batch/jira.webresources:bigpipe-init/jira.webresources:bigpipe-init.js","startTime":384.19999980926514,"connectEnd":384.19999980926514,"connectStart":384.19999980926514,"domainLookupEnd":384.19999980926514,"domainLookupStart":384.19999980926514,"fetchStart":384.19999980926514,"redirectEnd":0,"redirectStart":0,"requestStart":881.0999999046326,"responseEnd":1892.7999997138977,"responseStart":1797,"secureConnectionStart":384.19999980926514},{"duration":219.59999990463257,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":770.6999998092651,"connectEnd":770.6999998092651,"connectStart":770.6999998092651,"domainLookupEnd":770.6999998092651,"domainLookupStart":770.6999998092651,"fetchStart":770.6999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":881.2999997138977,"responseEnd":990.2999997138977,"responseStart":944.0999999046326,"secureConnectionStart":770.6999998092651},{"duration":206.5,"initiatorType":"xmlhttprequest","name":"https://jira.mariadb.org/rest/webResources/1.0/resources","startTime":1546.6999998092651,"connectEnd":1546.6999998092651,"connectStart":1546.6999998092651,"domainLookupEnd":1546.6999998092651,"domainLookupStart":1546.6999998092651,"fetchStart":1546.6999998092651,"redirectEnd":0,"redirectStart":0,"requestStart":1666.5,"responseEnd":1753.1999998092651,"responseStart":1744.8999996185303,"secureConnectionStart":1546.6999998092651},{"duration":172.89999961853027,"initiatorType":"script","name":"https://www.google-analytics.com/analytics.js","startTime":1661,"connectEnd":0,"connectStart":0,"domainLookupEnd":0,"domainLookupStart":0,"fetchStart":1661,"redirectEnd":0,"redirectStart":0,"requestStart":0,"responseEnd":1833.8999996185303,"responseStart":0,"secureConnectionStart":0}],"fetchStart":0,"domainLookupStart":83,"domainLookupEnd":92,"connectStart":92,"connectEnd":113,"secureConnectionStart":101,"requestStart":113,"responseStart":330,"responseEnd":381,"domLoading":334,"domInteractive":1975,"domContentLoadedEventStart":1975,"domContentLoadedEventEnd":2047,"domComplete":2320,"loadEventStart":2320,"loadEventEnd":2322,"userAgent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)","marks":[{"name":"bigPipe.sidebar-id.start","time":1926.2999997138977},{"name":"bigPipe.sidebar-id.end","time":1927.7999997138977},{"name":"bigPipe.activity-panel-pipe-id.start","time":1928},{"name":"bigPipe.activity-panel-pipe-id.end","time":1934.0999999046326},{"name":"activityTabFullyLoaded","time":2065.7999997138977}],"measures":[],"correlationId":"649f014028f9bd","effectiveType":"4g","downlink":9.1,"rtt":0,"serverDuration":157,"dbReadsTimeInMs":10,"dbConnsTimeInMs":18,"applicationHash":"9d11dbea5f4be3d4cc21f03a88dd11d8c8687422","experiments":[]}}
Task
Minor
The idempotent is provided by + '[' '!' -f debian/mariadb-plugin-columnstore.install ']'. Is that not being picked up? I guess that git checkout . is still leaving the file around. I'll look at doing something better still.
root@4405d53ce3e1:/source# ls -la debian/mariadb-plugin-columnstore.installls: cannot access 'debian/mariadb-plugin-columnstore.install': No such file or directoryroot@4405d53ce3e1:/source# AUTOBAKE_PREP_CONTROL_RULES_ONLY=1 bash -x debian/autobake-deb.sh+ set -e+ export 'DEB_BUILD_OPTIONS=nocheck '+ DEB_BUILD_OPTIONS='nocheck '+ source ./VERSION++ MYSQL_VERSION_MAJOR=10++ MYSQL_VERSION_MINOR=5++ MYSQL_VERSION_PATCH=16++ SERVER_MATURITY=stable+ [[ -n '' ]]+ '[' -d storage/columnstore/columnstore/debian ']'+ sed /-DPLUGIN_COLUMNSTORE=NO/d -i debian/rules+ '[' '!' -f debian/mariadb-plugin-columnstore.install ']'+ cp -v storage/columnstore/columnstore/debian/mariadb-plugin-columnstore.install storage/columnstore/columnstore/debian/mariadb-plugin-columnstore.postinst storage/columnstore/columnstore/debian/mariadb-plugin-columnstore.postrm storage/columnstore/columnstore/debian/mariadb-plugin-columnstore.prerm storage/columnstore/columnstore/debian/mariadb-plugin-columnstore.triggers debian/'storage/columnstore/columnstore/debian/mariadb-plugin-columnstore.install' -> 'debian/mariadb-plugin-columnstore.install''storage/columnstore/columnstore/debian/mariadb-plugin-columnstore.postinst' -> 'debian/mariadb-plugin-columnstore.postinst''storage/columnstore/columnstore/debian/mariadb-plugin-columnstore.postrm' -> 'debian/mariadb-plugin-columnstore.postrm''storage/columnstore/columnstore/debian/mariadb-plugin-columnstore.prerm' -> 'debian/mariadb-plugin-columnstore.prerm''storage/columnstore/columnstore/debian/mariadb-plugin-columnstore.triggers' -> 'debian/mariadb-plugin-columnstore.triggers'+ echo+ cat storage/columnstore/columnstore/debian/control++ dpkg-architecture -q DEB_BUILD_ARCH+ architecture=amd64++ lsb_release -sc+ CODENAME=sid+ case "${CODENAME}" in+ '[' -n 1 ']'+ exit 0diff --git a/debian/control b/debian/controlindex 5352e9902ed..560cd409611 100644--- a/debian/control+++ b/debian/control@@ -861,3 +861,24 @@ Description: MariaDB database regression test suite - data filesease of use..This package has the architecture independent data files for the test suite.++Package: mariadb-plugin-columnstore+Architecture: amd64+Depends: binutils,+ libjemalloc1 | libjemalloc2,+ libsnappy1 | libsnappy1v5,+ mariadb-server-10.5 (= ${server:Version}),+ net-tools,+ python3,+ ${misc:Depends},+ ${shlibs:Depends}+Breaks: libthrift-dev,+ mariadb-columnstore-libs,+ mariadb-columnstore-platform+Replaces: libthrift-dev,+ mariadb-columnstore-libs,+ mariadb-columnstore-platform+Description: MariaDB ColumnStore storage engine+ The MariaDB ColumnStore storage engine is a high-performance columnar+ analytical engine, aimed at rapid processing of analytical queries on very+ large amounts of data.diff --git a/debian/rules b/debian/rulesindex 57636eb028a..713970bb9d7 100755--- a/debian/rules+++ b/debian/rules@@ -45,7 +45,6 @@ endif# ColumnStore can build only on amd64 and arm64ifneq (,$(filter $(DEB_HOST_ARCH_CPU),amd64 arm64))- CMAKEFLAGS += -DPLUGIN_COLUMNSTORE=NOendififneq ($(DEB_BUILD_ARCH),$(DEB_HOST_ARCH))@@ -100,7 +99,6 @@ endif-DPLUGIN_TOKUDB=NO \-DPLUGIN_CASSANDRA=NO \-DPLUGIN_AWS_KEY_MANAGEMENT=NO \- -DPLUGIN_COLUMNSTORE=NO \-DDEB=$(DEB_VENDOR)# This is needed, otherwise 'make test' will run before binaries have been builtroot@4405d53ce3e1:/source# AUTOBAKE_PREP_CONTROL_RULES_ONLY=1 bash -x debian/autobake-deb.sh+ set -e+ export 'DEB_BUILD_OPTIONS=nocheck '+ DEB_BUILD_OPTIONS='nocheck '+ source ./VERSION++ MYSQL_VERSION_MAJOR=10++ MYSQL_VERSION_MINOR=5++ MYSQL_VERSION_PATCH=16++ SERVER_MATURITY=stable+ [[ -n '' ]]+ '[' -d storage/columnstore/columnstore/debian ']'+ sed /-DPLUGIN_COLUMNSTORE=NO/d -i debian/rules+ '[' '!' -f debian/mariadb-plugin-columnstore.install ']'++ dpkg-architecture -q DEB_BUILD_ARCH+ architecture=amd64++ lsb_release -sc+ CODENAME=sid+ case "${CODENAME}" in+ '[' -n 1 ']'+ exit 0~/repos/mariadb-server-10.5 10.5$ git diffdiff --git a/debian/control b/debian/controlindex 5352e9902ed..560cd409611 100644--- a/debian/control+++ b/debian/control@@ -861,3 +861,24 @@ Description: MariaDB database regression test suite - data filesease of use..This package has the architecture independent data files for the test suite.++Package: mariadb-plugin-columnstore+Architecture: amd64+Depends: binutils,+ libjemalloc1 | libjemalloc2,+ libsnappy1 | libsnappy1v5,+ mariadb-server-10.5 (= ${server:Version}),+ net-tools,+ python3,+ ${misc:Depends},+ ${shlibs:Depends}+Breaks: libthrift-dev,+ mariadb-columnstore-libs,+ mariadb-columnstore-platform+Replaces: libthrift-dev,+ mariadb-columnstore-libs,+ mariadb-columnstore-platform+Description: MariaDB ColumnStore storage engine+ The MariaDB ColumnStore storage engine is a high-performance columnar+ analytical engine, aimed at rapid processing of analytical queries on very+ large amounts of data.diff --git a/debian/rules b/debian/rulesindex 57636eb028a..713970bb9d7 100755--- a/debian/rules+++ b/debian/rules@@ -45,7 +45,6 @@ endif# ColumnStore can build only on amd64 and arm64ifneq (,$(filter $(DEB_HOST_ARCH_CPU),amd64 arm64))- CMAKEFLAGS += -DPLUGIN_COLUMNSTORE=NOendififneq ($(DEB_BUILD_ARCH),$(DEB_HOST_ARCH))@@ -100,7 +99,6 @@ endif-DPLUGIN_TOKUDB=NO \-DPLUGIN_CASSANDRA=NO \-DPLUGIN_AWS_KEY_MANAGEMENT=NO \- -DPLUGIN_COLUMNSTORE=NO \-DDEB=$(DEB_VENDOR)# This is needed, otherwise 'make test' will run before binaries have been built