[MCOL-4166] MCS doesn't survive minor upgrade. Created: 2020-07-13  Updated: 2021-01-12  Resolved: 2020-11-09

Status: Closed
Project: MariaDB ColumnStore
Component/s: installation
Affects Version/s: 1.5.3
Fix Version/s: 5.5.1

Type: Bug Priority: Minor
Reporter: Roman Assignee: Jose Rojas (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
PartOf
is part of MCOL-4134 Clean and fix remaining columnstore c... Closed

 Description   

Can't smoothly upgrade MCS from 10.5.4 to 10.5.5.

apt-get -y install ./mariadb-client-core-10.5_10.5.5+maria~bionic_amd64.deb ./mariadb-client-10.5_10.5.5+maria~bionic_amd64.deb ./mariadb-server-core-10.5_10.5.5+maria~bionic_amd64.deb ./mariadb-common_10.5.5+maria~bionic_all.deb ; apt-get -y install ./mariadb-server-10.5_10.5.5+maria~bionic_amd64.deb ; apt-get -y install ./mariadb-plugin-columnstore_10.5.5+maria~bionic_amd64.deb
 
Reading package lists... Done
 
Building dependency tree       
Reading state information... Done
 
Note, selecting 'mariadb-client-core-10.5' instead of './mariadb-client-core-10.5_10.5.5+maria~bionic_amd64.deb'
 
Note, selecting 'mariadb-client-10.5' instead of './mariadb-client-10.5_10.5.5+maria~bionic_amd64.deb'
 
Note, selecting 'mariadb-server-core-10.5' instead of './mariadb-server-core-10.5_10.5.5+maria~bionic_amd64.deb'
 
Note, selecting 'mariadb-common' instead of './mariadb-common_10.5.5+maria~bionic_all.deb'
 
The following packages were automatically installed and are no longer required:
 
  expect galera-4 libcgi-fast-perl libcgi-pm-perl libencode-locale-perl libfcgi-perl libhtml-parser-perl libhtml-tagset-perl libhtml-template-perl libhttp-date-perl libhttp-message-perl libio-html-perl libjemalloc1
 
  liblwp-mediatypes-perl libsnappy1v5 libtcl8.6 libtimedate-perl liburi-perl linux-gcp-5.3-headers-5.3.0-1026 socat tcl-expect tcl8.6
 
Use 'apt autoremove' to remove them.
 
The following packages will be REMOVED:
 
  mariadb-server-10.5
 
The following packages will be upgraded:
 
  mariadb-client-10.5 mariadb-client-core-10.5 mariadb-common mariadb-server-core-10.5
 
4 upgraded, 0 newly installed, 1 to remove and 34 not upgraded.
 
Need to get 0 B/8836 kB of archives.
 
After this operation, 71.0 MB disk space will be freed.
 
Get:1 /home/roman_nozdrin_mariadb_com/mariadb-common_10.5.5+maria~bionic_all.deb mariadb-common all 1:10.5.5+maria~bionic [4124 B]
 
Get:2 /home/roman_nozdrin_mariadb_com/mariadb-client-core-10.5_10.5.5+maria~bionic_amd64.deb mariadb-client-core-10.5 amd64 1:10.5.5+maria~bionic [764 kB]
 
Get:3 /home/roman_nozdrin_mariadb_com/mariadb-client-10.5_10.5.5+maria~bionic_amd64.deb mariadb-client-10.5 amd64 1:10.5.5+maria~bionic [1550 kB]
 
Get:4 /home/roman_nozdrin_mariadb_com/mariadb-server-core-10.5_10.5.5+maria~bionic_amd64.deb mariadb-server-core-10.5 amd64 1:10.5.5+maria~bionic [6518 kB]
 
(Reading database ... 214912 files and directories currently installed.)
 
Removing mariadb-server-10.5 (1:10.5.4+maria~bionic) ...
 
(Reading database ... 214787 files and directories currently installed.)
 
Preparing to unpack .../mariadb-common_10.5.5+maria~bionic_all.deb ...
 
Unpacking mariadb-common (1:10.5.5+maria~bionic) over (1:10.5.4+maria~bionic) ...
 
dpkg: considering removing mariadb-client-10.5 in favour of mariadb-client-core-10.5 ...
 
dpkg: yes, will remove mariadb-client-10.5 in favour of mariadb-client-core-10.5
 
Preparing to unpack .../mariadb-client-core-10.5_10.5.5+maria~bionic_amd64.deb ...
 
Unpacking mariadb-client-core-10.5 (1:10.5.5+maria~bionic) over (1:10.5.4+maria~bionic) ...
 
Selecting previously unselected package mariadb-client-10.5.
 
Preparing to unpack .../mariadb-client-10.5_10.5.5+maria~bionic_amd64.deb ...
 
Unpacking mariadb-client-10.5 (1:10.5.5+maria~bionic) ...
 
Preparing to unpack .../mariadb-server-core-10.5_10.5.5+maria~bionic_amd64.deb ...
 
Unpacking mariadb-server-core-10.5 (1:10.5.5+maria~bionic) over (1:10.5.4+maria~bionic) ...
 
Setting up mariadb-common (1:10.5.5+maria~bionic) ...
 
Notice: configure-symlinks trigger could not be called.
 
Please manually create symlinks by running: 
  mv -f /etc/mysql/my.cnf /etc/mysql/my.cnf.old
 
  ln -sf mariadb.cnf /etc/mysql/my.cnf
 
Setting up mariadb-client-core-10.5 (1:10.5.5+maria~bionic) ...
 
Setting up mariadb-client-10.5 (1:10.5.5+maria~bionic) ...
 
Setting up mariadb-server-core-10.5 (1:10.5.5+maria~bionic) ...
 
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
 
Reading package lists... Done
 
Building dependency tree       
Reading state information... Done
 
Note, selecting 'mariadb-server-10.5' instead of './mariadb-server-10.5_10.5.5+maria~bionic_amd64.deb'
 
The following packages were automatically installed and are no longer required:
 
  expect libjemalloc1 libsnappy1v5 libtcl8.6 linux-gcp-5.3-headers-5.3.0-1026 tcl-expect tcl8.6
 
Use 'apt autoremove' to remove them.
 
Suggested packages:
 
  mailx mariadb-test tinyca
 
The following NEW packages will be installed:
 
  mariadb-server-10.5
 
0 upgraded, 1 newly installed, 0 to remove and 34 not upgraded.
 
Need to get 0 B/4257 kB of archives.
 
After this operation, 67.8 MB of additional disk space will be used.
 
Get:1 /home/roman_nozdrin_mariadb_com/mariadb-server-10.5_10.5.5+maria~bionic_amd64.deb mariadb-server-10.5 amd64 1:10.5.5+maria~bionic [4257 kB]
 
Preconfiguring packages ...
 
Selecting previously unselected package mariadb-server-10.5.
 
(Reading database ... 214787 files and directories currently installed.)
 
Preparing to unpack .../mariadb-server-10.5_10.5.5+maria~bionic_amd64.deb ...
 
/var/lib/mysql: found previous version 10.5
 
Unpacking mariadb-server-10.5 (1:10.5.5+maria~bionic) ...
 
Setting up mariadb-server-10.5 (1:10.5.5+maria~bionic) ...
 
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
 
Processing triggers for ureadahead (0.100.0-21) ...
 
Processing triggers for systemd (237-3ubuntu10.39) ...
 
Reading package lists... Done
 
Building dependency tree       
Reading state information... Done
 
Note, selecting 'mariadb-plugin-columnstore' instead of './mariadb-plugin-columnstore_10.5.5+maria~bionic_amd64.deb'
 
The following package was automatically installed and is no longer required:
 
  linux-gcp-5.3-headers-5.3.0-1026
 
Use 'apt autoremove' to remove it.
 
The following NEW packages will be installed:
 
  mariadb-plugin-columnstore
 
0 upgraded, 1 newly installed, 0 to remove and 34 not upgraded.
 
Need to get 0 B/6350 kB of archives.
 
After this operation, 29.9 MB of additional disk space will be used.
 
Get:1 /home/roman_nozdrin_mariadb_com/mariadb-plugin-columnstore_10.5.5+maria~bionic_amd64.deb mariadb-plugin-columnstore amd64 1:10.5.5+maria~bionic [6350 kB]
 
Selecting previously unselected package mariadb-plugin-columnstore.
 
(Reading database ... 214911 files and directories currently installed.)
 
Preparing to unpack .../mariadb-plugin-columnstore_10.5.5+maria~bionic_amd64.deb ...
 
Unpacking mariadb-plugin-columnstore (1:10.5.5+maria~bionic) ...
 
Setting up mariadb-plugin-columnstore (1:10.5.5+maria~bionic) ...
 
 
 
Configuration file '/etc/columnstore/Columnstore.xml'
 
 ==> Modified (by you or by a script) since installation.
 
 ==> Package distributor has shipped an updated version.
 
   What would you like to do about it ?  Your options are:
 
    Y or I  : install the package maintainer's version
 
    N or O  : keep your currently-installed version
 
      D     : show the differences between the versions
 
      Z     : start a shell to examine the situation
 
 The default action is to keep your current version.
 
*** Columnstore.xml (Y/I/N/O/D/Z) [default=N] ? Y
 
Installing new version of config file /etc/columnstore/Columnstore.xml ...
 
Installing new version of config file /etc/mysql/mariadb.conf.d/columnstore.cnf ...
 
checking for engine columnstore...
 
columnstore doesn't exist
 
There was an error installing MariaDB ColumnStore engine plugin. Continue to install the engine though. \ 
Please resolve the issues and run necessary scripts manually.
 
Populating the engine initial system catalog.
 
Processing triggers for mariadb-server-10.5 (1:10.5.5+maria~bionic) ...
 
Processing triggers for libc-bin (2.27-3ubuntu1.2) ...
 
root@drrtuy-u18-2:/home/roman_nozdrin_mariadb_com# ls /tmp/



 Comments   
Comment by Roman [ 2020-07-13 ]

There are two issues:

  • Our UDF install SQL part from install_mcs_mysql.sh fails b/c INSERT commands violates PK contstraint in mysql.func.
  • The plugin is of a BETA maturity so MDB fails to load the plugin.
Comment by Roman [ 2020-07-13 ]

Plz review.

Comment by Daniel Lee (Inactive) [ 2020-07-23 ]

server service failed to start after upgrade from 1.5.3-1 to 1.5.5-1.

Comment by Roman [ 2020-07-28 ]

4QA:

You can test either latest community build from husky or our nightly build.

Comment by Todd Stoffel (Inactive) [ 2020-09-03 ]

dleeyh why does your comment say 1.5.5-1. We haven't even finished working on 1.5.4

Comment by Daniel Lee (Inactive) [ 2020-09-04 ]

toddstoffel The fixed version was set to 1.5.5-1

Comment by Daniel Lee (Inactive) [ 2020-10-22 ]

Build tested: Build tested: 5.5.1-1 (hasty BB, build #35830)

apt-get -y install ./mariadb-client-core-10.5_10.5.7+maria~bionic_amd64.deb ./mariadb-client-10.5_10.5.7+maria~bionic_amd64.deb ./mariadb-server-core-10.5_10.5.7+maria~bionic_amd64.deb ./mariadb-common_10.5.7+maria~bionic_all.deb ; apt-get -y install ./mariadb-server-10.5_10.5.7+maria~bionic_amd64.deb ; apt-get -y install ./mariadb-plugin-columnstore_10.5.7+maria~bionic_amd64.deb

Repeated the test again and the server failed to start.

Was there fixes implemented for this issue? The last merge took place 3 months ago.

Configuration file '/etc/columnstore/Columnstore.xml'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
What would you like to do about it ? Your options are:
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : start a shell to examine the situation
The default action is to keep your current version.

      • Columnstore.xml (Y/I/N/O/D/Z) [default=N] ? I
        Installing new version of config file /etc/columnstore/Columnstore.xml ...
        Installing new version of config file /etc/columnstore/storagemanager.cnf ...
        Job for mariadb.service failed because the control process exited with error code.
        See "systemctl status mariadb.service" and "journalctl -xe" for details.
        checking for engine columnstore...
        columnstore doesn't exist
        There was an error installing MariaDB ColumnStore engine plugin. Continue to install the engine though. \
        Please resolve the issues and run necessary scripts manually.
        Populating the engine initial system catalog.
        Processing triggers for systemd (237-3ubuntu10.39) ...
        Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
        Processing triggers for ureadahead (0.100.0-21) ...
        Processing triggers for libc-bin (2.27-3ubuntu1) ...
        Reading package lists... Done
        Building dependency tree
        Reading state information... Done
        Note, selecting 'mariadb-plugin-columnstore' instead of './mariadb-plugin-columnstore_10.5.7+maria~bionic_amd64.deb'
        mariadb-plugin-columnstore is already the newest version (1:10.5.7+maria~bionic).
        0 upgraded, 0 newly installed, 0 to remove and 139 not upgraded.
        [ubuntu18:root~]# mysql
        ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/run/mysqld/mysqld.sock' (2)
        [ubuntu18:root~]# systemctl start mariadb
        Job for mariadb.service failed because the control process exited with error code.
        See "systemctl status mariadb.service" and "journalctl -xe" for details.
        [ubuntu18:root~]# systemctl status mariadb-columnstore
        ● mariadb-columnstore.service - mariadb-columnstore
        Loaded: loaded (/lib/systemd/system/mariadb-columnstore.service; enabled; vendor preset: enabled)
        Active: active (exited) since Thu 2020-10-22 15:31:26 UTC; 1min 20s ago
        Main PID: 12330 (code=exited, status=0/SUCCESS)
        Tasks: 0 (limit: 4915)
        CGroup: /system.slice/mariadb-columnstore.service

Oct 22 15:31:13 vagrant systemd[1]: Starting mariadb-columnstore...
Oct 22 15:31:26 vagrant su[12457]: Successful su for mysql by root
Oct 22 15:31:26 vagrant su[12457]: + ??? root:mysql
Oct 22 15:31:26 vagrant su[12457]: pam_unix(su:session): session opened for user mysql by (uid=0)
Oct 22 15:31:26 vagrant su[12457]: pam_unix(su:session): session closed for user mysql
Oct 22 15:31:26 vagrant systemd[1]: Started mariadb-columnstore.
[ubuntu18:root~]# systemctl status mariadb
● mariadb.service - MariaDB 10.5.7 database server
Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/mariadb.service.d
└─migrated-from-my.cnf-settings.conf
Active: failed (Result: exit-code) since Thu 2020-10-22 15:32:01 UTC; 57s ago
Docs: man:mariadbd(8)
https://mariadb.com/kb/en/library/systemd/
Process: 13095 ExecStart=/usr/sbin/mariadbd $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=exited, status=7)
Process: 12868 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= || VAR=`cd /usr/bin/..; /usr/bin/galera_recovery`; [
Process: 12866 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
Process: 12865 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS)
Main PID: 13095 (code=exited, status=7)
Status: "MariaDB server is down"

Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing t
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [Note] InnoDB: 10.5.7 started; log sequence number 91511; transaction id 9
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [Note] Plugin 'FEEDBACK' is disabled.
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [ERROR] /usr/sbin/mariadbd: unknown variable 'server_audit=FORCE_PLUS_PERM
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [ERROR] Aborting
Oct 22 15:32:01 vagrant systemd[1]: mariadb.service: Main process exited, code=exited, status=7/NOTRUNNING
Oct 22 15:32:01 vagrant systemd[1]: mariadb.service: Failed with result 'exit-code'.
Oct 22 15:32:01 vagrant systemd[1]: Failed to start MariaDB 10.5.7 database server.
[ubuntu18:root~]#
[ubuntu18:root~]#
[ubuntu18:root~]# journalctl -xe
– Subject: Unit mariadb.service has begun start-up
– Defined-By: systemd
– Support: http://www.ubuntu.com/support
–
– Unit mariadb.service has begun starting up.
Oct 22 15:32:00 vagrant mariadbd[13095]: 2020-10-22 15:32:00 0 [Note] /usr/sbin/mariadbd (mysqld 10.5.7-MariaDB-1:10.5.7+maria~bionic-log
Oct 22 15:32:00 vagrant mariadbd[13095]: 2020-10-22 15:32:00 0 [Warning] Plugin 'Columnstore' is of maturity level gamma while the server
Oct 22 15:32:00 vagrant mariadbd[13095]: 2020-10-22 15:32:00 0 [Warning] Plugin 'COLUMNSTORE_COLUMNS' is of maturity level gamma while th
Oct 22 15:32:00 vagrant mariadbd[13095]: 2020-10-22 15:32:00 0 [Warning] Plugin 'COLUMNSTORE_TABLES' is of maturity level gamma while the
Oct 22 15:32:00 vagrant mariadbd[13095]: 2020-10-22 15:32:00 0 [Warning] Plugin 'COLUMNSTORE_FILES' is of maturity level gamma while the
Oct 22 15:32:00 vagrant mariadbd[13095]: 2020-10-22 15:32:00 0 [Warning] Plugin 'COLUMNSTORE_EXTENTS' is of maturity level gamma while th
Oct 22 15:32:00 vagrant mariadbd[13095]: 2020-10-22 15:32:00 0 [ERROR] mariadbd: Can't open shared library '/usr/lib/mysql/plugin/server_
Oct 22 15:32:00 vagrant mariadbd[13095]: 2020-10-22 15:32:00 0 [ERROR] Couldn't load plugins from 'server_audit2.so'.
Oct 22 15:32:00 vagrant mariadbd[13095]: 201022 15:32:00 Columnstore: Started; Version: 1.5.4-1
Oct 22 15:32:00 vagrant mariadbd[13095]: 2020-10-22 15:32:00 0 [Note] InnoDB: Using Linux native AIO
Oct 22 15:32:00 vagrant mariadbd[13095]: 2020-10-22 15:32:00 0 [Note] InnoDB: Uses event mutexes
Oct 22 15:32:00 vagrant mariadbd[13095]: 2020-10-22 15:32:00 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
Oct 22 15:32:00 vagrant mariadbd[13095]: 2020-10-22 15:32:00 0 [Note] InnoDB: Number of pools: 1
Oct 22 15:32:00 vagrant mariadbd[13095]: 2020-10-22 15:32:00 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
Oct 22 15:32:00 vagrant mariadbd[13095]: 2020-10-22 15:32:00 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future atte
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [Note] InnoDB: Initializing buffer pool, total size = 268435456, chunk siz
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [Note] InnoDB: Completed initialization of buffer pool
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner th
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [Note] InnoDB: 128 rollback segments are active.
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [Note] InnoDB: Creating shared tablespace for temporary tables
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing t
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [Note] InnoDB: 10.5.7 started; log sequence number 91511; transaction id 9
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [Note] Plugin 'FEEDBACK' is disabled.
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [ERROR] /usr/sbin/mariadbd: unknown variable 'server_audit=FORCE_PLUS_PERM
Oct 22 15:32:01 vagrant mariadbd[13095]: 2020-10-22 15:32:01 0 [ERROR] Aborting
Oct 22 15:32:01 vagrant systemd[1]: mariadb.service: Main process exited, code=exited, status=7/NOTRUNNING
Oct 22 15:32:01 vagrant systemd[1]: mariadb.service: Failed with result 'exit-code'.
Oct 22 15:32:01 vagrant systemd[1]: Failed to start MariaDB 10.5.7 database server.
– Subject: Unit mariadb.service has failed
– Defined-By: systemd
– Support: http://www.ubuntu.com/support
–
– Unit mariadb.service has failed.
–
– The result is RESULT.

Comment by Daniel Lee (Inactive) [ 2020-10-22 ]

Tried "apt-get upgrade" and got the same error.

Comment by Jose Rojas (Inactive) [ 2020-11-09 ]

Verified working upgrade from 10.5.5 to 10.5.6 on Ubuntu

Used 10.5.5 packages from https://cspkg.s3.amazonaws.com/index.html?prefix=columnstore-1.5.4-1/latest/934/ubuntu18.04/

Engine .deb:
mariadb-plugin-columnstore_10.5.5.3+maria~bionic_amd64.deb

Used 10.5.6 packages from https://cspkg.s3.amazonaws.com/index.html?prefix=columnstore-1.5.4-1/latest/940/ubuntu18.04/
Engine .deb:
mariadb-plugin-columnstore_10.5.6.4+Bmaria~bionic_amd64.deb

Generated at Thu Feb 08 02:48:18 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.