|
Supported platforms according to ColumnStore engineering:
ubuntu 16,18
debian 8,9
centos 7
suse 12
Centos 8 and Ubuntu 20 is coming.
|
|
Trying a bit CentOS 7 packages from bb-10.5-cs, as they got built.
Build 32571 of centos74-amd64, clean installation on CentOS 7.8 by yum, but from locally stored RPMs, without MariaDB repo.
Intuition-driven (without installation documentation).
Trying to install ColumnStore as a normal extra engine
sudo yum install MariaDB-server-10.5.3-1.el7.centos.x86_64.rpm MariaDB-client-10.5.3-1.el7.centos.x86_64.rpm MariaDB-common-10.5.3-1.el7.centos.x86_64.rpm MariaDB-compat-10.5.3-1.el7.centos.x86_64.rpm galera-4-26.4.4-1.rhel7.el7.centos.x86_64.rpm MariaDB-columnstore-engine-10.5.3-1.el7.centos.x86_64.rpm
|
(something that one would probably do to install the server with an extra engine).
=> No luck, lots of missing "requires" with names which mean nothing to me, except that they all start with "lib".
Trying to guess what else it needs
There are two more columnstore packages, "libs" and "platform".
"platform" sounds too much, I don't think I need it, I only need MariaDB with a storage engine. Since it was missing libraries, MariaDB-columnstore-libs-10.5.3-1.el7.centos.x86_64.rpm is probably what I need.
sudo yum install MariaDB-server-10.5.3-1.el7.centos.x86_64.rpm MariaDB-client-10.5.3-1.el7.centos.x86_64.rpm MariaDB-common-10.5.3-1.el7.centos.x86_64.rpm MariaDB-compat-10.5.3-1.el7.centos.x86_64.rpm galera-4-26.4.4-1.rhel7.el7.centos.x86_64.rpm MariaDB-columnstore-engine-10.5.3-1.el7.centos.x86_64.rpm MariaDB-columnstore-libs-10.5.3-1.el7.centos.x86_64.rpm
|
This actually works, installation starts and goes smoothly to the end.
But here come two notes:
First, if all those libs are required, then shouldn't they just be in the same package? Is there any practical reason to extract them into a separate one?
Second, if they have to be separate on whatever reason, shouldn't then MariaDB-columnstore-storage-engine package require MariaDB-columnstore-libs package instead of all those individual libraries?
If the expectation is that somebody would want to install each library individually from elsewhere, rather than from MariaDB-columnstore-libs, I doubt it's realistic.
Trying to run
[elenst@localhost rpms]$ sudo systemctl start mariadb
|
[elenst@localhost rpms]$ sudo mariadb
|
Welcome to the MariaDB monitor. Commands end with ; or \g.
|
Your MariaDB connection id is 3
|
Server version: 10.5.3-MariaDB MariaDB Server
|
|
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
|
|
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
|
MariaDB [(none)]> show engines;
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Columnstore | YES | ColumnStore storage engine | YES | NO | NO |
|
| MRG_MyISAM | YES | Collection of identical MyISAM tables | NO | NO | NO |
|
| MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |
|
| Aria | YES | Crash-safe tables with MyISAM heritage. Used for internal temporary tables and privilege tables | NO | NO | NO |
|
| MyISAM | YES | Non-transactional engine with good performance and small data footprint | NO | NO | NO |
|
| SEQUENCE | YES | Generated tables filled with sequential values | YES | NO | YES |
|
| InnoDB | DEFAULT | Supports transactions, row-level locking, foreign keys and encryption for tables | YES | YES | YES |
|
| PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO |
|
| CSV | YES | Stores tables as CSV files | NO | NO | NO |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
9 rows in set (0.002 sec)
|
|
MariaDB [(none)]>
|
So far so good, the engine is there and active.
MariaDB [(none)]> use test;
|
Database changed
|
MariaDB [test]> create table t1 (a int) engine=Columnstore;
|
ERROR 1815 (HY000): Internal error: Config::Config: error accessing config file /etc/columnstore/Columnstore.xml
|
MariaDB [test]>
|
MariaDB [test]> exit;
|
Bye
|
[elenst@localhost rpms]$ sudo ls /etc/columnstore/
|
ls: cannot access /etc/columnstore/: No such file or directory
|
[elenst@localhost rpms]$
|
Not good. Maybe I need "platform" after all, as I don't have better guesses.
Installing the "platform"
[elenst@localhost rpms]$ sudo yum install MariaDB-columnstore-platform-10.5.3-1.el7.centos.x86_64.rpm
|
Doesn't work, wants libmariadb.so.3.
[elenst@localhost rpms]$ sudo yum install MariaDB-columnstore-platform-10.5.3-1.el7.centos.x86_64.rpm MariaDB-shared-10.5.3-1.el7.centos.x86_64.rpm
|
This works.
Complete!
|
[elenst@localhost rpms]$ sudo ls /etc/columnstore/
|
AlarmConfig.xml ErrorMessage.txt
|
AlarmConfig.xml.installSave MessageFile.txt
|
Columnstore.xml ProcessConfig.xml
|
Columnstore.xml-20200510 storagemanager.cnf
|
Columnstore.xml.columnstoreSave storagemanager.cnf.example
|
ConsoleCmds.xml
|
Looks better. Back to the table creation
[elenst@localhost rpms]$ sudo mariadb test
|
Welcome to the MariaDB monitor. Commands end with ; or \g.
|
Your MariaDB connection id is 4
|
Server version: 10.5.3-MariaDB MariaDB Server
|
|
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
|
|
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
|
MariaDB [test]> create table t1 (a int) engine=Columnstore;
|
ERROR 1815 (HY000): Internal error: Cannot execute the statement. DBRM is read only!
|
MariaDB [test]>
|
Making a wild guess,
[elenst@localhost rpms]$ sudo systemctl status columnstore
|
● columnstore.service - MariaDB Columnstore
|
Loaded: loaded (/usr/lib/systemd/system/columnstore.service; enabled; vendor preset: disabled)
|
Active: inactive (dead)
|
[elenst@localhost rpms]$ sudo systemctl start columnstore
|
[elenst@localhost rpms]$ sudo systemctl status columnstore
|
● columnstore.service - MariaDB Columnstore
|
Loaded: loaded (/usr/lib/systemd/system/columnstore.service; enabled; vendor preset: disabled)
|
Active: active (running) since Sun 2020-05-10 18:49:03 EDT; 2s ago
|
Process: 6704 ExecStart=/usr/bin/columnstore start (code=exited, status=0/SUCCESS)
|
Main PID: 6736 (columnstore_run)
|
Tasks: 4
|
CGroup: /system.slice/columnstore.service
|
├─6736 /bin/bash /bin/columnstore_run.sh -l /tmp/columnstore_tmp_files Pr...
|
└─6756 ProcMon
|
|
May 10 18:49:02 localhost.localdomain systemd[1]: Starting MariaDB Columnstore...
|
May 10 18:49:03 localhost.localdomain columnstore[6704]: Starting module pm1 of Mari...
|
May 10 18:49:03 localhost.localdomain systemd[1]: Started MariaDB Columnstore.
|
May 10 18:49:03 localhost.localdomain ProcessMonitor[6756]: 03.331740 |0|0|0| I 18 C...
|
May 10 18:49:03 localhost.localdomain ProcessMonitor[6756]: 03.331984 |0|0|0| I 18 C...
|
May 10 18:49:03 localhost.localdomain ProcessMonitor[6756]: 03.332004 |0|0|0| D 18 C...
|
May 10 18:49:03 localhost.localdomain ProcessMonitor[6756]: 03.332023 |0|0|0| D 18 C...
|
May 10 18:49:03 localhost.localdomain ProcessMonitor[6756]: 03.335273 |0|0|0| D 18 C...
|
May 10 18:49:03 localhost.localdomain ProcessMonitor[6756]: 03.335346 |0|0|0| D 18 C...
|
Hint: Some lines were ellipsized, use -l to show in full.
|
Note: Maybe it could get started automatically upon installation?
Still doesn't work, though:
[elenst@localhost rpms]$ sudo mariadb test
|
Welcome to the MariaDB monitor. Commands end with ; or \g.
|
Your MariaDB connection id is 5
|
Server version: 10.5.3-MariaDB MariaDB Server
|
|
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
|
|
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
|
MariaDB [test]> create table t1 (a int) engine=Columnstore;
|
ERROR 1815 (HY000): Internal error: Cannot execute the statement. DBRM is read only!
|
MariaDB [test]>
|
Trying to restart both
[elenst@localhost rpms]$ sudo systemctl restart columnstore
|
[elenst@localhost rpms]$ sudo systemctl restart mariadb
|
[elenst@localhost rpms]$ sudo mariadb test
|
|
Welcome to the MariaDB monitor. Commands end with ; or \g.
|
Your MariaDB connection id is 3
|
Server version: 10.5.3-MariaDB MariaDB Server
|
|
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
|
|
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
|
MariaDB [test]> create table t1 (a int) engine=Columnstore;
|
ERROR 1815 (HY000): Internal error: Cannot execute the statement. DBRM is read only!
|
Still doesn't work. Same after attempt to shutdown both and then start both, in any order.
No errors in /var/log/messages upon startup.
Google time
For "Internal error: Cannot execute the statement. DBRM is read only", Google returns me two hits.
First, a 3-year old complaint from another victim in Google groups, where David Thompson is giving some advice. Apparently the user made some progress, but didn't reach a resolution.
Another hit is MCOL-3682 filed by me, where Todd suggests to restart MariaDB server. Well, I already did that. A few times, actually.
Trying David's advice, just in case, even though I have no idea what these incantations mean:
[elenst@localhost rpms]$ sudo mcsadmin resumeDatabaseWrites
|
resumedatabasewrites Sun May 10 18:58:58 2020
|
|
This command resumes the DDL/DML writes to the MariaDB ColumnStore Database
|
Do you want to proceed: (y or n) [n]: y
|
DBRM::send_recv caught: InetStreamSocket::connect: connect() error: Connection refused to: InetStreamSocket: sd: 10 inet: 0.0.0.0 port: 8616
|
DBRM::send_recv caught: InetStreamSocket::connect: connect() error: Connection refused to: InetStreamSocket: sd: 10 inet: 0.0.0.0 port: 8616
|
|
Resume MariaDB ColumnStore Database Writes Request successfully completed
|
[elenst@localhost rpms]$ sudo mariadb test
|
Welcome to the MariaDB monitor. Commands end with ; or \g.
|
Your MariaDB connection id is 4
|
Server version: 10.5.3-MariaDB MariaDB Server
|
|
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
|
|
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
|
MariaDB [test]> create table t1 (a int) engine=Columnstore;
|
ERROR 1815 (HY000): Internal error: Cannot execute the statement. DBRM is read only!
|
MariaDB [test]> exit
|
[elenst@localhost rpms]$ sudo dbrmctl status
|
DBRM::send_recv caught: InetStreamSocket::connect: connect() error: Connection refused to: InetStreamSocket: sd: 3 inet: 0.0.0.0 port: 8616
|
DBRM::send_recv caught: InetStreamSocket::connect: connect() error: Connection refused to: InetStreamSocket: sd: 3 inet: 0.0.0.0 port: 8616
|
Failure: an unspecific communication error.
|
[elenst@localhost rpms]$ sudo dbrmctl readwrite
|
DBRM::send_recv caught: InetStreamSocket::connect: connect() error: Connection refused to: InetStreamSocket: sd: 3 inet: 0.0.0.0 port: 8616
|
DBRM::send_recv caught: InetStreamSocket::connect: connect() error: Connection refused to: InetStreamSocket: sd: 3 inet: 0.0.0.0 port: 8616
|
Failure: an unspecific communication error.
|
[elenst@localhost rpms]$ sudo dbrmctl reload
|
DBRM::send_recv caught: InetStreamSocket::connect: connect() error: Connection refused to: InetStreamSocket: sd: 3 inet: 0.0.0.0 port: 8616
|
DBRM::send_recv caught: InetStreamSocket::connect: connect() error: Connection refused to: InetStreamSocket: sd: 3 inet: 0.0.0.0 port: 8616
|
Failure: an unspecific communication error.
|
Well, that's it for now I guess. The conclusion is it's non-functional, now the only remaining step is to remove it.
Uninstallation
[elenst@localhost rpms]$ sudo yum remove MariaDB-columnstore-platform MariaDB-columnstore-libs MariaDB-columnstore-engine
|
...
|
Complete!
|
[elenst@localhost rpms]$ sudo mariadb
|
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
|
It's understandable that uninstallation has to stop MariaDB server, but would be nice if it restarted it afterwards, if it was running before.
[elenst@localhost rpms]$ sudo find / -name "*columnstore*"
|
find: ‘/run/user/1000/gvfs’: Permission denied
|
/etc/my.cnf.d/columnstore.cnf.rpmsave
|
/etc/columnstore
|
/etc/columnstore/Columnstore.xml.columnstoreSave
|
/var/lib/columnstore
|
/var/log/mariadb/columnstore
|
/tmp/columnstore_tmp_files
|
/usr/share/columnstore
|
A lot of stuff is left after uninstallation. Maybe some of it is legitimate (after all, it's not "purge"), but temporary files or share?
To be continued on a luckier day.
|
|
Thanks a lot for a valuable input.
We are going to merge three packages together to avoid the situation you got into. Most of these issues wouldn't appear if you install all three packages and follow this guide. There is no docs for 1.5 yet though.
We will also remove the needless files left after uninstallation.
JFYI The logs are in /var/log/mariadb/columnstore/.
|
RPM (installation by YUM from local RPMs)
Clean simultaneous installation of server, client and ColumnStore
Packages from build-33027/kvm-rpm-centos74-amd64, CentOS 7.8.2003
I don't remember any details of my previous trial, which is good.
Trying same packages as one needs to install the server from local RPMs, plus ColumnStore
Side note: Why does common need compat?
$ sudo yum install MariaDB-server-10.5.3-1.el7.centos.x86_64.rpm MariaDB-client-10.5.3-1.el7.centos.x86_64.rpm MariaDB-common-10.5.3-1.el7.centos.x86_64.rpm MariaDB-compat-10.5.3-1.el7.centos.x86_64.rpm galera-4-26.4.4-1.rhel7.el7.centos.x86_64.rpm MariaDB-columnstore-engine-10.5.3-1.el7.centos.x86_64.rpm
|
=> doesn't work, it wants libmariadb.so.3. It's strange that it doesn't find any in the default CentOS repos, it should be somewhere there, right? Is it something wrong with dependencies in ColumnStore or with "provides" in MariaDB packages created and provided by CentOS?
Now installing with shared:
$ sudo yum install MariaDB-server-10.5.3-1.el7.centos.x86_64.rpm MariaDB-client-10.5.3-1.el7.centos.x86_64.rpm MariaDB-common-10.5.3-1.el7.centos.x86_64.rpm MariaDB-shared-10.5.3-1.el7.centos.x86_64.rpm MariaDB-compat-10.5.3-1.el7.centos.x86_64.rpm galera-4-26.4.4-1.rhel7.el7.centos.x86_64.rpm MariaDB-columnstore-engine-10.5.3-1.el7.centos.x86_64.rpm
|
The transaction check works.
Installation complains in the process:
/bin/columnstore-post-install: line 28: /usr/bin/mysqld_safe: No such file or directory
|
|
Performing the Single Server Install.
|
|
|
Running the MariaDB ColumnStore setup scripts
|
|
post-mysqld-install Successfully Completed
|
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
|
Error running post-mysql-install, /tmp/columnstore_tmp_files/post-mysql-install.log
|
Exiting...
|
|
/tmp/columnstore_tmp_files/post-mysql-install.log
|
checking for engine columnstore...
|
columnstore doesn't exist
|
This doesn't inspire confidence.
Installation eventually exits with zero code, though.
$ sudo systemctl start mariadb
|
|
$ sudo mysql -e "show engines"
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Columnstore | YES | ColumnStore storage engine | YES | NO | NO |
|
...
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
So it did get installed.
MariaDB [test]> create table t1 (a int) engine=ColumnStore;
|
Query OK, 0 rows affected (2.999 sec)
|
|
MariaDB [test]> insert into t1 values (1),(2);
|
Query OK, 2 rows affected (0.242 sec)
|
Records: 2 Duplicates: 0 Warnings: 0
|
Cool.
MariaDB [test]> select * from t1;
|
ERROR 1815 (HY000): Internal error: The system is not yet ready to accept queries
|
Not cool.
Documentation time (as suggested in previous comments).
Set SELinux to permissive mode
I have enforcing. I'll change it to permissive, although our users aren't going to be happy.
$ sudo setenforce permissive
|
$ sudo getenforce
|
Permissive
|
it is recommended to use UTF-8
I already have UTF-8.
The actual installation process in the documentation is outdated (it still refers to the previous multi-package variant), plus it's focused on enterprise. I can't use it. Unfortunately, I also have no idea whether the rest of the documentation is in any way applicable.
Let's see whether setting permissive SELinux helped.
$ sudo systemctl restart mariadb
|
$ sudo mysql -e "select * from test.t1"
|
ERROR 1815 (HY000) at line 1: Internal error: The system is not yet ready to accept queries
|
There is a command in the documentation for "restarting MariaDB ColumnStore". Let's try that.
$ sudo systemctl stop mariadb
|
$ sudo mcsadmin restartSystem y
|
sudo: mcsadmin: command not found
|
|
$ rpm -ql MariaDB-columnstore-engine-10.5.3-1.el7.centos.x86_64 | grep admin
|
<nothing>
|
That's embarrassing. Now what?
$ systemctl | grep -i columnstore
|
mariadb-columnstore.service
|
|
$ sudo systemctl stop mariadb-columnstore
|
$ sudo systemctl start mariadb-columnstore mariadb
|
|
$ sudo mysql -e "select * from test.t1"
|
+------+
|
| a |
|
+------+
|
| 1 |
|
| 2 |
|
+------+
|
Okay, now it works.
Maybe it will even work with enforcing? After all, the documentation is outdated.
$ sudo setenforce enforcing
|
$ sudo systemctl stop mariadb mariadb-columnstore
|
$ sudo systemctl start mariadb-columnstore mariadb
|
$ sudo mysql -e "select * from test.t1"
|
+------+
|
| a |
|
+------+
|
| 1 |
|
| 2 |
|
+------+
|
|
$ sudo mysql -e "create table test.t2 (b int) engine=ColumnStore; insert into test.t2 values (1); insert into test.t1 values (3)"
|
$ sudo mysql -e "select * from test.t2"
|
+------+
|
| b |
|
+------+
|
| 1 |
|
+------+
|
$ sudo mysql -e "select * from test.t1"
|
+------+
|
| a |
|
+------+
|
| 1 |
|
| 2 |
|
| 3 |
|
+------+
|
It seems to work. I don't know enough about SELinux to foresee all implications, but maybe the requirement about permanent permissive should be re-considered, if there is a possibility to avoid it.
Looking around a bit more.
[elenst@localhost ~]$ sudo systemctl status mariadb-columnstore
|
● mariadb-columnstore.service - mariadb-columnstore
|
Loaded: loaded (/usr/lib/systemd/system/mariadb-columnstore.service; enabled; vendor preset: disabled)
|
Active: active (exited) since Sat 2020-05-30 11:44:57 EDT; 3s ago
|
Process: 3831 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
|
Main PID: 3831 (code=exited, status=0/SUCCESS)
|
Tasks: 0
|
CGroup: /system.slice/mariadb-columnstore.service
|
|
May 30 11:44:57 localhost.localdomain systemd[1]: Starting mariadb-columnstore...
|
May 30 11:44:57 localhost.localdomain systemd[1]: Started mariadb-columnstore.
|
[elenst@localhost ~]$
|
"active (exited)" looks weird, but judging by the contents of the service file, it is intentional.
$ ls /etc/systemd/system/mariadb-columnstore.service.wants/
|
mcs-controllernode.service mcs-exemgr.service mcs-workernode.service
|
mcs-ddlproc.service mcs-loadbrm.service mcs-writeengineserver.service
|
mcs-dmlproc.service mcs-primproc.service
|
That's a lot of services.
$ sudo systemctl | grep mcs
|
mcs-controllernode.service loaded active running mcs-controllernode
|
mcs-ddlproc.service loaded active running mcs-ddlproc
|
mcs-dmlproc.service loaded active running mcs-dmlproc
|
mcs-exemgr.service loaded active running mcs-exemgr
|
mcs-primproc.service loaded active running mcs-primproc
|
mcs-workernode.service loaded active running mcs-workernode
|
mcs-writeengineserver.service loaded active running WriteEngineServer
|
mcs-loadbrm isn't running. I wonder if it should be.
$ sudo systemctl stop mariadb-columnstore
|
$ sudo systemctl | grep mcs
|
mcs-controllernode.service loaded deactivating stop stop mcs-controllernode
|
mcs-workernode.service loaded deactivating stop-sigterm stop mcs-workernode
|
This looks a bit dirty, but maybe it's normal.
Now rebooting.
$ sudo systemctl enable mariadb
|
$ sudo reboot now
|
...
|
$ sudo systemctl | egrep -i 'maria|mcs-'
|
mariadb-columnstore.service loaded active exited mariadb-columnstore
|
mariadb.service loaded active running MariaDB 10.5.3 database server
|
mcs-controllernode.service loaded active running mcs-controllernode
|
mcs-ddlproc.service loaded active running mcs-ddlproc
|
mcs-dmlproc.service loaded active running mcs-dmlproc
|
mcs-exemgr.service loaded active running mcs-exemgr
|
mcs-primproc.service loaded active running mcs-primproc
|
mcs-workernode.service loaded active running mcs-workernode
|
mcs-writeengineserver.service loaded active running WriteEngineServer
|
$ sudo mysql -e "select * from test.t2"
|
+------+
|
| b |
|
+------+
|
| 1 |
|
+------+
|
Still works.
$ cat /etc/my.cnf.d/columnstore.cnf
|
[mysqld]
|
lower_case_table_names=1
|
|
plugin-load-add=ha_columnstore.so
|
|
<cut, the rest is all commented>
|
lower_case_table_names=1 can be a big deal for instances which don't expect it. Is it really necessary? If it is, I think it must be very clearly and explicitly documented everywhere
MariaDB [db]> select * from information_schema.plugins where plugin_name='columnstore' \G
|
*************************** 1. row ***************************
|
PLUGIN_NAME: Columnstore
|
PLUGIN_VERSION: 1.5
|
PLUGIN_STATUS: ACTIVE
|
PLUGIN_TYPE: STORAGE ENGINE
|
PLUGIN_TYPE_VERSION: 100503.0
|
PLUGIN_LIBRARY: ha_columnstore.so
|
PLUGIN_LIBRARY_VERSION: 1.14
|
PLUGIN_AUTHOR: MariaDB Corporation
|
PLUGIN_DESCRIPTION: ColumnStore storage engine
|
PLUGIN_LICENSE: GPL
|
LOAD_OPTION: ON
|
PLUGIN_MATURITY: Gamma
|
PLUGIN_AUTH_VERSION: 1.5.0
|
1 row in set (0.001 sec)
|
Installation of ColumnStore with pre-installed server and client
It should be even easier, but there are some subtle differences, so just in case
<standard uneventful installation of MariaDB-server, MariaDB-client, MariaDB-common and MariaDB-compat plus galera>
$ sudo systemctl start mariadb
|
$ sudo mysql -e "show engines" | grep -i columnstore
|
<expectedly nothing>
|
We already know that ColumnStore wants shared, so I'll skip that trial-and-fail.
$ sudo yum install MariaDB-columnstore-engine-10.5.3-1.el7.centos.x86_64.rpm MariaDB-shared-10.5.3-1.el7.centos.x86_64.rpm
|
Now it doesn't complain about missing mysqld_safe, but it still throws an obscure error about post-mysql-install:
Installing : MariaDB-columnstore-engine-10.5.3-1.el7.centos.x86_64 8/8
|
|
Performing the Single Server Install.
|
|
|
Running the MariaDB ColumnStore setup scripts
|
|
post-mysqld-install Successfully Completed
|
Error running post-mysql-install, /tmp/columnstore_tmp_files/post-mysql-install.log
|
Exiting...
|
Verifying : 1:tcl-8.5.13-8.el7.x86_64 1/8
|
|
/tmp/columnstore_tmp_files/post-mysql-install.log
|
checking for engine columnstore...
|
columnstore doesn't exist
|
$ sudo mysql -e "show engines"
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| CSV | YES | Stores tables as CSV files | NO | NO | NO |
|
| MRG_MyISAM | YES | Collection of identical MyISAM tables | NO | NO | NO |
|
| MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |
|
| Aria | YES | Crash-safe tables with MyISAM heritage. Used for internal temporary tables and privilege tables | NO | NO | NO |
|
| MyISAM | YES | Non-transactional engine with good performance and small data footprint | NO | NO | NO |
|
| SEQUENCE | YES | Generated tables filled with sequential values | YES | NO | YES |
|
| InnoDB | DEFAULT | Supports transactions, row-level locking, foreign keys and encryption for tables | YES | YES | YES |
|
| PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
I guess it doesn't restart MariaDB server.
$ sudo systemctl restart mariadb
|
$ sudo mysql -e "show engines"
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Columnstore | YES | ColumnStore storage engine | YES | NO | NO |
|
| MRG_MyISAM | YES | Collection of identical MyISAM tables | NO | NO | NO |
|
| MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |
|
| Aria | YES | Crash-safe tables with MyISAM heritage. Used for internal temporary tables and privilege tables | NO | NO | NO |
|
| MyISAM | YES | Non-transactional engine with good performance and small data footprint | NO | NO | NO |
|
| SEQUENCE | YES | Generated tables filled with sequential values | YES | NO | YES |
|
| InnoDB | DEFAULT | Supports transactions, row-level locking, foreign keys and encryption for tables | YES | YES | YES |
|
| PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO |
|
| CSV | YES | Stores tables as CSV files | NO | NO | NO |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
Now it's there.
Installation of some other engines, for example RocksDB, does cause server restart so that they are available right after thie installation. Maybe ColumnStore should too?
$ sudo mysql test -e "create table t1 (a int) engine=ColumnStore"
|
$ sudo mysql test -e "insert into t1 values (1),(2); select * from t1"
|
+------+
|
| a |
|
+------+
|
| 1 |
|
| 2 |
|
+------+
|
At least this works right away, ColumnStore service restart isn't needed.
Uninstallation
I'm not as eager to uninstall it as I was at the previous trial, but that's a part of the exercise.
$ sudo yum remove MariaDB-columnstore-engine-10.5.3-1.el7.centos.x86_64
|
...
|
Removed:
|
MariaDB-columnstore-engine.x86_64 0:10.5.3-1.el7.centos
|
|
Complete!
|
$ sudo find / -name "*columnstore*"
|
/etc/my.cnf.d/columnstore.cnf.rpmsave
|
/etc/rsyslog.d/49-columnstore.conf
|
/etc/columnstore
|
/etc/columnstore/Columnstore.xml.columnstoreSave
|
/var/lib/mysql/columnstore_info
|
/var/lib/columnstore
|
/tmp/columnstore_tmp_files
|
There is still some stuff left. I haven't found any clear rules about what yum remove should remove, and whether it should preserve anything. Apparently it can remove everything, except when it doesn't. So, please just check whether this is kept on purpose or orphaned by mistake.
$ sudo mysql -e "show engines" | grep -i columnstore
|
Columnstore YES ColumnStore storage engine YES NO NO
|
The server is again not restarted, which might be worse than not doing it upon installation, because it can cause all sorts of errors.
Some other installation variations likely to happen due to human errors or wrong expectations
Dependencies of MariaDB-columnstore-engine:
$ rpm -qpR MariaDB-columnstore-engine-10.5.3-1.el7.centos.x86_64.rpm | grep -i MariaDB
|
MariaDB-common
|
libmariadb.so.3()(64bit)
|
Oddly, it doesn't depend on MariaDB-server. it means that it can be installed with any MariaDB server (or even without one at all). It's going to bring problems.
I'll enable MariaDB 10.4 repo, but won't pre-install anything, to see what ColumnStore pulls automatically upon installation.
Installation without the server
$ sudo yum install MariaDB-columnstore-engine-10.5.3-1.el7.centos.x86_64.rpm
|
...
|
Installing:
|
MariaDB-columnstore-engine x86_64 10.5.3-1.el7.centos /MariaDB-columnstore-engine-10.5.3-1.el7.centos.x86_64
|
25 M
|
MariaDB-compat x86_64 10.4.13-1.el7.centos mariadb 2.2 M
|
replacing mariadb-libs.x86_64 1:5.5.65-1.el7
|
Installing for dependencies:
|
MariaDB-common x86_64 10.4.13-1.el7.centos mariadb 81 k
|
MariaDB-shared x86_64 10.4.13-1.el7.centos mariadb 113 k
|
boost-atomic x86_64 1.53.0-28.el7 base 35 k
|
boost-chrono x86_64 1.53.0-28.el7 base 44 k
|
boost-filesystem x86_64 1.53.0-28.el7 base 68 k
|
boost-regex x86_64 1.53.0-28.el7 base 296 k
|
expect x86_64 5.45-14.el7_1 base 262 k
|
tcl x86_64 1:8.5.13-8.el7 base 1.9 M
|
|
Transaction Summary
|
So it is indeed going to install without a server.
Same error in the process as upon the first installation:
/bin/columnstore-post-install: line 28: /usr/bin/mysqld_safe: No such file or directory
|
|
Performing the Single Server Install.
|
|
|
Running the MariaDB ColumnStore setup scripts
|
|
post-mysqld-install Successfully Completed
|
/bin/post-mysql-install: line 26: mysql: command not found
|
Error running post-mysql-install, /tmp/columnstore_tmp_files/post-mysql-install.log
|
Exiting...
|
Erasing : 1:mariadb-libs-5.5.65-1.el7.x86_64 11/11
|
Verifying : 1:tcl-8.5.13-8.el7.x86_64 1/11
|
And it got installed:
$ sudo systemctl status mariadb-columnstore
|
● mariadb-columnstore.service - mariadb-columnstore
|
Loaded: loaded (/usr/lib/systemd/system/mariadb-columnstore.service; enabled; vendor preset: disabled)
|
Active: active (exited) since Sun 2020-05-10 18:29:15 EDT; 1min 0s ago
|
Process: 6192 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
|
Main PID: 6192 (code=exited, status=0/SUCCESS)
|
Tasks: 0
|
CGroup: /system.slice/mariadb-columnstore.service
|
|
May 10 18:29:15 localhost.localdomain systemd[1]: Starting mariadb-columnstore...
|
May 10 18:29:15 localhost.localdomain systemd[1]: Started mariadb-columnstore.
|
This is of course useless, there is no point in having a storage engine without the server. Let's try to install the server from the same 10.4 repo.
Installation with a mismatching server
$ sudo yum install MariaDB-server
|
|
Installing:
|
MariaDB-server x86_64 10.4.13-1.el7.centos mariadb 26 M
|
Installing for dependencies:
|
MariaDB-client x86_64 10.4.13-1.el7.centos mariadb 12 M
|
boost-program-options x86_64 1.53.0-28.el7 base 156 k
|
galera-4 x86_64 26.4.4-1.rhel7.el7.centos mariadb 9.5 M
|
|
Transaction Summary
|
It is again willing to install, and doesn't remove the mismatching ColumnStore.
But not surprisingly, nothing good comes out of it:
$ sudo systemctl start mariadb
|
Job for mariadb.service failed because a fatal signal was delivered to the control process. See "systemctl status mariadb.service" and "journalctl -xe" for details.
|
|
2020-05-10 18:34:24 0 [ERROR] mysqld: Can't open shared library '/usr/lib64/mysql/plugin/ha_columnstore.so' (errno: 22, undefined symbol: _ZNK7handler28check_if_updates_are_ignoredEPKc)
|
2020-05-10 18:34:24 0 [ERROR] Couldn't load plugins from 'ha_columnstore.so'.
|
2020-05-10 18:34:24 0 [Note] InnoDB: Using Linux native AIO
|
2020-05-10 18:34:24 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
|
2020-05-10 18:34:24 0 [Note] InnoDB: Uses event mutexes
|
2020-05-10 18:34:24 0 [Note] InnoDB: Compressed tables use zlib 1.2.7
|
2020-05-10 18:34:24 0 [Note] InnoDB: Number of pools: 1
|
2020-05-10 18:34:24 0 [Note] InnoDB: Using SSE2 crc32 instructions
|
2020-05-10 18:34:24 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
|
2020-05-10 18:34:24 0 [Note] InnoDB: Completed initialization of buffer pool
|
2020-05-10 18:34:24 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
|
2020-05-10 18:34:24 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=14602
|
2020-05-10 18:34:24 0 [Note] InnoDB: Starting final batch to recover 32 pages from redo log.
|
libgcc_s.so.1 must be installed for pthread_cancel to work
|
200510 18:34:25 [ERROR] mysqld got signal 6 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
Server version: 10.4.13-MariaDB
|
key_buffer_size=134217728
|
read_buffer_size=131072
|
max_used_connections=0
|
max_threads=153
|
thread_count=0
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467752 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
Thread pointer: 0x0
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x0 thread_stack 0x49000
|
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55f046a4a6ce]
|
/usr/sbin/mysqld(handle_fatal_signal+0x30f)[0x55f0464e0f7f]
|
sigaction.c:0(__restore_rt)[0x7f5b427d1630]
|
:0(__GI_raise)[0x7f5b40aa2387]
|
:0(__GI_abort)[0x7f5b40aa3a78]
|
:0(__libc_message)[0x7f5b40ae4ed7]
|
:0(__GI___libc_fatal)[0x7f5b40ae4fbe]
|
:0(pthread_cancel_init)[0x7f5b427d1e94]
|
:0(_Unwind_ForcedUnwind)[0x7f5b427d1f7c]
|
:0(__GI___pthread_unwind)[0x7f5b427d0362]
|
:0(__pthread_exit)[0x7f5b427caef7]
|
/usr/sbin/mysqld(+0x5af06a)[0x55f0461e006a]
|
/usr/sbin/mysqld(+0xab2bcf)[0x55f0466e3bcf]
|
pthread_create.c:0(start_thread)[0x7f5b427c9ea5]
|
May 10 18:34:25 localhost kernel: mysqld[7254]: segfault at 0 ip 000055f046a47726 sp 00007f5b07ffccf0 error 6 in mysqld[55f045c31000+1479000]
|
/lib64/libc.so.6(clone+0x6d)[0x7f5b40b6a8dd]
|
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
|
information that should help you find out what is causing the crash.
|
Writing a core file...
|
Working directory at /var/lib/mysql
|
Resource Limits:
|
Fatal signal 11 while backtracing
|
May 10 18:34:25 localhost abrt-hook-ccpp: Process 7239 (mysqld) of user 987 killed by SIGSEGV - ignoring (repeated crash)
|
May 10 18:34:25 localhost systemd: mariadb.service: main process exited, code=killed, status=11/SEGV
|
May 10 18:34:25 localhost systemd: Failed to start MariaDB 10.4.13 database server.
|
May 10 18:34:25 localhost systemd: Unit mariadb.service entered failed state.
|
May 10 18:34:25 localhost systemd: mariadb.service failed.
|
So, ColumnStore is not unique, just as other engines/plugins it needs a strict match with the server version.
|
|
now I think bb-10.5-cs is reasonably ok, build-wise. it succeeds building packages on all platforms where it can, and doesn't try to build on platforms where it cannot
|
DEB (installation by apt-get from the local repo)
Server repo as created by buildbot, plus http://yum.mariadb.org/galera4/repo4/deb for the Galera dependency.
Clean simultaneous installation of server, client and ColumnStore
Packages from build-33233/kvm-deb-xenial-amd64, Ubuntu 16.04.6 LTS
sudo apt-get install mariadb-server mariadb-client mariadb-plugin-columnstore
|
The following additional packages will be installed:
|
galera-4 gawk libaio1 libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0
|
libboost-thread1.58.0 libdbd-mysql-perl libdbi-perl libhtml-template-perl libjemalloc1
|
libmariadb3 libmysqlclient20 libpcre2-8-0 libreadline5 libsigsegv2 libsnappy1v5
|
libterm-readkey-perl mariadb-client-10.5 mariadb-client-core-10.5 mariadb-common
|
mariadb-server-10.5 mariadb-server-core-10.5 mysql-common python-mysqldb socat
|
Suggested packages:
|
gawk-doc libmldbm-perl libnet-daemon-perl libsql-statement-perl libipc-sharedcache-perl mailx
|
mariadb-test tinyca mysql-server python-egenix-mxdatetime python-mysqldb-dbg
|
The following NEW packages will be installed:
|
galera-4 gawk libaio1 libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0
|
libboost-thread1.58.0 libdbd-mysql-perl libdbi-perl libhtml-template-perl libjemalloc1
|
libmariadb3 libmysqlclient20 libpcre2-8-0 libreadline5 libsigsegv2 libsnappy1v5
|
libterm-readkey-perl mariadb-client mariadb-client-10.5 mariadb-client-core-10.5
|
mariadb-common mariadb-plugin-columnstore mariadb-server mariadb-server-10.5
|
mariadb-server-core-10.5 mysql-common python-mysqldb socat
|
0 upgraded, 29 newly installed, 0 to remove and 38 not upgraded.
|
mysql-server is a strange suggestion. Apparently it comes from ColumnStore, because it doesn't show up if I only try to install the server and client.
Same post-install error, only with uglier log contents:
Running the MariaDB ColumnStore setup scripts
|
|
post-mysqld-install Successfully Completed
|
Error running post-mysql-install, /tmp/columnstore_tmp_files/post-mysql-install.log
|
Exiting...
|
MariaDB ColumnStore install completed
|
elenst@xenial:~$ sudo cat /tmp/columnstore_tmp_files/post-mysql-install.log
|
columnstore doesn't exist
|
elenst@xenial:~$ sudo hexdump -v -C /tmp/columnstore_tmp_files/post-mysql-install.log
|
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
|
00000020 00 00 00 63 6f 6c 75 6d 6e 73 74 6f 72 65 20 64 |...columnstore d|
|
00000030 6f 65 73 6e 27 74 20 65 78 69 73 74 0a |oesn't exist.|
|
0000003d
|
Still, installation claimed to have succeeded at the end.
As usual upon deb installation, the server gets started automatically right away, no need to start it.
elenst@xenial:~$ sudo mysql
|
Welcome to the MariaDB monitor. Commands end with ; or \g.
|
Your MariaDB connection id is 13
|
Server version: 10.5.4-MariaDB-1:10.5.4+maria~xenial-log mariadb.org binary distribution
|
|
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
|
|
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
|
MariaDB [(none)]> show engines;
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| MRG_MyISAM | YES | Collection of identical MyISAM tables | NO | NO | NO |
|
| CSV | YES | Stores tables as CSV files | NO | NO | NO |
|
| Aria | YES | Crash-safe tables with MyISAM heritage. Used for internal temporary tables and privilege tables | NO | NO | NO |
|
| SEQUENCE | YES | Generated tables filled with sequential values | YES | NO | YES |
|
| MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |
|
| MyISAM | YES | Non-transactional engine with good performance and small data footprint | NO | NO | NO |
|
| PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO |
|
| InnoDB | DEFAULT | Supports transactions, row-level locking, foreign keys and encryption for tables | YES | YES | YES |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
8 rows in set (0.000 sec)
|
Not good, ColumnStore isn't there.
MariaDB [(none)]> select * from information_schema.all_plugins where plugin_name = 'columnstore' \G
|
*************************** 1. row ***************************
|
PLUGIN_NAME: Columnstore
|
PLUGIN_VERSION: 1.5
|
PLUGIN_STATUS: NOT INSTALLED
|
PLUGIN_TYPE: STORAGE ENGINE
|
PLUGIN_TYPE_VERSION: 100504.0
|
PLUGIN_LIBRARY: ha_columnstore.so
|
PLUGIN_LIBRARY_VERSION: 1.14
|
PLUGIN_AUTHOR: MariaDB Corporation
|
PLUGIN_DESCRIPTION: ColumnStore storage engine
|
PLUGIN_LICENSE: GPL
|
LOAD_OPTION: OFF
|
PLUGIN_MATURITY: Gamma
|
PLUGIN_AUTH_VERSION: 1.5.0
|
1 row in set (0.014 sec)
|
So the library is there and can be read, but it wasn't loaded.
elenst@xenial:~$ tail -5 /etc/mysql/my.cnf
|
# * IMPORTANT: Additional settings that can override those from this file!
|
# The files must end with '.cnf', otherwise they'll be ignored.
|
#
|
!include /etc/mysql/mariadb.cnf
|
!includedir /etc/mysql/conf.d/
|
elenst@xenial:~$ ls -l /etc/mysql/conf.d/
|
total 8
|
-rw-r--r-- 1 root root 529 kesä 5 11:43 columnstore.cnf
|
-rw-r--r-- 1 root root 36 kesä 5 11:43 mysqld_safe_syslog.cnf
|
elenst@xenial:~$ cat /etc/mysql/conf.d/columnstore.cnf
|
[mysqld]
|
lower_case_table_names=1
|
|
plugin-load-add=ha_columnstore.so
|
|
# Enable compression by default on create, set to NONE to turn off
|
#columnstore_compression_type=SNAPPY
|
|
# ColumnStore local query flag
|
#columnstore_local_query=0
|
|
#columnstore_diskjoin_smallsidelimit=0
|
#columnstore_diskjoin_largesidelimit=0
|
#columnstore_diskjoin_bucketsize=100
|
#columnstore_um_mem_limit=0
|
|
# Required for Schema Sync
|
#server-id = 1
|
#log_bin
|
|
# Uncomment for this UM to apply slave DML on ColumnStore tables
|
#columnstore_replication_slave=ON
|
The config is there and should have been read. Apparently, installation goes in such order that the server isn't restarted after ColumnStore is installed.
And lower_case_table_names is still there.
Manual restart of mariadb service helps:
elenst@xenial:~$ sudo systemctl restart mariadb
|
elenst@xenial:~$ sudo mysql
|
Welcome to the MariaDB monitor. Commands end with ; or \g.
|
Your MariaDB connection id is 4
|
Server version: 10.5.4-MariaDB-1:10.5.4+maria~xenial-log mariadb.org binary distribution
|
|
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
|
|
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
|
MariaDB [(none)]> show engines;
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Columnstore | YES | ColumnStore storage engine | YES | NO | NO |
|
...
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
9 rows in set (0.001 sec)
|
MariaDB [(none)]> create database db;
|
Query OK, 1 row affected (0.001 sec)
|
|
MariaDB [(none)]> use db;
|
Database changed
|
MariaDB [db]> create table t1 (a int) engine=ColumnStore;
|
Query OK, 0 rows affected (0.660 sec)
|
|
MariaDB [db]> insert into t1 values (1),(2);
|
Query OK, 2 rows affected (0.273 sec)
|
Records: 2 Duplicates: 0 Warnings: 0
|
|
MariaDB [db]> select * from t1;
|
+------+
|
| a |
|
+------+
|
| 1 |
|
| 2 |
|
+------+
|
2 rows in set (0.113 sec)
|
Nice, this worked right away, no need to restart ColumnStore service.
Same active (exited) state of mariadb-columnstore service, and same 8 services in /etc/systemd/system/mariadb-columnstore.service.wants, only 7 of which are running:
$ sudo systemctl | grep mcs
|
mcs-controllernode.service loaded active running mcs-controllernode
|
mcs-ddlproc.service loaded active running mcs-ddlproc
|
mcs-dmlproc.service loaded active running mcs-dmlproc
|
mcs-exemgr.service loaded active running mcs-exemgr
|
● mcs-loadbrm.service loaded failed failed loadbrm
|
mcs-primproc.service loaded active running mcs-primproc
|
mcs-workernode.service loaded active running mcs-workernode
|
mcs-writeengineserver.service loaded active running WriteEngineServer
|
After reboot:
$ sudo systemctl enable mariadb
|
$ sudo reboot now
|
|
...
|
|
elenst@xenial:~$ sudo mysql -e "show engines" | grep ColumnStore
|
Columnstore YES ColumnStore storage engine YES NO NO
|
|
elenst@xenial:~$ sudo mysql db
|
...
|
MariaDB [db]> select * from t1;
|
ERROR 1815 (HY000): Internal error: IDB-2006: 'db.t1' does not exist in Columnstore.
|
MariaDB [db]>
|
Problem.
Troubleshooting
elenst@xenial:~$ sudo mysql db -e "select * from t1"
|
ERROR 1815 (HY000) at line 1: Internal error: IDB-2006: 'db.t1' does not exist in Columnstore.
|
elenst@xenial:~$ date
|
la 6.6.2020 19.38.14 +0300
|
elenst@xenial:~$ ls -l /var/log/mariadb/columnstore/
|
total 20
|
drwxrwxrwx 2 root root 4096 kesä 6 18:32 archive
|
drwxrwxrwx 2 root root 4096 kesä 6 18:32 corefiles
|
-rw-r----- 1 syslog syslog 3558 kesä 6 19:38 debug.log
|
-rw-r----- 1 syslog syslog 719 kesä 1 02:26 info.log
|
drwxrwxrwx 2 root root 4096 kesä 6 18:32 trace
|
elenst@xenial:~$ sudo tail -3 /var/log/mariadb/columnstore/debug.log
|
Jun 6 19:27:25 xenial ExeMgr[1047]: 25.206186 |2147483653|0|0| D 16 CAL0042: End SQL statement
|
Jun 6 19:38:06 xenial ExeMgr[1047]: 06.761795 |2147483654|0|0| D 16 CAL0041: Start SQL statement: select objectid,columnname from syscolumn where schema='db' and tablename='t1' --columnRIDs/FE; ||
|
Jun 6 19:38:06 xenial ExeMgr[1047]: 06.782298 |2147483654|0|0| D 16 CAL0042: End SQL statement
|
That's not really helpful at all. And I don't see any other logs. Nothing in journal, either:
Jun 6 19:36:56 xenial rtkit-daemon[1979]: message repeated 10 times: [ Supervising 4 threads of 2 processes of 1 users.]
|
Jun 6 19:38:06 xenial ExeMgr[1047]: 06.761795 |2147483654|0|0| D 16 CAL0041: Start SQL statement: select objectid,columnname from syscolumn where schema='db' and tablename='t1' --columnRIDs/FE; ||
|
Jun 6 19:38:06 xenial ExeMgr[1047]: 06.782298 |2147483654|0|0| D 16 CAL0042: End SQL statement
|
Jun 6 19:39:09 xenial systemd[1]: Starting Cleanup of Temporary Directories...
|
Services look the same as before:
elenst@xenial:~$ sudo systemctl | egrep 'mcs|columnstore'
|
mariadb-columnstore.service loaded active exited mariadb-columnstore
|
mcs-controllernode.service loaded active running mcs-controllernode
|
mcs-ddlproc.service loaded active running mcs-ddlproc
|
mcs-dmlproc.service loaded active running mcs-dmlproc
|
mcs-exemgr.service loaded active running mcs-exemgr
|
● mcs-loadbrm.service loaded failed failed loadbrm
|
mcs-primproc.service loaded active running mcs-primproc
|
mcs-workernode.service loaded active running mcs-workernode
|
mcs-writeengineserver.service loaded active running WriteEngineServer
|
I don't know what else to check to understand what's wrong, hopefully there will be a troubleshooting section in documentation, if it's not there yet.
Just trying to restart the service:
elenst@xenial:~$ sudo systemctl restart mariadb-columnstore
|
elenst@xenial:~$ sudo mysql db -e "select * from t1"
|
ERROR 1815 (HY000) at line 1: Internal error: DBRM is not responding. Cannot accept queries
|
elenst@xenial:~$ sudo mysql db -e "select * from t1"
|
ERROR 1815 (HY000) at line 1: Internal error: IDB-2006: 'db.t1' does not exist in Columnstore.
|
Still no luck. CREATE doesn't work either:
elenst@xenial:~$ sudo mysql db -e "create table t2 (a int) engine=ColumnStore"
|
ERROR 1815 (HY000) at line 1: Internal error: CAL0009: Error while calling getSysCatDBRoot
|
Trying to restart everything:
elenst@xenial:~$ sudo systemctl stop mariadb mariadb-columnstore
|
Warning: mariadb.service changed on disk. Run 'systemctl daemon-reload' to reload units.
|
That's unexpected (although maybe irrelevant to ColumnStore).
elenst@xenial:~$ sudo systemctl daemon-reload
|
elenst@xenial:~$ sudo systemctl start mariadb-columnstore mariadb
|
elenst@xenial:~$ sudo mysql db -e "select * from t1"
|
ERROR 1815 (HY000) at line 1: Internal error: IDB-2006: 'db.t1' does not exist in Columnstore.
|
I'm out of ideas.
I'm saving a snapshot of this moment, in case I'm asked later to check something I haven't checked.
Uninstallation
elenst@xenial:~$ dpkg -l | egrep -i 'columnstore|mcs'
|
ii mariadb-plugin-columnstore 1:10.5.4+maria~xenial amd64 MariaDB ColumnStore storage engine
|
elenst@xenial:~$ sudo apt-get remove mariadb-plugin-columnstore
|
Reading package lists... Done
|
Building dependency tree
|
Reading state information... Done
|
The following packages were automatically installed and are no longer required:
|
libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libjemalloc1 libsnappy1v5
|
python-mysqldb snapd-login-service
|
Use 'sudo apt autoremove' to remove them.
|
The following packages will be REMOVED:
|
mariadb-plugin-columnstore
|
0 upgraded, 0 newly installed, 1 to remove and 38 not upgraded.
|
After this operation, 28,6 MB disk space will be freed.
|
Do you want to continue? [Y/n] y
|
(Reading database ... 178165 files and directories currently installed.)
|
Removing mariadb-plugin-columnstore (1:10.5.4+maria~xenial) ...
|
|
Mariab Columnstore uninstall completed
|
elenst@xenial:~$ dpkg -l | egrep -i 'columnstore|mcs'
|
rc mariadb-plugin-columnstore 1:10.5.4+maria~xenial amd64 MariaDB ColumnStore storage engine
|
elenst@xenial:~$ sudo apt-get purge mariadb-plugin-columnstore
|
Reading package lists... Done
|
Building dependency tree
|
Reading state information... Done
|
The following packages were automatically installed and are no longer required:
|
libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libjemalloc1 libsnappy1v5
|
python-mysqldb snapd-login-service
|
Use 'sudo apt autoremove' to remove them.
|
The following packages will be REMOVED:
|
mariadb-plugin-columnstore*
|
0 upgraded, 0 newly installed, 1 to remove and 38 not upgraded.
|
After this operation, 0 B of additional disk space will be used.
|
Do you want to continue? [Y/n] y
|
(Reading database ... 178012 files and directories currently installed.)
|
Removing mariadb-plugin-columnstore (1:10.5.4+maria~xenial) ...
|
Purging configuration files for mariadb-plugin-columnstore (1:10.5.4+maria~xenial) ...
|
elenst@xenial:~$ dpkg -l | egrep -i 'columnstore|mcs'
|
The package got removed all right.
elenst@xenial:~$ sudo mysql -e "show engines" | grep -i columnstore
|
Columnstore YES ColumnStore storage engine YES NO NO
|
The server still thinks ColumnStore is present, apparently it wasn't restarted.
elenst@xenial:~$ sudo systemctl restart mariadb
|
elenst@xenial:~$ sudo mysql -e "show engines" | grep -i columnstore
|
Now it's gone.
elenst@xenial:~$ ls -l /etc/mysql/conf.d/
|
total 8
|
-rw-r--r-- 1 root root 529 kesä 6 20:00 columnstore.cnf.rpmsave
|
-rw-r--r-- 1 root root 36 kesä 5 11:43 mysqld_safe_syslog.cnf
|
rpmsave?
elenst@xenial:~$ sudo mysql -e "show databases"
|
+---------------------+
|
| Database |
|
+---------------------+
|
| calpontsys |
|
| columnstore_info |
|
| db |
|
| infinidb_querystats |
|
| information_schema |
|
| mysql |
|
| performance_schema |
|
+---------------------+
|
Should ColumnStore databases still be there?
Maybe it's because I didn't drop the columnstore table before uninstalling it? Will try that next time.
|
|
I suppose retest packages again. At least .deb packages had been tested being in a broken condition, namely MDB .deb packages automaton lack the important file lack at that moment.
|
|
Sure. It has been removed last week. Here is the config we ship now.
|
|
The last push of bb-10.5-cs (2b2481f5a7) failed on all deb builders.
Error from one (I didn't check all, but I expect it's the same everywhere):
dh_install: mariadb-plugin-columnstore missing files: usr/bin/alarmReport.sh
|
So there are no deb packages to re-test.
Please also note that installation keeps failing on SLES/openSUSE 150, even in buildbot, e.g.
http://buildbot.askmonty.org/buildbot/builders/kvm-zyp-sles150-amd64/builds/3461
30/39) Installing: MariaDB-columnstore-engine-10.5.4-1.x86_64 [............done]
|
Additional rpm output:
|
/usr/bin/columnstore-post-install: line 58: /usr/bin/mysqld_safe: No such file or directory
|
|
*** No System Logging Application found (syslog, rsyslog, or syslog-ng)
|
*** For MariaDB Columnstore System Logging functionality, install a System Logging package and reinstall MariaDB Columnstore
|
|
/usr/bin/columnstore-post-install: line 217: /usr/bin/mysqld_safe: No such file or directory
|
checking for engine columnstore...
|
columnstore doesn't exist
|
warning: %post(MariaDB-columnstore-engine-10.5.4-1.x86_64) scriptlet failed, exit status 1
|
|
|
(31/39) Installing: galera-4-26.4.4-1.suse15.0.x86_64 [............done]
|
Additional rpm output:
|
warning: group man does not exist - using root
|
|
|
(32/39) Installing: libpcre2-posix2-10.31-1.14.x86_64 [......done]
|
(33/39) Installing: MariaDB-server-10.5.4-1.x86_64 [............done]
|
Additional rpm output:
|
200613 7:44:35 Columnstore: Started; Version: 1.5.0-1
|
|
|
Two all-privilege accounts were created.
|
One is root@localhost, it has no password, but you need to
|
be system 'root' user to connect. Use, for example, sudo mysql
|
The second is mysql@localhost, it has no password either, but
|
you need to be the system 'mysql' user to connect.
|
After connecting you can set the password, if you would need to be
|
able to connect as any of these users with a password and without sudo
|
|
See the MariaDB Knowledgebase at https://mariadb.com/kb or the
|
MySQL manual for more instructions.
|
|
Please report any problems at https://mariadb.org/jira
|
|
The latest information about MariaDB is available at https://mariadb.org/.
|
You can find additional information about the MySQL part at:
|
https://dev.mysql.com
|
Consider joining MariaDB's strong and vibrant community:
|
https://mariadb.org/get-involved/
|
|
|
|
(34/39) Installing: MariaDB-test-10.5.4-1.x86_64 [.............done]
|
(35/39) Installing: MariaDB-rocksdb-engine-10.5.4-1.x86_64 [............done]
|
(36/39) Installing: MariaDB-oqgraph-engine-10.5.4-1.x86_64 [.........done]
|
(37/39) Installing: MariaDB-gssapi-server-10.5.4-1.x86_64 [.....done]
|
(38/39) Installing: MariaDB-cracklib-password-check-10.5.4-1.x86_64 [.....done]
|
(39/39) Installing: MariaDB-connect-engine-10.5.4-1.x86_64 [............done]
|
Executing %posttrans script 'MariaDB-server-10.5.4-1.x86_64.rpm' [.........done]
|
If it's not supported, it shouldn't be built there.
Build on rhel8-ppc64le and centos73-ppc64le fails, and it doesn't fail on current 10.5, so something is wrong with bb-10.5-cs:
buildbot.askmonty.org/buildbot/builders/kvm-rpm-rhel8-ppc64le/builds/1264/steps/compile/logs/stdio
|
DEB (installation by apt-get from the local repo) [FAILED]
Server repo as created by buildbot, plus http://yum.mariadb.org/galera4/repo4/deb for the Galera dependency.
Clean simultaneous installation of server, client and ColumnStore
Packages from build-33483/kvm-deb-xenial-amd64, Ubuntu 16.04.6 LTS (Xenial)
sudo apt-get install mariadb-server mariadb-client mariadb-plugin-columnstore
|
The following additional packages will be installed:
|
galera-4 gawk libaio1 libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libdbd-mysql-perl
|
libdbi-perl libhtml-template-perl libjemalloc1 libmariadb3 libmysqlclient20 libpcre2-8-0 libreadline5 libsigsegv2 libsnappy1v5
|
libterm-readkey-perl mariadb-client-10.5 mariadb-client-core-10.5 mariadb-common mariadb-server-10.5 mariadb-server-core-10.5
|
mysql-common python-mysqldb socat
|
Suggested packages:
|
gawk-doc libmldbm-perl libnet-daemon-perl libsql-statement-perl libipc-sharedcache-perl mailx mariadb-test tinyca mysql-server
|
python-egenix-mxdatetime python-mysqldb-dbg
|
The following NEW packages will be installed:
|
galera-4 gawk libaio1 libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libdbd-mysql-perl
|
libdbi-perl libhtml-template-perl libjemalloc1 libmariadb3 libmysqlclient20 libpcre2-8-0 libreadline5 libsigsegv2 libsnappy1v5
|
libterm-readkey-perl mariadb-client mariadb-client-10.5 mariadb-client-core-10.5 mariadb-common mariadb-plugin-columnstore
|
mariadb-server mariadb-server-10.5 mariadb-server-core-10.5 mysql-common python-mysqldb socat
|
0 upgraded, 29 newly installed, 0 to remove and 4 not upgraded.
|
Need to get 12,1 MB/31,6 MB of archives.
|
After this operation, 238 MB of additional disk space will be used.
|
Still a strange suggestion mysql-server. Doesn't happen without ColumnStore. And it's actually a bad suggestion, because it wouldn't work at all:
elenst@xenial:~$ sudo apt-get install mariadb-server mariadb-client mariadb-plugin-columnstore mysql-server
|
Reading package lists... Done
|
Building dependency tree
|
Reading state information... Done
|
Some packages could not be installed. This may mean that you have
|
requested an impossible situation or if you are using the unstable
|
distribution that some required packages have not yet been created
|
or been moved out of Incoming.
|
The following information may help to resolve the situation:
|
|
The following packages have unmet dependencies:
|
mariadb-plugin-columnstore : Depends: mariadb-server-10.5 (= 1:10.5.4+maria~xenial) but it is not going to be installed
|
mariadb-server : Depends: mariadb-server-10.5 (>= 1:10.5.4+maria~xenial) but it is not going to be installed
|
mysql-server : Depends: mysql-server-5.7 but it is not going to be installed
|
E: Unable to correct problems, you have held broken packages.
|
Back to the original
sudo apt-get install mariadb-server mariadb-client mariadb-plugin-columnstore
|
Problem
When installation reaches these lines in console
Columnstore YES ColumnStore storage engine YES NO NO
|
columnstore exist
|
a "system problem" message window pops up. A screenshot ColumnStore_system_problem_on_Xenial.png and a full crash report columnstore_crash_report produced by Ubuntu are attached, but probably the essential part is this:
PythonArgs: ['/usr/bin/mcs-loadbrm.py']
|
Traceback:
|
File "/usr/bin/mcs-loadbrm.py", line 17
|
loadbrm = f'/usr/bin/load_brm'
|
^
|
SyntaxError: invalid syntax
|
Despite the error, installation process proceeds behind the screen, and ends with a success exit code.
elenst@xenial:~$ sudo mysql -e "show engines"
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Columnstore | YES | ColumnStore storage engine | YES | NO | NO |
|
...
|
Good, Columnstore is there and active.
elenst@xenial:~$ sudo mysql -e "create database if not exists test; create table test.t1 (a int) engine=Columnstore; insert into test.t1 values (1); select * from test.t1"
|
+------+
|
| a |
|
+------+
|
| 1 |
|
+------+
|
Good, all of CREATE/INSERT/SELECT work.
elenst@xenial:~$ sudo mysql test -e "drop table if exists t1; create table t1 (a int, c varchar(100)) engine=Columnstore; show create table t1; insert into t1 select seq, concat('seq',seq) from seq_1_to_100000; delete from t1 limit 100; select count(*) from t1"+-------+-----------------------------------------------------------------------------------------------------------------------------+
|
| Table | Create Table |
|
+-------+-----------------------------------------------------------------------------------------------------------------------------+
|
| t1 | CREATE TABLE `t1` (
|
`a` int(11) DEFAULT NULL,
|
`c` varchar(100) DEFAULT NULL
|
) ENGINE=Columnstore DEFAULT CHARSET=latin1 |
|
+-------+-----------------------------------------------------------------------------------------------------------------------------+
|
+----------+
|
| count(*) |
|
+----------+
|
| 99900 |
|
+----------+
|
All good.
elenst@xenial:~$ sudo systemctl restart mariadb
|
elenst@xenial:~$ sudo mysql test -e "update t1 set a = a + 100000; select min(a), max(a) from t1"+--------+--------+
|
| min(a) | max(a) |
|
+--------+--------+
|
| 100101 | 200000 |
|
+--------+--------+
|
Still works.
elenst@xenial:~$ sudo systemctl restart mariadb-columnstore
|
System error screen again.
elenst@xenial:~$ sudo mysql test -e "update t1 set a = a + 100000; select min(a), max(a) from t1"
|
ERROR 1815 (HY000) at line 1: Internal error: IDB-2006: 'test.t1' does not exist in Columnstore.
|
elenst@xenial:~$ sudo systemctl stop mariadb mariadb-columnstore
|
elenst@xenial:~$ sudo systemctl start mariadb-columnstore mariadb
|
elenst@xenial:~$ sudo mysql test -e "update t1 set a = a + 100000; select min(a), max(a) from t1"
|
ERROR 1815 (HY000) at line 1: Internal error: IDB-2006: 'test.t1' does not exist in Columnstore.
|
elenst@xenial:~$ sudo systemctl | grep -Ei 'mariadb|column|mcs'
|
mariadb-columnstore.service loaded active exited mariadb-columnstore
|
mariadb.service loaded active running MariaDB 10.5.4 database server
|
mcs-controllernode.service loaded active running mcs-controllernode
|
mcs-ddlproc.service loaded active running mcs-ddlproc
|
mcs-dmlproc.service loaded active running mcs-dmlproc
|
mcs-exemgr.service loaded active running mcs-exemgr
|
● mcs-loadbrm.service loaded failed failed loadbrm
|
mcs-primproc.service loaded active running mcs-primproc
|
● mcs-storagemanager.service loaded failed failed storagemanager
|
mcs-workernode.service loaded active running mcs-workernode
|
mcs-writeengineserver.service loaded active running WriteEngineServer
|
|
syslog
|
Jun 14 15:33:05 xenial systemd[1]: Started MariaDB 10.5.4 database server.
|
Jun 14 15:33:06 xenial systemd[1]: Stopped mcs-workernode.
|
Jun 14 15:33:06 xenial systemd[1]: Stopped storagemanager.
|
Jun 14 15:33:06 xenial systemd[1]: Stopped mariadb-columnstore.
|
Jun 14 15:33:06 xenial systemd[1]: Starting mariadb-columnstore...
|
Jun 14 15:33:06 xenial systemd[1]: Starting storagemanager...
|
Jun 14 15:33:06 xenial systemd[1]: mcs-storagemanager.service: Control process exited, code=exited status=1
|
Jun 14 15:33:06 xenial systemd[1]: Failed to start storagemanager.
|
Jun 14 15:33:06 xenial systemd[1]: mcs-storagemanager.service: Unit entered failed state.
|
Jun 14 15:33:06 xenial systemd[1]: mcs-storagemanager.service: Failed with result 'exit-code'.
|
Jun 14 15:33:06 xenial mariadb-columnstore-start.sh[25510]: Job for mcs-storagemanager.service failed because the control process exited with error code. See "systemctl status mcs-storagemanager.service" and "journalctl -xe" for details.
|
Jun 14 15:33:06 xenial systemd[1]: Starting loadbrm...
|
Jun 14 15:33:06 xenial env[25516]: File "/usr/bin/mcs-loadbrm.py", line 17
|
Jun 14 15:33:06 xenial env[25516]: loadbrm = f'/usr/bin/load_brm'
|
Jun 14 15:33:06 xenial env[25516]: ^
|
Jun 14 15:33:06 xenial env[25516]: SyntaxError: invalid syntax
|
Jun 14 15:33:06 xenial systemd[1]: mcs-loadbrm.service: Main process exited, code=exited, status=1/FAILURE
|
Jun 14 15:33:06 xenial systemd[1]: Failed to start loadbrm.
|
Jun 14 15:33:06 xenial systemd[1]: mcs-loadbrm.service: Unit entered failed state.
|
Jun 14 15:33:06 xenial systemd[1]: mcs-loadbrm.service: Failed with result 'exit-code'.
|
Jun 14 15:33:06 xenial mariadb-columnstore-start.sh[25510]: Job for mcs-loadbrm.service failed because the control process exited with error code. See "systemctl status mcs-loadbrm.service" and "journalctl -xe" for details.
|
Jun 14 15:33:06 xenial systemd[1]: Starting storagemanager...
|
Jun 14 15:33:06 xenial systemd[1]: mcs-storagemanager.service: Control process exited, code=exited status=1
|
Jun 14 15:33:06 xenial systemd[1]: Failed to start storagemanager.
|
Jun 14 15:33:06 xenial systemd[1]: mcs-storagemanager.service: Unit entered failed state.
|
Jun 14 15:33:06 xenial systemd[1]: mcs-storagemanager.service: Failed with result 'exit-code'.
|
Jun 14 15:33:06 xenial systemd[1]: Starting loadbrm...
|
Jun 14 15:33:06 xenial env[25522]: File "/usr/bin/mcs-loadbrm.py", line 17
|
Jun 14 15:33:06 xenial env[25522]: loadbrm = f'/usr/bin/load_brm'
|
Jun 14 15:33:06 xenial env[25522]: ^
|
Jun 14 15:33:06 xenial env[25522]: SyntaxError: invalid syntax
|
Jun 14 15:33:06 xenial systemd[1]: mcs-loadbrm.service: Main process exited, code=exited, status=1/FAILURE
|
Jun 14 15:33:06 xenial systemd[1]: Failed to start loadbrm.
|
Jun 14 15:33:06 xenial systemd[1]: mcs-loadbrm.service: Unit entered failed state.
|
Jun 14 15:33:06 xenial systemd[1]: mcs-loadbrm.service: Failed with result 'exit-code'.
|
Jun 14 15:33:06 xenial systemd[1]: Started mcs-workernode.
|
Jun 14 15:33:06 xenial systemd[1]: Started mcs-controllernode.
|
Jun 14 15:33:06 xenial systemd[1]: Starting mcs-primproc...
|
I guess that system error isn't just cosmetics, and really needs to be fixed.
Test aborted.
|
RPM (installation by YUM from local RPMs) [FAILED]
Clean simultaneous installation of server, client and ColumnStore
Packages from build-33483/kvm-rpm-centos74-amd64, CentOS 7.8.2003
Trying same packages as one needs to install the server from local RPMs, plus ColumnStore
$ sudo yum install MariaDB-server-10.5.4-1.el7.centos.x86_64.rpm MariaDB-client-10.5.4-1.el7.centos.x86_64.rpm MariaDB-common-10.5.4-1.el7.centos.x86_64.rpm MariaDB-compat-10.5.4-1.el7.centos.x86_64.rpm MariaDB-shared-10.5.4-1.el7.centos.x86_64.rpm galera-4-26.4.4-1.rhel7.el7.centos.x86_64.rpm MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64.rpm
|
Installing for dependencies:
|
boost-atomic x86_64 1.53.0-28.el7 base 35 k
|
boost-chrono x86_64 1.53.0-28.el7 base 44 k
|
boost-filesystem x86_64 1.53.0-28.el7 base 68 k
|
boost-program-options x86_64 1.53.0-28.el7 base 156 k
|
boost-regex x86_64 1.53.0-28.el7 base 296 k
|
expect x86_64 5.45-14.el7_1 base 262 k
|
tcl x86_64 1:8.5.13-8.el7 base 1.9 M
|
Installation passes, no visible errors.
[elenst@localhost ~]$ sudo systemctl start mariadb
|
[elenst@localhost ~]$ sudo mysql -e "show engines"
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Columnstore | YES | ColumnStore storage engine | YES | NO | NO |
|
[elenst@localhost ~]$ sudo mysql test -e "create table t1 (a int) engine=Columnstore; insert into t1 values (1); select * from t1"
|
+------+
|
| a |
|
+------+
|
| 1 |
|
+------+
|
Good, the engine is loaded and it works.
[elenst@localhost ~]$ sudo systemctl restart mariadb
|
[elenst@localhost ~]$ sudo mysql test -e "insert into t1 values (2); select * from t1"
|
+------+
|
| a |
|
+------+
|
| 1 |
|
| 2 |
|
+------+
|
Works after MariaDB restart, too.
[elenst@localhost ~]$ sudo systemctl restart mariadb-columnstore
|
[elenst@localhost ~]$ sudo mysql test -e "insert into t1 values (3); select * from t1"
|
ERROR 1815 (HY000) at line 1: Internal error: CAL0001: dmlprocessor.cpp PackageHandler::run() package type 0 IDB-2006: 'test.t1' does not exist in Columnstore.
|
|
[elenst@localhost ~]$ sudo systemctl stop mariadb mariadb-columnstore
|
[elenst@localhost ~]$ sudo systemctl start mariadb-columnstore mariadb
|
[elenst@localhost ~]$
|
[elenst@localhost ~]$ sudo mysql test -e "insert into t1 values (3); select * from t1"
|
ERROR 1815 (HY000) at line 1: Internal error: CAL0001: dmlprocessor.cpp PackageHandler::run() package type 0 IDB-2006: 'test.t1' does not exist in Columnstore.
|
Problem: it stops working after columnstore restart.
A lot of worrisome messages in the messages log:
Jun 14 12:24:04 localhost systemd: Starting storagemanager...
|
Jun 14 12:24:04 localhost mcs-start-storagemanager.py: /usr/bin/env: python3: No such f
|
ile or directory
|
Jun 14 12:24:04 localhost systemd: mcs-storagemanager.service: control process exited,
|
code=exited status=127
|
Jun 14 12:24:04 localhost systemd: Failed to start storagemanager.
|
Jun 14 12:24:04 localhost systemd: Unit mcs-storagemanager.service entered failed state
|
.
|
Jun 14 12:24:04 localhost systemd: mcs-storagemanager.service failed.
|
Jun 14 12:24:04 localhost mariadb-columnstore-start.sh: Job for mcs-storagemanager.serv
|
ice failed because the control process exited with error code. See "systemctl status mc
|
s-storagemanager.service" and "journalctl -xe" for details.
|
Jun 14 12:24:04 localhost systemd: Starting loadbrm...
|
Jun 14 12:24:04 localhost env: /usr/bin/env: python3: No such file or directory
|
Jun 14 12:24:04 localhost systemd: mcs-loadbrm.service: main process exited, code=exite
|
d, status=127/n/a
|
Jun 14 12:24:04 localhost systemd: Failed to start loadbrm.
|
Jun 14 12:24:04 localhost systemd: Unit mcs-loadbrm.service entered failed state.
|
Jun 14 12:24:04 localhost systemd: mcs-loadbrm.service failed.
|
Jun 14 12:24:04 localhost mariadb-columnstore-start.sh: Job for mcs-loadbrm.service fai
|
led because the control process exited with error code. See "systemctl status mcs-loadb
|
rm.service" and "journalctl -xe" for details.
|
[elenst@localhost ~]$ sudo reboot now
|
...
|
[elenst@localhost ~]$ sudo systemctl | grep -iE 'mariadb|mcs|columnst'
|
mariadb-columnstore.service loaded active exited mariadb-columnstore
|
mariadb.service loaded active running MariaDB 10.5.4 database server
|
mcs-controllernode.service loaded active running mcs-controllernode
|
mcs-ddlproc.service loaded active running mcs-ddlproc
|
mcs-dmlproc.service loaded active running mcs-dmlproc
|
mcs-exemgr.service loaded active running mcs-exemgr
|
● mcs-loadbrm.service loaded failed failed loadbrm
|
mcs-primproc.service loaded active running mcs-primproc
|
● mcs-storagemanager.service loaded failed failed storagemanager
|
mcs-workernode.service loaded active running mcs-workernode
|
mcs-writeengineserver.service loaded active running WriteEngineServer
|
|
[elenst@localhost ~]$ sudo mysql test -e "insert into t1 values (3); select * from t1"
|
ERROR 1815 (HY000) at line 1: Internal error: CAL0001: dmlprocessor.cpp PackageHandler::run() package type 0 IDB-2006: 'test.t1' does not exist in Columnstore.
|
Reboot didn't help either.
Test aborted
|
|
Once again thx for the valuable information elenst
It seems our assumption that all platforms are python3 capable by default was too frivolous. We've fixed these issues with the latest push.
|
|
Having a separate user-visible Columnstore_cache engine is not practical, it should be an internal cache, automatically enabled in the replication thread.
|
|
We were very limited on time/resources to come up with a more appropriate solution for the purpose.
|
|
Resource questions aside - this has been introduced with the so called "cache" feature, whose goal is to enable bulked operations on INSERTs, in the expectation that it would help HTAP replication rates.
Evidently, the proposed architecture is not fully compliant with the server's. There is no time right now to either argue, or change. As such, the feature will have to be disabled for this round, and the problem to be resolved for later.
drrtuy, tntnatbry - please leave the code in (assuming it does not impact builds), but make sure it is never reached. We will go offline figuring out the proper solution.
As for tracking - please make a separate ticket (MCOL) but do not declare it a "blocker" or anything for the 10.5 release, not to be part of MDEV-22197.
|
DEB (installation by apt-get from the local repo), PART I [PASSED with notes]
Server repo as created by buildbot, plus http://yum.mariadb.org/galera4/repo4/deb for the Galera dependency.
Clean simultaneous installation of server, client and ColumnStore
Packages from build-33565/kvm-deb-xenial-amd64, Ubuntu 16.04.6 LTS (Xenial)
sudo apt-get install mariadb-server mariadb-client mariadb-plugin-columnstore
|
The following package was automatically installed and is no longer required:
|
snapd-login-service
|
Use 'sudo apt autoremove' to remove it.
|
The following additional packages will be installed:
|
galera-4 gawk libaio1 libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libdbd-mysql-perl
|
libdbi-perl libhtml-template-perl libjemalloc1 libmariadb3 libmysqlclient20 libpcre2-8-0 libreadline5 libsigsegv2 libsnappy1v5
|
libterm-readkey-perl mariadb-client-10.5 mariadb-client-core-10.5 mariadb-common mariadb-server-10.5 mariadb-server-core-10.5
|
mysql-common python-mysqldb socat
|
Suggested packages:
|
gawk-doc libmldbm-perl libnet-daemon-perl libsql-statement-perl libipc-sharedcache-perl mailx mariadb-test tinyca mysql-server
|
python-egenix-mxdatetime python-mysqldb-dbg
|
The following NEW packages will be installed:
|
galera-4 gawk libaio1 libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libdbd-mysql-perl
|
libdbi-perl libhtml-template-perl libjemalloc1 libmariadb3 libmysqlclient20 libpcre2-8-0 libreadline5 libsigsegv2 libsnappy1v5
|
libterm-readkey-perl mariadb-client mariadb-client-10.5 mariadb-client-core-10.5 mariadb-common mariadb-plugin-columnstore
|
mariadb-server mariadb-server-10.5 mariadb-server-core-10.5 mysql-common python-mysqldb socat
|
0 upgraded, 29 newly installed, 0 to remove and 3 not upgraded.
|
Need to get 12,1 MB/31,6 MB of archives.
|
After this operation, 238 MB of additional disk space will be used.
|
Still a strange suggestion mysql-server. Doesn't happen without ColumnStore. And it's actually a bad suggestion, because it wouldn't work at all:
elenst@xenial:~$ sudo apt-get install mariadb-server mariadb-client mariadb-plugin-columnstore mysql-server
|
...
|
The following packages have unmet dependencies:
|
mariadb-plugin-columnstore : Depends: mariadb-server-10.5 (= 1:10.5.4+maria~xenial) but it is not going to be installed
|
mariadb-server : Depends: mariadb-server-10.5 (>= 1:10.5.4+maria~xenial) but it is not going to be installed
|
mysql-server : Depends: mysql-server-5.7 but it is not going to be installed
|
E: Unable to correct problems, you have held broken packages.
|
Back to the original
sudo apt-get install mariadb-server mariadb-client mariadb-plugin-columnstore
|
It goes smoothly. Installation produces this a bit unusual output
Setting up mariadb-plugin-columnstore (1:10.5.4+maria~xenial) ...
|
checking for engine columnstore...
|
Columnstore YES ColumnStore storage engine YES NO NO
|
columnstore exist
|
MariaDB ColumnStore install completed
|
Setting up mariadb-server (1:10.5.4+maria~xenial) ...
|
but it doesn't hurt, so not a big deal.
elenst@xenial:~$ sudo mysql -e "show engines"
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Columnstore | YES | ColumnStore storage engine | YES | NO | NO |
|
...
|
|
elenst@xenial:~$ sudo mysql -e "select * from information_schema.plugins where plugin_name = 'Columnstore'\G"
|
*************************** 1. row ***************************
|
PLUGIN_NAME: Columnstore
|
PLUGIN_VERSION: 1.5
|
PLUGIN_STATUS: ACTIVE
|
PLUGIN_TYPE: STORAGE ENGINE
|
PLUGIN_TYPE_VERSION: 100504.0
|
PLUGIN_LIBRARY: ha_columnstore.so
|
PLUGIN_LIBRARY_VERSION: 1.14
|
PLUGIN_AUTHOR: MariaDB Corporation
|
PLUGIN_DESCRIPTION: ColumnStore storage engine
|
PLUGIN_LICENSE: GPL
|
LOAD_OPTION: ON
|
PLUGIN_MATURITY: Gamma
|
PLUGIN_AUTH_VERSION: 1.5.0
|
Good, it's there and active.
elenst@xenial:~$ sudo mysql -e "create database if not exists test; create table test.t1 (a int) engine=Columnstore; insert into test.t1 values (1); select * from test.t1"
|
+------+
|
| a |
|
+------+
|
| 1 |
|
+------+
|
Good, all of CREATE/INSERT/SELECT work.
elenst@xenial:~$ sudo mysql test -e "drop table if exists t1; create table t1 (a int, c varchar(100)) engine=Columnstore; show create table t1; insert into t1 select seq, concat('seq',seq) from seq_1_to_100000; delete from t1 limit 100; select count(*) from t1"
|
+-------+------------------------------------------------------------------------------------------------------------------------------+
|
| Table | Create Table |
|
+-------+------------------------------------------------------------------------------------------------------------------------------+
|
| t1 | CREATE TABLE `t1` (
|
`a` int(11) DEFAULT NULL,
|
`c` varchar(100) DEFAULT NULL
|
) ENGINE=Columnstore DEFAULT CHARSET=utf8mb4 |
|
+-------+------------------------------------------------------------------------------------------------------------------------------+
|
+----------+
|
| count(*) |
|
+----------+
|
| 99900 |
|
+----------+
|
All good.
elenst@xenial:~$ sudo systemctl restart mariadb
|
elenst@xenial:~$ sudo mysql test -e "update t1 set a = a + 100000; select min(a), max(a) from t1"
|
+--------+--------+
|
| min(a) | max(a) |
|
+--------+--------+
|
| 100101 | 200000 |
|
+--------+--------+
|
Still works after server restart.
elenst@xenial:~$ sudo systemctl restart mariadb-columnstore
|
It feels a bit long, but ends without errors.
elenst@xenial:~$ sudo mysql test -e "update t1 set a = a + 100000; select min(a), max(a) from t1"
|
+--------+--------+
|
| min(a) | max(a) |
|
+--------+--------+
|
| 200101 | 300000 |
|
+--------+--------+
|
Now, that was really long. But it worked eventually.
Let's try it again, now with time.
elenst@xenial:~$ time sudo systemctl restart mariadb-columnstore
|
|
real 0m22.130s
|
user 0m0.016s
|
sys 0m0.009s
|
|
elenst@xenial:~$ time sudo mysql test -e "update t1 set a = a + 100000; select min(a), max(a) from t1"
|
+--------+--------+
|
| min(a) | max(a) |
|
+--------+--------+
|
| 300101 | 400000 |
|
+--------+--------+
|
|
real 0m30.617s
|
user 0m0.009s
|
sys 0m0.000s
|
Still very long. But it seems only for the restart and first query, next one works all right:
elenst@xenial:~$ time sudo mysql test -e "update t1 set a = a + 100000; select min(a), max(a) from t1"
|
+--------+--------+
|
| min(a) | max(a) |
|
+--------+--------+
|
| 400101 | 500000 |
|
+--------+--------+
|
|
real 0m0.372s
|
user 0m0.008s
|
sys 0m0.000s
|
That's where it spends most of the time on restart
|
syslog
|
Jun 17 02:52:07 xenial systemd[1]: Stopping mcs-primproc...
|
Jun 17 02:52:07 xenial systemd[1]: Stopped mcs-primproc.
|
Jun 17 02:52:07 xenial systemd[1]: Stopping mcs-controllernode...
|
Jun 17 02:52:07 xenial controllernode[5057]: DBRM Controller: Waiting for threads to finish...
|
Jun 17 02:52:22 xenial controllernode[5057]: Exiting...
|
Jun 17 02:52:22 xenial systemd[1]: Stopped mcs-controllernode.
|
And this is upon the queries, apparently it's all about closing connection:
Jun 17 02:52:31 xenial env[5508]: DMLProc is ready...
|
Jun 17 02:53:02 xenial mysqld[4909]: DBRM::send_recv: controller node closed the connection
|
Jun 17 02:53:12 xenial mysqld[4909]: DBRM::send_recv: controller node closed the connection
|
Jun 17 02:53:22 xenial ExeMgr[5502]: 22.438228 |7|0|0| D 16 CAL0041: Start SQL statement: ; ||
|
Jun 17 02:53:22 xenial ExeMgr[5502]: 22.487137 |7|0|0| D 16 CAL0042: End SQL statement
|
Jun 17 02:53:22 xenial ExeMgr[5502]: 22.487742 |2147483655|0|0| D 16 CAL0041: Start SQL statement: select objectid from systable where s
|
chema='test' and tablename='t1' --tableRID/; ||
|
Jun 17 02:53:22 xenial ExeMgr[5502]: 22.508282 |2147483655|0|0| D 16 CAL0042: End SQL statement
|
Jun 17 02:53:22 xenial ExeMgr[5502]: 22.508916 |2147483655|0|0| D 16 CAL0041: Start SQL statement: select objectid,columnname from sysco
|
lumn where schema='test' and tablename='t1' --columnRIDs/FE; ||
|
Jun 17 02:53:22 xenial ExeMgr[5502]: 22.537124 |2147483655|0|0| D 16 CAL0042: End SQL statement
|
Jun 17 02:53:22 xenial dmlpackageproc[5527]: 22.554430 |7|10|0| D 21 CAL0001: Start SQL statement: update t1 set a = a + 100000;|test|
|
Jun 17 02:53:22 xenial ExeMgr[5502]: 22.574119 |7|0|0| D 16 CAL0041: Start SQL statement: update t1 set a = a + 100000; ||
|
Jun 17 02:53:22 xenial dmlpackageproc[5527]: 22.947206 |7|10|0| D 21 CAL0001: End SQL statement
|
Jun 17 02:53:22 xenial ExeMgr[5502]: 22.960580 |7|0|0| D 16 CAL0042: End SQL statement
|
Jun 17 02:53:22 xenial dmlpackageproc[5527]: 22.965301 |7|10|0| D 21 CAL0001: Start SQL statement: COMMIT
|
Jun 17 02:53:22 xenial dmlpackageproc[5527]: 22.978548 |7|10|0| D 21 CAL0001: End SQL statement
|
Jun 17 02:53:22 xenial mysqld[4909]: DBRM::send_recv: controller node closed the connection
|
Jun 17 02:53:22 xenial env[5508]: Read 0 bytes. Closing connection 1
|
Jun 17 02:53:32 xenial ExeMgr[5502]: 32.987653 |7|0|0| D 16 CAL0041: Start SQL statement: select min(a), max(a) from t1; |test|
|
Jun 17 02:53:33 xenial ExeMgr[5502]: 33.028279 |7|0|0| D 16 CAL0042: End SQL statement
|
Anyway, it works.
elenst@xenial:~$ sudo systemctl | grep -Ei 'mariadb|column|mcs'
|
mariadb-columnstore.service loaded active exited mariadb-columnstore
|
mariadb.service loaded active running MariaDB 10.5.4 database server
|
mcs-controllernode.service loaded active running mcs-controllernode
|
mcs-ddlproc.service loaded active running mcs-ddlproc
|
mcs-dmlproc.service loaded active running mcs-dmlproc
|
mcs-exemgr.service loaded active running mcs-exemgr
|
mcs-primproc.service loaded active running mcs-primproc
|
● mcs-storagemanager.service loaded failed failed storagemanager
|
mcs-workernode.service loaded active running mcs-workernode
|
mcs-writeengineserver.service loaded active running WriteEngineServer
|
Should we be worried about mcs-storagemenager?
Config
elenst@xenial:~$ cat /etc/mysql/mariadb.conf.d/columnstore.cnf
|
[mysqld]
|
plugin-load-add=ha_columnstore.so
|
|
# Enable compression by default on create, set to NONE to turn off
|
#columnstore_compression_type=SNAPPY
|
|
# ColumnStore local query flag
|
#columnstore_local_query=0
|
|
#columnstore_diskjoin_smallsidelimit=0
|
#columnstore_diskjoin_largesidelimit=0
|
#columnstore_diskjoin_bucketsize=100
|
#columnstore_um_mem_limit=0
|
|
# Required for Schema Sync
|
#server-id = 1
|
#log_bin
|
|
# Uncomment for this UM to apply slave DML on ColumnStore tables
|
#columnstore_replication_slave=ON
|
I don't know what most of these mean, but since they're there, they should work. Let's enable them all and restart.
elenst@xenial:~$ cat /etc/mysql/mariadb.conf.d/columnstore.cnf
|
[mysqld]
|
plugin-load-add=ha_columnstore.so
|
|
# Enable compression by default on create, set to NONE to turn off
|
columnstore_compression_type=SNAPPY
|
|
# ColumnStore local query flag
|
columnstore_local_query=0
|
|
columnstore_diskjoin_smallsidelimit=0
|
columnstore_diskjoin_largesidelimit=0
|
columnstore_diskjoin_bucketsize=100
|
columnstore_um_mem_limit=0
|
|
# Required for Schema Sync
|
server-id = 1
|
log_bin
|
|
# Uncomment for this UM to apply slave DML on ColumnStore tables
|
columnstore_replication_slave=ON
|
|
elenst@xenial:~$ sudo systemctl restart mariadb mariadb-columnstore
|
|
elenst@xenial:~$ sudo mysql
|
Welcome to the MariaDB monitor. Commands end with ; or \g.
|
Your MariaDB connection id is 4
|
Server version: 10.5.4-MariaDB-1:10.5.4+maria~xenial-log mariadb.org binary distribution
|
|
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
|
|
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
|
MariaDB [(none)]> select min(a), max(a) from test.t1;
|
+--------+--------+
|
| min(a) | max(a) |
|
+--------+--------+
|
| 400101 | 500000 |
|
+--------+--------+
|
1 row in set (0.127 sec)
|
|
MariaDB [(none)]> select @@log_bin;
|
+-----------+
|
| @@log_bin |
|
+-----------+
|
| 1 |
|
+-----------+
|
1 row in set (0.000 sec)
|
|
MariaDB [(none)]> create table test.t2 like test.t1;
|
Query OK, 0 rows affected (0.814 sec)
|
|
MariaDB [(none)]> insert into test.t2 select * from test.t1;
|
Query OK, 99900 rows affected (1.274 sec)
|
Records: 99900 Duplicates: 0 Warnings: 0
|
|
MariaDB [(none)]> show binlog events;
|
+-------------------+-----+-------------------+-----------+-------------+---------------------------------------------------------------------+
|
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
|
+-------------------+-----+-------------------+-----------+-------------+---------------------------------------------------------------------+
|
| mysqld-bin.000001 | 4 | Format_desc | 1 | 256 | Server ver: 10.5.4-MariaDB-1:10.5.4+maria~xenial-log, Binlog ver: 4 |
|
| mysqld-bin.000001 | 256 | Gtid_list | 1 | 285 | [] |
|
| mysqld-bin.000001 | 285 | Binlog_checkpoint | 1 | 329 | mysqld-bin.000001 |
|
| mysqld-bin.000001 | 329 | Gtid | 1 | 371 | GTID 0-1-1 |
|
| mysqld-bin.000001 | 371 | Query | 1 | 467 | create table test.t2 like test.t1 |
|
| mysqld-bin.000001 | 467 | Gtid | 1 | 509 | BEGIN GTID 0-1-2 |
|
| mysqld-bin.000001 | 509 | Query | 1 | 613 | insert into test.t2 select * from test.t1 |
|
| mysqld-bin.000001 | 613 | Query | 1 | 682 | COMMIT |
|
+-------------------+-----+-------------------+-----------+-------------+---------------------------------------------------------------------+
|
8 rows in set (0.000 sec)
|
|
MariaDB [(none)]> select @@binlog_format;
|
+-----------------+
|
| @@binlog_format |
|
+-----------------+
|
| MIXED |
|
+-----------------+
|
1 row in set (0.000 sec)
|
|
MariaDB [(none)]> set binlog_format=row;
|
Query OK, 0 rows affected (0.000 sec)
|
|
MariaDB [(none)]> insert into test.t2 select * from test.t1 limit 10;
|
Query OK, 10 rows affected (1.275 sec)
|
Records: 10 Duplicates: 0 Warnings: 0
|
|
MariaDB [(none)]> show binlog events;
|
+-------------------+------+-------------------+-----------+-------------+---------------------------------------------------------------------+
|
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
|
+-------------------+------+-------------------+-----------+-------------+---------------------------------------------------------------------+
|
| mysqld-bin.000001 | 4 | Format_desc | 1 | 256 | Server ver: 10.5.4-MariaDB-1:10.5.4+maria~xenial-log, Binlog ver: 4 |
|
| mysqld-bin.000001 | 256 | Gtid_list | 1 | 285 | [] |
|
| mysqld-bin.000001 | 285 | Binlog_checkpoint | 1 | 329 | mysqld-bin.000001 |
|
| mysqld-bin.000001 | 329 | Gtid | 1 | 371 | GTID 0-1-1 |
|
| mysqld-bin.000001 | 371 | Query | 1 | 467 | create table test.t2 like test.t1 |
|
| mysqld-bin.000001 | 467 | Gtid | 1 | 509 | BEGIN GTID 0-1-2 |
|
| mysqld-bin.000001 | 509 | Query | 1 | 613 | insert into test.t2 select * from test.t1 |
|
| mysqld-bin.000001 | 613 | Query | 1 | 682 | COMMIT |
|
| mysqld-bin.000001 | 682 | Gtid | 1 | 724 | BEGIN GTID 0-1-3 |
|
| mysqld-bin.000001 | 724 | Annotate_rows | 1 | 797 | insert into test.t2 select * from test.t1 limit 10 |
|
| mysqld-bin.000001 | 797 | Table_map | 1 | 845 | table_id: 22 (test.t2) |
|
| mysqld-bin.000001 | 845 | Write_rows_v1 | 1 | 1008 | table_id: 22 flags: STMT_END_F |
|
| mysqld-bin.000001 | 1008 | Query | 1 | 1077 | COMMIT |
|
+-------------------+------+-------------------+-----------+-------------+---------------------------------------------------------------------+
|
13 rows in set (0.000 sec)
|
|
MariaDB [(none)]> drop table test.t2;
|
Query OK, 0 rows affected (0.427 sec)
|
Seems to be working.
Reboot
elenst@xenial:~$ time systemctl enable mariadb mariadb-columnstore
|
Synchronizing state of mariadb.service with SysV init with /lib/systemd/systemd-sysv-install...
|
Executing /lib/systemd/systemd-sysv-install enable mariadb
|
elenst@xenial:~$ sudo reboot now
|
elenst@xenial:~$ sudo mysql -e "select count(*) from test.t1"
|
+----------+
|
| count(*) |
|
+----------+
|
| 99900 |
|
+----------+
|
elenst@xenial:~$ sudo mysql test -e "create table t2 like t1; insert into t2 select * from t1 limit 10000; show create table t2; select count(*) from t2"
|
+-------+------------------------------------------------------------------------------------------------------------------------------+
|
| Table | Create Table |
|
+-------+------------------------------------------------------------------------------------------------------------------------------+
|
| t2 | CREATE TABLE `t2` (
|
`a` int(11) DEFAULT NULL,
|
`c` varchar(100) DEFAULT NULL
|
) ENGINE=Columnstore DEFAULT CHARSET=utf8mb4 |
|
+-------+------------------------------------------------------------------------------------------------------------------------------+
|
+----------+
|
| count(*) |
|
+----------+
|
| 10000 |
|
+----------+
|
Looks good.
MTR
Installing MTR test suite
elenst@xenial:~$ sudo apt-get install mariadb-test
|
...
|
|
elenst@xenial:~$ ls /usr/share/mysql/mysql-test/plugin/
|
auth_gssapi func_test mroonga query_response_time spider type_test
|
connect heap myisam rocksdb test_sql_discovery user_variables
|
disks innobase myisammrg sequence tokudb wsrep_info
|
example metadata_lock_info oqgraph sphinx type_inet
|
I've seen one test in the source tree under columnstore/mysql-test, shouldn't it be here?
For now I'll just take it from the source tarball.
$ wget http://hasky.askmonty.org/archive/bb-10.5-cs/build-33565/kvm-tarbake-jaunty-x86/mariadb-10.5.4.tar.gz
|
...
|
sudo cp -r storage/columnstore/mysql-test/columnstore/ /usr/share/mysql/mysql-test/suite/
|
...
|
elenst@xenial:/usr/share/mysql/mysql-test$ perl ./mtr columnstore.basic --vardir=/dev/shm/var
|
Logging: ./mtr columnstore.basic --vardir=/dev/shm/var
|
...
|
|
==============================================================================
|
|
TEST RESULT TIME (ms) or COMMENT
|
--------------------------------------------------------------------------
|
|
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
|
columnstore.basic [ pass ] 1796
|
--------------------------------------------------------------------------
|
That's nice.
Uninstallation
elenst@xenial:~$ dpkg -l | grep -i columnstore
|
ii mariadb-plugin-columnstore 1:10.5.4+maria~xenial amd64 MariaDB ColumnStore storage engine
|
elenst@xenial:~$ sudo apt-get remove mariadb-plugin-columnstore
|
Reading package lists... Done
|
Building dependency tree
|
Reading state information... Done
|
The following packages were automatically installed and are no longer required:
|
libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libjemalloc1
|
libsnappy1v5 python-mysqldb snapd-login-service
|
Use 'sudo apt autoremove' to remove them.
|
The following packages will be REMOVED:
|
mariadb-plugin-columnstore
|
0 upgraded, 0 newly installed, 1 to remove and 3 not upgraded.
|
After this operation, 32,3 MB disk space will be freed.
|
Do you want to continue? [Y/n] y
|
(Reading database ... 234556 files and directories currently installed.)
|
Removing mariadb-plugin-columnstore (1:10.5.4+maria~xenial) ...
|
|
Mariab Columnstore uninstall completed
|
elenst@xenial:~$ dpkg -l | grep -i columnstore
|
rc mariadb-plugin-columnstore 1:10.5.4+maria~xenial amd64 MariaDB ColumnStore storage engine
|
elenst@xenial:~$ sudo apt-get purge mariadb-plugin-columnstore
|
Reading package lists... Done
|
Building dependency tree
|
Reading state information... Done
|
The following packages were automatically installed and are no longer required:
|
libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libjemalloc1
|
libsnappy1v5 python-mysqldb snapd-login-service
|
Use 'sudo apt autoremove' to remove them.
|
The following packages will be REMOVED:
|
mariadb-plugin-columnstore*
|
0 upgraded, 0 newly installed, 1 to remove and 3 not upgraded.
|
After this operation, 0 B of additional disk space will be used.
|
Do you want to continue? [Y/n] y
|
(Reading database ... 234399 files and directories currently installed.)
|
Removing mariadb-plugin-columnstore (1:10.5.4+maria~xenial) ...
|
Purging configuration files for mariadb-plugin-columnstore (1:10.5.4+maria~xenial) ...
|
elenst@xenial:~$ dpkg -l | grep -i columnstore
|
elenst@xenial:~$ ls -l /etc/mysql/mariadb.conf.d/
|
total 24
|
-rw-r--r-- 1 root root 677 kesä 16 17:15 50-client.cnf
|
-rw-r--r-- 1 root root 231 kesä 16 17:15 50-mysql-clients.cnf
|
-rw-r--r-- 1 root root 917 kesä 16 17:15 50-mysqld_safe.cnf
|
-rw-r--r-- 1 root root 3635 kesä 16 17:15 50-server.cnf
|
-rw-r--r-- 1 root root 417 kesä 16 17:15 60-galera.cnf
|
-rw-r--r-- 1 root root 494 kesä 17 04:12 columnstore.cnf.rpmsave
|
Still rpmsave? Not that it's a big deal, but I thought it was supposed to be gone.
elenst@xenial:~$ sudo mysql -e "show engines"
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Columnstore | YES | ColumnStore storage engine | YES | NO | NO |
|
Server still has it, apparently not restarted upon uninstallation. Let's see how it goes, then.
elenst@xenial:~$ sudo mysql -e "select count(*) from test.t1"
|
ERROR 1815 (HY000) at line 1: Internal error: DBRM is not responding. Cannot accept queries
|
elenst@xenial:~$ sudo mysql -e "drop table test.t1"
|
ERROR 1815 (HY000) at line 1: Internal error: Cannot execute the statement. DBRM is read only!
|
Restart manually, then.
elenst@xenial:~$ sudo systemctl restart mariadb
|
elenst@xenial:~$ sudo mysql -e "drop table test.t1"
|
elenst@xenial:~$ sudo mysql -e "select count(*) from test.t1"
|
ERROR 1146 (42S02) at line 1: Table 'test.t1' doesn't exist
|
That's better.
I don't find any obvious remains of ColumnStore on disk, other than the rpmsave file above and the databases, but it's not expected to drop those, I suppose
elenst@xenial:~$ ls -l /var/lib/mysql/
|
total 124952
|
-rw-rw---- 1 mysql mysql 49152 kesä 17 03:54 aria_log.00000001
|
-rw-rw---- 1 mysql mysql 52 kesä 17 03:54 aria_log_control
|
drwx------ 2 mysql mysql 4096 kesä 17 02:43 calpontsys
|
drwx------ 2 mysql mysql 4096 kesä 17 02:43 columnstore_info
|
-rw-r--r-- 1 root root 0 kesä 17 02:43 debian-10.5.flag
|
-rw-rw---- 1 mysql mysql 1066 kesä 17 03:54 ib_buffer_pool
|
-rw-rw---- 1 mysql mysql 12582912 kesä 17 03:54 ibdata1
|
-rw-rw---- 1 mysql mysql 100663296 kesä 17 03:54 ib_logfile0
|
-rw-rw---- 1 mysql mysql 12582912 kesä 17 03:54 ibtmp1
|
drwx------ 2 mysql mysql 4096 kesä 17 02:43 infinidb_querystats
|
-rw-rw---- 1 mysql mysql 0 kesä 17 02:43 multi-master.info
|
drwx------ 2 mysql mysql 4096 kesä 17 02:43 mysql
|
-rw-rw---- 1 mysql mysql 1253 kesä 17 03:15 mysqld-bin.000001
|
-rw-rw---- 1 mysql mysql 343 kesä 14 15:34 mysqld-bin.000002
|
-rw-rw---- 1 mysql mysql 1642681 kesä 17 03:54 mysqld-bin.000003
|
-rw-rw---- 1 mysql mysql 43691 kesä 17 04:10 mysqld-bin.000004
|
-rw-rw---- 1 mysql mysql 80 kesä 17 03:54 mysqld-bin.index
|
drwx------ 2 mysql mysql 4096 kesä 17 02:43 performance_schema
|
All in all, it seems usable. To be continued with a couple of other installation variants.
|
|
Answering the questions:
it is safe to see mcs-storagemanager failed. This happens if one doesn't use S3 as a storage. We will add a log message to explicitly notify about the state of affairs.
MTR test is rudimentary so we don't even consider adding them into the package. We have plans to migrate our testing to MTR in future. We'll package the tests after we make the test anyhow representable. Meanwhile we continue to rely on our own testing suite.
.rpmsafe goes away with apt purge on debian platforms.
|
DEB (installation by apt-get from the local repo), PART II [PASSED with notes]
Server repo as created by buildbot, plus http://yum.mariadb.org/galera4/repo4/deb for the Galera dependency.
Installation of ColumnStore on top of a previously installed 10.5 server
Packages from build-33565/kvm-deb-xenial-amd64, Ubuntu 16.04.6 LTS (Xenial)
elenst@xenial:~$ dpkg -l | grep -E 'mysql|maria'
|
ii libdbd-mysql-perl 4.033-1ubuntu0.1 amd64 Perl5 database interface to the MySQL database
|
ii libmariadb3:amd64 1:10.5.4+maria~xenial amd64 MariaDB database client library
|
ii libmysqlclient20:amd64 5.7.30-0ubuntu0.16.04.1 amd64 MySQL database client library
|
ii mariadb-client 1:10.5.4+maria~xenial all MariaDB database client (metapackage depending on the latest version)
|
ii mariadb-client-10.5 1:10.5.4+maria~xenial amd64 MariaDB database client binaries
|
ii mariadb-client-core-10.5 1:10.5.4+maria~xenial amd64 MariaDB database core client binaries
|
ii mariadb-common 1:10.5.4+maria~xenial all MariaDB common configuration files
|
ii mariadb-server 1:10.5.4+maria~xenial all MariaDB database server (metapackage depending on the latest version)
|
ii mariadb-server-10.5 1:10.5.4+maria~xenial amd64 MariaDB database server binaries
|
ii mariadb-server-core-10.5 1:10.5.4+maria~xenial amd64 MariaDB database core server files
|
ii mysql-common 1:10.5.4+maria~xenial all MariaDB database common files (e.g. /etc/mysql/my.cnf)
|
|
elenst@xenial:~$ sudo systemctl status mariadb
|
● mariadb.service - MariaDB 10.5.4 database server
|
Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
|
Drop-In: /etc/systemd/system/mariadb.service.d
|
└─migrated-from-my.cnf-settings.conf
|
Active: active (running) since ke 2020-06-17 16:08:02 EEST; 1min 0s ago
|
...
|
elenst@xenial:~$ sudo apt-get install mariadb-plugin-columnstore
|
Reading package lists... Done
|
Building dependency tree
|
Reading state information... Done
|
The following package was automatically installed and is no longer required:
|
snapd-login-service
|
Use 'sudo apt autoremove' to remove it.
|
The following additional packages will be installed:
|
libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libjemalloc1 libsnappy1v5
|
python-mysqldb
|
Suggested packages:
|
mysql-server python-egenix-mxdatetime python-mysqldb-dbg
|
The following NEW packages will be installed:
|
libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libjemalloc1 libsnappy1v5
|
mariadb-plugin-columnstore python-mysqldb
|
0 upgraded, 8 newly installed, 0 to remove and 3 not upgraded.
|
Need to get 465 kB/7 034 kB of archives.
|
After this operation, 34,2 MB of additional disk space will be used.
|
...
|
elenst@xenial:~$ sudo mysql -e "show engines"
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Columnstore | YES | ColumnStore storage engine | YES | NO | NO |
|
...
|
Columnstore is there.
elenst@xenial:~$ sudo mysql -e "create database test; use test; create table t1 (a int) engine=Columnstore; insert into t1 select seq from seq_1_to_1000; select count(*) from t1"
|
+----------+
|
| count(*) |
|
+----------+
|
| 1000 |
|
+----------+
|
And it works.
<cut: some more SQL working as expected>
MariaDB [test]> create table t7 (a int) engine=Columnstore;
|
Query OK, 0 rows affected (0.518 sec)
|
|
MariaDB [test]> create table t8 (a int) engine=InnoDB;
|
Query OK, 0 rows affected (0.023 sec)
|
|
MariaDB [test]> drop table t7, t8;
|
ERROR 1178 (42000): The storage engine for the table doesn't support The syntax or the data type(s) is not supported by Columnstore. Please check the Columnstore syntax guide for supported syntax or data types.
|
MariaDB [test]> drop table t7;
|
Query OK, 0 rows affected (0.340 sec)
|
|
MariaDB [test]> drop table t8;
|
Query OK, 0 rows affected, 1 warning (0.072 sec)
|
|
MariaDB [test]> show warnings;
|
+---------+------+--------------------------------------------------------------------------------------------------+
|
| Level | Code | Message |
|
+---------+------+--------------------------------------------------------------------------------------------------+
|
| Warning | 9999 | Error occured during file deletion. Restart DDLProc or use command tool ddlcleanup to clean up. |
|
+---------+------+--------------------------------------------------------------------------------------------------+
|
1 row in set (0.000 sec)
|
That's a strange effect, especially given that t8 is not even a Columnstore table, has never been.
Uninstallation
I will do it with apt rather than apt-get this time, as previous comments say apt would remove rpmsave. apt-get purge didn't do it in the previous test, but maybe apt will be different.
elenst@xenial:~$ sudo apt remove mariadb-plugin-columnstore
|
Reading package lists... Done
|
Building dependency tree
|
Reading state information... Done
|
The following packages were automatically installed and are no longer required:
|
libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libjemalloc1 libsnappy1v5
|
python-mysqldb snapd-login-service
|
Use 'sudo apt autoremove' to remove them.
|
The following packages will be REMOVED:
|
mariadb-plugin-columnstore
|
0 upgraded, 0 newly installed, 1 to remove and 4 not upgraded.
|
After this operation, 32,3 MB disk space will be freed.
|
Do you want to continue? [Y/n] y
|
(Reading database ... 213681 files and directories currently installed.)
|
Removing mariadb-plugin-columnstore (1:10.5.4+maria~xenial) ...
|
|
Mariab Columnstore uninstall completed
|
elenst@xenial:~$ dpkg -l | grep -i columnstore
|
rc mariadb-plugin-columnstore 1:10.5.4+maria~xenial amd64 MariaDB ColumnStore storage engine
|
elenst@xenial:~$ ls -l /etc/mysql/mariadb.conf.d/
|
total 28
|
-rw-r--r-- 1 root root 677 kesä 16 17:15 50-client.cnf
|
-rw-r--r-- 1 root root 231 kesä 16 17:15 50-mysql-clients.cnf
|
-rw-r--r-- 1 root root 917 kesä 16 17:15 50-mysqld_safe.cnf
|
-rw-r--r-- 1 root root 3635 kesä 16 17:15 50-server.cnf
|
-rw-r--r-- 1 root root 417 kesä 16 17:15 60-galera.cnf
|
-rw-r--r-- 1 root root 503 kesä 17 16:43 columnstore.cnf
|
-rw-r--r-- 1 root root 503 kesä 17 16:50 columnstore.cnf.rpmsave
|
elenst@xenial:~$ sudo apt purge mariadb-plugin-columnstore
|
Reading package lists... Done
|
Building dependency tree
|
Reading state information... Done
|
The following packages were automatically installed and are no longer required:
|
libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libjemalloc1 libsnappy1v5
|
python-mysqldb snapd-login-service
|
Use 'sudo apt autoremove' to remove them.
|
The following packages will be REMOVED:
|
mariadb-plugin-columnstore*
|
0 upgraded, 0 newly installed, 1 to remove and 4 not upgraded.
|
After this operation, 0 B of additional disk space will be used.
|
Do you want to continue? [Y/n] y
|
(Reading database ... 213524 files and directories currently installed.)
|
Removing mariadb-plugin-columnstore (1:10.5.4+maria~xenial) ...
|
Purging configuration files for mariadb-plugin-columnstore (1:10.5.4+maria~xenial) ...
|
elenst@xenial:~$ ls -l /etc/mysql/mariadb.conf.d/
|
total 24
|
-rw-r--r-- 1 root root 677 kesä 16 17:15 50-client.cnf
|
-rw-r--r-- 1 root root 231 kesä 16 17:15 50-mysql-clients.cnf
|
-rw-r--r-- 1 root root 917 kesä 16 17:15 50-mysqld_safe.cnf
|
-rw-r--r-- 1 root root 3635 kesä 16 17:15 50-server.cnf
|
-rw-r--r-- 1 root root 417 kesä 16 17:15 60-galera.cnf
|
-rw-r--r-- 1 root root 503 kesä 17 16:50 columnstore.cnf.rpmsave
|
No, rpmsave is not removed by apt purge either.
All in all, it worked like the simultaneous installation, no extra surprises.
|
DEB (installation by apt-get from the local repo), PART III [FAILED]
Server repo as created by buildbot, plus http://yum.mariadb.org/galera4/repo4/deb for the Galera dependency.
Installation of ColumnStore on top of an older server (major upgrade)
Packages from build-33565/kvm-deb-xenial-amd64, Ubuntu 16.04.6 LTS (Xenial)
The goal is to make sure that Columnstore won't attempt to install with an old server. Xenial has 10.0 in their repos, I won't use it, the packaging is too different. Instead I'll take 10.3 from MariaDB repo.
elenst@xenial:~$ dpkg -l | grep -iE 'maria|mysql'
|
ii libdbd-mysql-perl 4.033-1ubuntu0.1 amd64 Perl5 database interface to the MySQL database
|
ii libmariadb3:amd64 1:10.3.23+maria~xenial amd64 MariaDB database client library
|
ii libmysqlclient20:amd64 5.7.30-0ubuntu0.16.04.1 amd64 MySQL database client library
|
ii mariadb-client 1:10.3.23+maria~xenial all MariaDB database client (metapackage depending on the latest version)
|
ii mariadb-client-10.3 1:10.3.23+maria~xenial amd64 MariaDB database client binaries
|
ii mariadb-client-core-10.3 1:10.3.23+maria~xenial amd64 MariaDB database core client binaries
|
ii mariadb-common 1:10.3.23+maria~xenial all MariaDB database common files (e.g. /etc/mysql/conf.d/mariadb.cnf)
|
ii mariadb-server 1:10.3.23+maria~xenial all MariaDB database server (metapackage depending on the latest version)
|
ii mariadb-server-10.3 1:10.3.23+maria~xenial amd64 MariaDB database server binaries
|
ii mariadb-server-core-10.3 1:10.3.23+maria~xenial amd64 MariaDB database core server files
|
ii mysql-common 1:10.3.23+maria~xenial all MariaDB database common files (e.g. /etc/mysql/my.cnf)
|
elenst@xenial:~$ sudo systemctl status mariadb
|
● mariadb.service - MariaDB 10.3.23 database server
|
Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
|
Drop-In: /etc/systemd/system/mariadb.service.d
|
└─migrated-from-my.cnf-settings.conf
|
Active: active (running) since ke 2020-06-17 18:01:16 EEST; 47s ago
|
elenst@xenial:~$ sudo apt-get install mariadb-plugin-columnstore
|
Reading package lists... Done
|
Building dependency tree
|
Reading state information... Done
|
The following package was automatically installed and is no longer required:
|
snapd-login-service
|
Use 'sudo apt autoremove' to remove it.
|
The following additional packages will be installed:
|
galera-4 libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libjemalloc1 libpcre2-8-0
|
libsnappy1v5 mariadb-client mariadb-client-10.5 mariadb-client-core-10.5 mariadb-common mariadb-server mariadb-server-10.5
|
mariadb-server-core-10.5 python-mysqldb
|
Suggested packages:
|
mailx mariadb-test tinyca mysql-server python-egenix-mxdatetime python-mysqldb-dbg
|
The following packages will be REMOVED:
|
galera-3 mariadb-client-10.3 mariadb-client-core-10.3 mariadb-server-10.3 mariadb-server-core-10.3
|
The following NEW packages will be installed:
|
galera-4 libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libjemalloc1 libpcre2-8-0
|
libsnappy1v5 mariadb-client-10.5 mariadb-client-core-10.5 mariadb-plugin-columnstore mariadb-server-10.5 mariadb-server-core-10.5
|
python-mysqldb
|
The following packages will be upgraded:
|
mariadb-client mariadb-common mariadb-server
|
3 upgraded, 14 newly installed, 5 to remove and 6 not upgraded.
|
Need to get 9 652 kB/29,0 MB of archives.
|
After this operation, 48,5 MB of additional disk space will be used.
|
Do you want to continue? [Y/n]
|
A system crash pop-up again, and
Setting up mariadb-plugin-columnstore (1:10.5.4+maria~xenial) ...
|
MySQL Password file missing or incorrect, check .my.cnf file
|
dpkg: error processing package mariadb-plugin-columnstore (--configure):
|
subprocess installed post-installation script returned error exit status 2
|
Setting up python-mysqldb (1.3.7-1build2) ...
|
Processing triggers for libc-bin (2.23-0ubuntu11) ...
|
Processing triggers for ureadahead (0.100.0-19.1) ...
|
Processing triggers for systemd (229-4ubuntu21.28) ...
|
Processing triggers for mariadb-server-10.5 (1:10.5.4+maria~xenial) ...
|
Errors were encountered while processing:
|
mariadb-plugin-columnstore
|
E: Sub-process /usr/bin/dpkg returned an error code (1)
|
I don't really know what it means, but it didn't go well.
elenst@xenial:~$ dpkg -l | grep -iE 'maria|mysql'
|
ii libdbd-mysql-perl 4.033-1ubuntu0.1 amd64 Perl5 database interface to the MySQL database
|
ii libmariadb3:amd64 1:10.3.23+maria~xenial amd64 MariaDB database client library
|
ii libmysqlclient20:amd64 5.7.30-0ubuntu0.16.04.1 amd64 MySQL database client library
|
ii mariadb-client 1:10.5.4+maria~xenial all MariaDB database client (metapackage depending on the latest version)
|
rc mariadb-client-10.3 1:10.3.23+maria~xenial amd64 MariaDB database client binaries
|
ii mariadb-client-10.5 1:10.5.4+maria~xenial amd64 MariaDB database client binaries
|
ii mariadb-client-core-10.5 1:10.5.4+maria~xenial amd64 MariaDB database core client binaries
|
ii mariadb-common 1:10.5.4+maria~xenial all MariaDB common configuration files
|
iF mariadb-plugin-columnstore 1:10.5.4+maria~xenial amd64 MariaDB ColumnStore storage engine
|
ii mariadb-server 1:10.5.4+maria~xenial all MariaDB database server (metapackage depending on the latest version)
|
rc mariadb-server-10.3 1:10.3.23+maria~xenial amd64 MariaDB database server binaries
|
ii mariadb-server-10.5 1:10.5.4+maria~xenial amd64 MariaDB database server binaries
|
ii mariadb-server-core-10.5 1:10.5.4+maria~xenial amd64 MariaDB database core server files
|
ii mysql-common 1:10.3.23+maria~xenial all MariaDB database common files (e.g. /etc/mysql/my.cnf)
|
ii python-mysqldb 1.3.7-1build2 amd64 Python interface to MySQL
|
Server has been upgraded:
elenst@xenial:~$ sudo systemctl status mariadb
|
● mariadb.service - MariaDB 10.5.4 database server
|
Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
|
Drop-In: /etc/systemd/system/mariadb.service.d
|
└─migrated-from-my.cnf-settings.conf
|
Active: active (running) since ke 2020-06-17 18:04:56 EEST; 11min ago
|
And even Columnstore is there:
elenst@xenial:~$ mysql -uroot -e "select * from information_schema.plugins where plugin_name like 'columnstore'"
|
+-------------+----------------+---------------+----------------+---------------------+-------------------+------------------------+---------------------+----------------------------+----------------+-------------+-----------------+---------------------+
|
| PLUGIN_NAME | PLUGIN_VERSION | PLUGIN_STATUS | PLUGIN_TYPE | PLUGIN_TYPE_VERSION | PLUGIN_LIBRARY | PLUGIN_LIBRARY_VERSION | PLUGIN_AUTHOR | PLUGIN_DESCRIPTION | PLUGIN_LICENSE | LOAD_OPTION | PLUGIN_MATURITY | PLUGIN_AUTH_VERSION |
|
+-------------+----------------+---------------+----------------+---------------------+-------------------+------------------------+---------------------+----------------------------+----------------+-------------+-----------------+---------------------+
|
| Columnstore | 1.5 | ACTIVE | STORAGE ENGINE | 100504.0 | ha_columnstore.so | 1.14 | MariaDB Corporation | ColumnStore storage engine | GPL | ON | Gamma | 1.5.0 |
|
+-------------+----------------+---------------+----------------+---------------------+-------------------+------------------------+---------------------+----------------------------+----------------+-------------+-----------------+---------------------+
|
But it doesn't work:
elenst@xenial:~$ mysql -uroot -e "create database test; use test; create table t1 (a int) engine=Columnstore"
|
ERROR 1815 (HY000) at line 1: Internal error: Cannot execute the statement. DBRM is read only!
|
And the services aren't there, so it really didn't install properly at all:
elenst@xenial:~$ sudo systemctl | grep -Ei 'mariadb|column|mcs'
|
mariadb.service loaded active running MariaDB 10.5.4 database server
|
apt-get install -f doesn't work either:
elenst@xenial:~$ sudo apt-get install -f
|
Reading package lists... Done
|
Building dependency tree
|
Reading state information... Done
|
The following package was automatically installed and is no longer required:
|
snapd-login-service
|
Use 'sudo apt autoremove' to remove it.
|
0 upgraded, 0 newly installed, 0 to remove and 6 not upgraded.
|
1 not fully installed or removed.
|
After this operation, 0 B of additional disk space will be used.
|
Setting up mariadb-plugin-columnstore (1:10.5.4+maria~xenial) ...
|
MySQL Password file missing or incorrect, check .my.cnf file
|
dpkg: error processing package mariadb-plugin-columnstore (--configure):
|
subprocess installed post-installation script returned error exit status 2
|
Errors were encountered while processing:
|
mariadb-plugin-columnstore
|
E: Sub-process /usr/bin/dpkg returned an error code (1)
|
Note: Same happens if I run apt-get install mariadb-server mariadb-client mariadb-plugin-columnstore on top of the previous version; so it's not specific to Columnstore forcing the server upgrade.
Uninstallation works, other than the notorious rpmsave:
elenst@xenial:~$ sudo apt-get remove mariadb-plugin-columnstore
|
Reading package lists... Done
|
Building dependency tree
|
Reading state information... Done
|
The following packages were automatically installed and are no longer required:
|
libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libjemalloc1 libsnappy1v5 python-mysqldb snapd-login-service
|
Use 'sudo apt autoremove' to remove them.
|
The following packages will be REMOVED:
|
mariadb-plugin-columnstore
|
0 upgraded, 0 newly installed, 1 to remove and 6 not upgraded.
|
1 not fully installed or removed.
|
After this operation, 32,3 MB disk space will be freed.
|
Do you want to continue? [Y/n] y
|
(Reading database ... 213684 files and directories currently installed.)
|
Removing mariadb-plugin-columnstore (1:10.5.4+maria~xenial) ...
|
|
Mariab Columnstore uninstall completed
|
elenst@xenial:~$ dpkg -l | grep -i columnstore
|
rc mariadb-plugin-columnstore 1:10.5.4+maria~xenial amd64 MariaDB ColumnStore storage engine
|
elenst@xenial:~$ sudo apt-get purge mariadb-plugin-columnstore
|
Reading package lists... Done
|
Building dependency tree
|
Reading state information... Done
|
The following packages were automatically installed and are no longer required:
|
libboost-atomic1.58.0 libboost-chrono1.58.0 libboost-regex1.58.0 libboost-thread1.58.0 libjemalloc1 libsnappy1v5 python-mysqldb snapd-login-service
|
Use 'sudo apt autoremove' to remove them.
|
The following packages will be REMOVED:
|
mariadb-plugin-columnstore*
|
0 upgraded, 0 newly installed, 1 to remove and 6 not upgraded.
|
After this operation, 0 B of additional disk space will be used.
|
Do you want to continue? [Y/n] y
|
(Reading database ... 213527 files and directories currently installed.)
|
Removing mariadb-plugin-columnstore (1:10.5.4+maria~xenial) ...
|
Purging configuration files for mariadb-plugin-columnstore (1:10.5.4+maria~xenial) ...
|
elenst@xenial:~$ ls -l /var/lib/mysql/
|
total 122916
|
-rw-rw---- 1 mysql mysql 16384 kesä 17 18:29 aria_log.00000001
|
-rw-rw---- 1 mysql mysql 52 kesä 17 18:29 aria_log_control
|
-rw-r--r-- 1 root root 0 kesä 17 18:04 debian-10.5.flag
|
-rw-rw---- 1 mysql mysql 976 kesä 17 18:29 ib_buffer_pool
|
-rw-rw---- 1 mysql mysql 12582912 kesä 17 18:29 ibdata1
|
-rw-rw---- 1 mysql mysql 100663296 kesä 17 18:29 ib_logfile0
|
-rw-rw---- 1 mysql mysql 12582912 kesä 17 18:29 ibtmp1
|
-rw-rw---- 1 mysql mysql 0 kesä 17 18:01 multi-master.info
|
drwx------ 2 mysql mysql 4096 kesä 17 18:01 mysql
|
drwx------ 2 mysql mysql 4096 kesä 17 18:01 performance_schema
|
drwx------ 2 mysql mysql 4096 kesä 17 18:19 test
|
elenst@xenial:~$ ls -l /etc/mysql/mariadb.conf.d/
|
total 24
|
-rw-r--r-- 1 root root 677 kesä 16 17:15 50-client.cnf
|
-rw-r--r-- 1 root root 231 kesä 16 17:15 50-mysql-clients.cnf
|
-rw-r--r-- 1 root root 917 kesä 16 17:15 50-mysqld_safe.cnf
|
-rw-r--r-- 1 root root 3635 kesä 16 17:15 50-server.cnf
|
-rw-r--r-- 1 root root 417 kesä 16 17:15 60-galera.cnf
|
-rw-r--r-- 1 root root 503 kesä 17 18:31 columnstore.cnf.rpmsave
|
For a baseline, I performed the same exercise with RocksDB. I won't paste the whole log; the summary is, it works better, although not perfectly. It also pulls the matching server with it, upgrade goes all right and ends without errors, the new server is running after upgrade, but RocksDB is not loaded, the server needs to be restarted to pick it up. It's a nuisance, but acceptable for a major upgrade.
|
|
I'll follow your steps and fix the major upgrade issues.
|
RPM (installation by YUM from local RPMs) [FAILED] for Gamma, [PASSED with notes] for Beta.
Clean simultaneous installation of server, client and ColumnStore
Packages from build-33595/kvm-rpm-centos74-amd64, CentOS 7.8.2003
$ sudo yum install MariaDB-server-10.5.4-1.el7.centos.x86_64.rpm MariaDB-client-10.5.4-1.el7.centos.x86_64.rpm MariaDB-common-10.5.4-1.el7.centos.x86_64.rpm MariaDB-shared-10.5.4-1.el7.centos.x86_64.rpm MariaDB-compat-10.5.4-1.el7.centos.x86_64.rpm galera-4-26.4.4-1.rhel7.el7.centos.x86_64.rpm MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64.rpm
|
The transaction check works.
=======================================================================================
|
Package Arch Version Repository Size
|
=======================================================================================
|
Installing:
|
MariaDB-client x86_64 10.5.4-1.el7.centos /MariaDB-client-10.5.4-1.el7.centos.x86_64
|
65 M
|
MariaDB-columnstore-engine x86_64 10.5.4-1.el7.centos /MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64
|
29 M
|
MariaDB-common x86_64 10.5.4-1.el7.centos /MariaDB-common-10.5.4-1.el7.centos.x86_64
|
303 k
|
MariaDB-compat x86_64 10.5.4-1.el7.centos /MariaDB-compat-10.5.4-1.el7.centos.x86_64
|
11 M
|
replacing mariadb-libs.x86_64 1:5.5.65-1.el7
|
MariaDB-server x86_64 10.5.4-1.el7.centos /MariaDB-server-10.5.4-1.el7.centos.x86_64
|
129 M
|
MariaDB-shared x86_64 10.5.4-1.el7.centos /MariaDB-shared-10.5.4-1.el7.centos.x86_64
|
343 k
|
galera-4 x86_64 26.4.4-1.rhel7.el7.centos /galera-4-26.4.4-1.rhel7.el7.centos.x86_64
|
40 M
|
Installing for dependencies:
|
boost-atomic x86_64 1.53.0-28.el7 base 35 k
|
boost-chrono x86_64 1.53.0-28.el7 base 44 k
|
boost-filesystem x86_64 1.53.0-28.el7 base 68 k
|
boost-program-options x86_64 1.53.0-28.el7 base 156 k
|
boost-regex x86_64 1.53.0-28.el7 base 296 k
|
expect x86_64 5.45-14.el7_1 base 262 k
|
tcl x86_64 1:8.5.13-8.el7 base 1.9 M
|
|
Transaction Summary
|
=======================================================================================
|
Install 7 Packages (+7 Dependent packages)
|
|
Total size: 278 M
|
It succeeded without errors, the same Columnstore check as with debs on the way:
Installing : MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64 14/15
|
checking for engine columnstore...
|
Columnstore YES ColumnStore storage engine YES NO NO
|
columnstore exist
|
[elenst@localhost rpms]$ sudo systemctl start mariadb
|
[elenst@localhost rpms]$ sudo mysql -e "show engines"
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Columnstore | YES | ColumnStore storage engine | YES | NO | NO |
|
...
|
[elenst@localhost rpms]$ sudo mysql test -e "create table t1 (a int) engine=Columnstore; insert into t1 select seq from seq_1_to_10000; select count(*) from t1"
|
ERROR 1815 (HY000) at line 1: Internal error: CAL0009: Error while calling getSysCatDBRoot
|
Doesn't work, but maybe if the server doesn't get started automatically, Columnstore doesn't either? It would be fair.
[elenst@localhost rpms]$ sudo systemctl | grep -iE 'mariadb|mcs'
|
mariadb-columnstore.service loaded active exited mariadb-columnstore
|
mariadb.service loaded active running MariaDB 10.5.4 database server
|
mcs-controllernode.service loaded active running mcs-controllernode
|
mcs-ddlproc.service loaded active running mcs-ddlproc
|
mcs-dmlproc.service loaded active running mcs-dmlproc
|
mcs-exemgr.service loaded active running mcs-exemgr
|
mcs-primproc.service loaded active running mcs-primproc
|
● mcs-storagemanager.service loaded failed failed storagemanager
|
mcs-workernode.service loaded active running mcs-workernode
|
mcs-writeengineserver.service loaded active running WriteEngineServer
|
It looks like it's running though. Let's restart everything anyway.
[elenst@localhost rpms]$ sudo systemctl restart mariadb-columnstore mariadb
|
[elenst@localhost rpms]$ sudo mysql test -e "create table t1 (a int) engine=Columnstore; insert into t1 select seq from seq_1_to_10000; select count(*) from t1"
|
ERROR 1815 (HY000) at line 1: Internal error: CAL0009: Error while calling getSysCatDBRoot
|
Still doesn't work.
|
/var/log/mariadb/columnstore/err.log
|
Jun 17 17:46:51 localhost controllernode[4924]: 51.111809 |0|0|0| E 29 CAL0000: DBRM: error: SessionManager::setSystemState() failed (network)
|
Jun 17 17:46:52 localhost writeengine[4950]: 52.269898 |0|0|0| C 19 CAL0060: dbbuilder system catalog error: Creating TableName column OID: 1001 a BRM Allocate extent error. [BRM error status: network error]
|
Jun 17 17:46:53 localhost DMLProc[4963]: 53.205081 |0|0|0| C 20 CAL0002: DMLProc failed to start due to : Rollback will be deferred due to DBRM is in read only state.
|
Jun 17 17:50:29 localhost ddlpackageproc[4964]: 29.466021 |0|0|0| E 23 CAL0009: Error while calling getSysCatDBRoot
|
Jun 17 17:53:46 localhost controllernode[5353]: 46.917015 |0|0|0| C 29 CAL0000: ExtentMap::save(): got request to save an empty BRM
|
Jun 17 17:54:06 localhost ddlpackageproc[5468]: 06.097911 |0|0|0| E 23 CAL0009: Error while calling getSysCatDBRoot
|
|
/var/log/mariadb/columnstore/debug.log
|
Jun 17 17:46:42 localhost oamcpp[4460]: 42.514946 |0|0|0| I 08 CAL0019: CMD ***** MariaDB Columnstore Installed *****
|
Jun 17 17:46:51 localhost controllernode[4924]: 51.111809 |0|0|0| E 29 CAL0000: DBRM: error: SessionManager::setSystemState() failed (network)
|
Jun 17 17:46:52 localhost writeengine[4950]: 52.269898 |0|0|0| C 19 CAL0060: dbbuilder system catalog error: Creating TableName column OID: 1001 a BRM Allocate extent error. [BRM error status: network error]
|
Jun 17 17:46:53 localhost DMLProc[4963]: 53.204080 |0|0|0| I 20 CAL0002: DMLProc starts rollbackAll.
|
Jun 17 17:46:53 localhost DMLProc[4963]: 53.205081 |0|0|0| C 20 CAL0002: DMLProc failed to start due to : Rollback will be deferred due to DBRM is in read only state.
|
Jun 17 17:46:53 localhost IDBFile[4890]: 53.401109 |0|0|0| D 35 CAL0002: Failed to open file: /var/lib/columnstore/data1/systemFiles/dbrm/tablelocks, exception: unable to open Buffered file
|
Jun 17 17:46:53 localhost controllernode[4890]: 53.401914 |0|0|0| D 29 CAL0000: TableLockServer::load(): could not open the save file/var/lib/columnstore/data1/systemFiles/dbrm/tablelocks
|
Jun 17 17:46:55 localhost DMLProc[4974]: 55.524588 |0|0|0| I 20 CAL0002: DMLProc starts rollbackAll.
|
Jun 17 17:46:57 localhost DMLProc[4974]: 57.630145 |0|0|0| I 20 CAL0002: DMLProc will rollback 0 tables.
|
Jun 17 17:46:57 localhost DMLProc[4974]: 57.643446 |0|0|0| I 20 CAL0002: DMLProc finished rollbackAll.
|
Jun 17 17:50:29 localhost ddlpackageproc[4964]: 29.457362 |4|1|0| D 23 CAL0041: Start SQL statement: create table t1 (a int) engine=Columnstore;|test|
|
Jun 17 17:50:29 localhost controllernode[4964]: 29.463395 |0|0|0| W 29 CAL0000: ExtentMap::getSysCatDBRoot(): OID not found: 1001
|
Jun 17 17:50:29 localhost ddlpackageproc[4964]: 29.464887 |4|1|0| D 23 CAL0042: End SQL statement
|
Jun 17 17:50:29 localhost ddlpackageproc[4964]: 29.466021 |0|0|0| E 23 CAL0009: Error while calling getSysCatDBRoot
|
Jun 17 17:53:46 localhost controllernode[5353]: 46.917015 |0|0|0| C 29 CAL0000: ExtentMap::save(): got request to save an empty BRM
|
Jun 17 17:53:51 localhost IDBFile[5397]: 51.628247 |0|0|0| D 35 CAL0002: Failed to open file: /var/lib/columnstore/data1/systemFiles/dbrm/tablelocks, exception: unable to open Buffered file
|
Jun 17 17:53:51 localhost controllernode[5397]: 51.628296 |0|0|0| D 29 CAL0000: TableLockServer::load(): could not open the save file/var/lib/columnstore/data1/systemFiles/dbrm/tablelocks
|
Jun 17 17:53:55 localhost DMLProc[5467]: 55.845135 |0|0|0| I 20 CAL0002: DMLProc starts rollbackAll.
|
Jun 17 17:53:55 localhost DMLProc[5467]: 55.869107 |0|0|0| I 20 CAL0002: DMLProc will rollback 0 tables.
|
Jun 17 17:53:55 localhost DMLProc[5467]: 55.880028 |0|0|0| I 20 CAL0002: DMLProc finished rollbackAll.
|
Jun 17 17:54:06 localhost ddlpackageproc[5468]: 06.091260 |3|2|0| D 23 CAL0041: Start SQL statement: create table t1 (a int) engine=Columnstore;|test|
|
Jun 17 17:54:06 localhost controllernode[5468]: 06.095768 |0|0|0| W 29 CAL0000: ExtentMap::getSysCatDBRoot(): OID not found: 1001
|
Jun 17 17:54:06 localhost ddlpackageproc[5468]: 06.096732 |3|2|0| D 23 CAL0042: End SQL statement
|
Jun 17 17:54:06 localhost ddlpackageproc[5468]: 06.097911 |0|0|0| E 23 CAL0009: Error while calling getSysCatDBRoot
|
All I see here with an untrained eye is "network error", don't know what it means in this context. My network works.
PING mariadb.com (104.20.67.208) 56(84) bytes of data.
|
64 bytes from 104.20.67.208 (104.20.67.208): icmp_seq=1 ttl=63 time=10.3 ms
|
64 bytes from 104.20.67.208 (104.20.67.208): icmp_seq=2 ttl=63 time=14.3 ms
|
64 bytes from 104.20.67.208 (104.20.67.208): icmp_seq=3 ttl=63 time=11.7 ms
|
There is also something about not being able to write a file, so just in case:
[elenst@localhost rpms]$ ls -l /var/lib/columnstore/
|
total 0
|
drwxr-xr-x. 3 root root 18 Jun 17 17:46 data
|
drwxr-xr-t. 4 root root 40 Jun 17 17:46 data1
|
drwxr-xr-x. 2 root root 20 Jun 17 17:46 local
|
[elenst@localhost rpms]$ ls -l /var/lib/columnstore/data1/
|
total 0
|
drwxr-xr-x. 3 root root 21 Jun 17 17:46 000.dir
|
drwxr-xr-t. 3 root root 18 Jun 17 17:46 systemFiles
|
[elenst@localhost rpms]$ ls -l /var/lib/columnstore/data1/systemFiles/
|
total 0
|
drwxr-xr-t. 2 root root 63 Jun 17 17:46 dbrm
|
[elenst@localhost rpms]$ ls -l /var/lib/columnstore/data1/systemFiles/dbrm/
|
total 2060
|
-rw-r--r--. 1 root root 10 Jun 17 17:53 BRM_saves_journal
|
-rw-rw-r--. 1 root root 2099202 Jun 17 17:54 oidbitmap
|
-rw-r--r--. 1 root root 12 Jun 17 17:54 SMTxnID
|
[elenst@localhost rpms]$ sudo systemctl stop mariadb-columnstore mariadb
|
[elenst@localhost rpms]$ sudo systemctl start mariadb-columnstore
|
[elenst@localhost rpms]$ sudo tail -n 100 /var/log/messages
|
...
|
Jun 17 18:01:56 localhost systemd: Stopping mariadb-columnstore...
|
Jun 17 18:01:56 localhost systemd: Stopping mcs-dmlproc...
|
Jun 17 18:01:56 localhost systemd: Stopped mcs-dmlproc.
|
Jun 17 18:01:56 localhost systemd: Stopping mcs-ddlproc...
|
Jun 17 18:01:56 localhost systemd: Stopped mcs-ddlproc.
|
Jun 17 18:01:56 localhost systemd: Stopping WriteEngineServer...
|
Jun 17 18:01:56 localhost systemd: Stopped WriteEngineServer.
|
Jun 17 18:01:56 localhost systemd: Stopping mcs-exemgr...
|
Jun 17 18:01:56 localhost systemd: Stopped mcs-exemgr.
|
Jun 17 18:01:56 localhost systemd: Stopping mcs-primproc...
|
Jun 17 18:01:56 localhost systemd: Stopped mcs-primproc.
|
Jun 17 18:01:56 localhost systemd: Stopping mcs-controllernode...
|
Jun 17 18:01:56 localhost controllernode: DBRM Controller: Waiting for threads to finish...
|
Jun 17 18:01:56 localhost controllernode: Exiting...
|
Jun 17 18:01:56 localhost systemd: Stopped mcs-controllernode.
|
Jun 17 18:01:56 localhost systemd: Stopping mcs-workernode...
|
Jun 17 18:01:56 localhost controllernode[6053]: 56.649132 |0|0|0| C 29 CAL0000: ExtentMap::save(): got request to save an empty BRM
|
Jun 17 18:01:56 localhost save_brm: ExtentMap::save(): got request to save an empty BRM
|
Jun 17 18:01:56 localhost save_brm: Save failed
|
Jun 17 18:01:57 localhost systemd: Stopped mcs-workernode.
|
Jun 17 18:01:57 localhost systemd: Stopped mariadb-columnstore.
|
Jun 17 18:02:00 localhost systemd: Starting mariadb-columnstore...
|
Jun 17 18:02:00 localhost systemd: Starting storagemanager...
|
Jun 17 18:02:00 localhost systemd: mcs-storagemanager.service: control process exited, code=exited status=1
|
Jun 17 18:02:00 localhost systemd: Failed to start storagemanager.
|
Jun 17 18:02:00 localhost systemd: Unit mcs-storagemanager.service entered failed state.
|
Jun 17 18:02:00 localhost systemd: mcs-storagemanager.service failed.
|
Jun 17 18:02:00 localhost mariadb-columnstore-start.sh: Job for mcs-storagemanager.service failed because the control process exited with error code. See "systemctl status mcs-storagemanager.service" and "journalctl -xe" for details.
|
Jun 17 18:02:00 localhost systemd: Starting loadbrm...
|
Jun 17 18:02:00 localhost env: cat: /var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves_current: No such file or directory
|
Jun 17 18:02:00 localhost env: /var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves_current does not exist.
|
Jun 17 18:02:02 localhost systemd: Started loadbrm.
|
Jun 17 18:02:02 localhost systemd: Starting storagemanager...
|
Jun 17 18:02:02 localhost systemd: mcs-storagemanager.service: control process exited, code=exited status=1
|
Jun 17 18:02:02 localhost systemd: Failed to start storagemanager.
|
Jun 17 18:02:02 localhost systemd: Unit mcs-storagemanager.service entered failed state.
|
Jun 17 18:02:02 localhost systemd: mcs-storagemanager.service failed.
|
Jun 17 18:02:02 localhost systemd: Starting loadbrm...
|
Jun 17 18:02:02 localhost env: cat: /var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves_current: No such file or directory
|
Jun 17 18:02:02 localhost env: /var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves_current does not exist.
|
Jun 17 18:02:04 localhost systemd: Started loadbrm.
|
Jun 17 18:02:04 localhost systemd: Started mcs-workernode.
|
Jun 17 18:02:04 localhost systemd: Started mcs-controllernode.
|
Jun 17 18:02:04 localhost systemd: Starting mcs-primproc...
|
Jun 17 18:02:04 localhost env: Starting PrimitiveServer: st = 1, sq = 10, pw = 128, pq = 10240, nb = 500573, nt = 2, nc = 1, ra = 512, db = 128, mb = 512, rd = 0, tr = 0, ss = 67108864, bp = 2
|
Jun 17 18:02:06 localhost systemd: Started mcs-primproc.
|
Jun 17 18:02:06 localhost systemd: Started WriteEngineServer.
|
Jun 17 18:02:06 localhost systemd: Started mcs-exemgr.
|
Jun 17 18:02:06 localhost systemd: Started mcs-dmlproc.
|
Jun 17 18:02:06 localhost systemd: Started mcs-ddlproc.
|
Jun 17 18:02:06 localhost systemd: Started mariadb-columnstore.
|
Jun 17 18:02:06 localhost env: WriteEngineServer is ready
|
Jun 17 18:02:06 localhost env: Starting ExeMgr: st = 50, qs = 20, mx = 95, cf = /etc/columnstore/Columnstore.xml
|
Jun 17 18:02:08 localhost DMLProc[6177]: 08.997327 |0|0|0| I 20 CAL0002: DMLProc starts rollbackAll.
|
Jun 17 18:02:09 localhost env: DDLProc is ready...
|
Jun 17 18:02:09 localhost DMLProc[6177]: 09.022187 |0|0|0| I 20 CAL0002: DMLProc will rollback 0 tables.
|
Jun 17 18:02:09 localhost DMLProc[6177]: 09.035105 |0|0|0| I 20 CAL0002: DMLProc finished rollbackAll.
|
Jun 17 18:02:09 localhost env: DMLProc is ready...
|
[elenst@localhost rpms]$ sudo mysql test -e "create table t1 (a int) engine=Columnstore; insert into t1 select seq from seq_1_to_10000; select count(*) from t1"
|
ERROR 1815 (HY000) at line 1: Internal error: CAL0009: Error while calling getSysCatDBRoot
|
It has already failed, but I'll also try to reboot.
[elenst@localhost rpms]$ sudo systemctl enable mariadb
|
Created symlink from /etc/systemd/system/multi-user.target.wants/mariadb.service to /usr/lib/systemd/system/mariadb.service.
|
[elenst@localhost rpms]$ sudo reboot now
|
[elenst@localhost ~]$ sudo mysql test -e "create table t1 (a int) engine=Columnstore; insert into t1 select seq from seq_1_to_10000; select count(*) from t1"
|
ERROR 1815 (HY000) at line 1: Internal error: CAL0009: Error while calling getSysCatDBRoot
|
No, still doesn't work.
Test aborted
|
RPM (installation by YUM from local RPMs) [FAILED] for Gamma, [PASSED with notes] for Beta
Clean simultaneous installation of server, client and ColumnStore
Packages from build-33618/kvm-rpm-centos74-amd64, CentOS 7.8.2003
$ sudo yum install MariaDB-server-10.5.4-1.el7.centos.x86_64.rpm MariaDB-client-10.5.4-1.el7.centos.x86_64.rpm MariaDB-common-10.5.4-1.el7.centos.x86_64.rpm MariaDB-shared-10.5.4-1.el7.centos.x86_64.rpm MariaDB-compat-10.5.4-1.el7.centos.x86_64.rpm galera-4-26.4.4-1.rhel7.el7.centos.x86_64.rpm MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64.rpm
|
The transaction check works.
=======================================================================================
|
Package Arch Version Repository Size
|
=======================================================================================
|
Installing:
|
MariaDB-client x86_64 10.5.4-1.el7.centos /MariaDB-client-10.5.4-1.el7.centos.x86_64
|
65 M
|
MariaDB-columnstore-engine x86_64 10.5.4-1.el7.centos /MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64
|
29 M
|
MariaDB-common x86_64 10.5.4-1.el7.centos /MariaDB-common-10.5.4-1.el7.centos.x86_64
|
303 k
|
MariaDB-compat x86_64 10.5.4-1.el7.centos /MariaDB-compat-10.5.4-1.el7.centos.x86_64
|
11 M
|
replacing mariadb-libs.x86_64 1:5.5.65-1.el7
|
MariaDB-server x86_64 10.5.4-1.el7.centos /MariaDB-server-10.5.4-1.el7.centos.x86_64
|
129 M
|
MariaDB-shared x86_64 10.5.4-1.el7.centos /MariaDB-shared-10.5.4-1.el7.centos.x86_64
|
343 k
|
galera-4 x86_64 26.4.4-1.rhel7.el7.centos /galera-4-26.4.4-1.rhel7.el7.centos.x86_64
|
40 M
|
Installing for dependencies:
|
boost-atomic x86_64 1.53.0-28.el7 base 35 k
|
boost-chrono x86_64 1.53.0-28.el7 base 44 k
|
boost-filesystem x86_64 1.53.0-28.el7 base 68 k
|
boost-program-options x86_64 1.53.0-28.el7 base 156 k
|
boost-regex x86_64 1.53.0-28.el7 base 296 k
|
expect x86_64 5.45-14.el7_1 base 262 k
|
tcl x86_64 1:8.5.13-8.el7 base 1.9 M
|
|
Transaction Summary
|
=======================================================================================
|
Install 7 Packages (+7 Dependent packages)
|
|
Total size: 278 M
|
Total download size: 2.7 M
|
During installation process:
checking for engine columnstore...
|
Columnstore YES ColumnStore storage engine YES NO NO
|
columnstore exist
|
Otherwise ends uneventfully.
[elenst@localhost rpms]$ sudo systemctl start mariadb
|
[elenst@localhost rpms]$ sudo mysql -e "show engines"
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Columnstore | YES | ColumnStore storage engine | YES | NO | NO |
|
...
|
[elenst@localhost rpms]$ sudo mysql -e "create database if not exists test; use test; create table t1 (a int) engine=Columnstore; insert into t1 select seq from seq_1_to_10000; select count(*) from t1"
|
+----------+
|
| count(*) |
|
+----------+
|
| 10000 |
|
+----------+
|
[elenst@localhost rpms]$ sudo mysql -e "select * from information_schema.plugins where plugin_name = 'columnstore'\G"
|
*************************** 1. row ***************************
|
PLUGIN_NAME: Columnstore
|
PLUGIN_VERSION: 1.5
|
PLUGIN_STATUS: ACTIVE
|
PLUGIN_TYPE: STORAGE ENGINE
|
PLUGIN_TYPE_VERSION: 100504.0
|
PLUGIN_LIBRARY: ha_columnstore.so
|
PLUGIN_LIBRARY_VERSION: 1.14
|
PLUGIN_AUTHOR: MariaDB Corporation
|
PLUGIN_DESCRIPTION: ColumnStore storage engine
|
PLUGIN_LICENSE: GPL
|
LOAD_OPTION: ON
|
PLUGIN_MATURITY: Gamma
|
PLUGIN_AUTH_VERSION: 1.5.2
|
It's still Gamma, probably should be reconsidered.
[elenst@localhost rpms]$ sudo systemctl | grep -iE 'maria|columnstore|mcs'
|
mariadb-columnstore.service loaded active exited mariadb-columnstore
|
mariadb.service loaded active running MariaDB 10.5.4 database server
|
mcs-controllernode.service loaded active running mcs-controllernode
|
mcs-ddlproc.service loaded active running mcs-ddlproc
|
mcs-dmlproc.service loaded active running mcs-dmlproc
|
mcs-exemgr.service loaded active running mcs-exemgr
|
mcs-primproc.service loaded active running mcs-primproc
|
● mcs-storagemanager.service loaded failed failed storagemanager
|
mcs-workernode.service loaded active running mcs-workernode
|
mcs-writeengineserver.service loaded active running WriteEngineServer
|
storagemanager has alerady been discussed, otherwise it looks normal.
[elenst@localhost rpms]$ sudo systemctl restart mariadb
|
|
[elenst@localhost rpms]$ sudo mysql test -e "select * from t1 limit 1; create table t2 (a int) engine=Columnstore; insert into t2 values (1),(2); select * from t2"
|
+------+
|
| a |
|
+------+
|
| 1 |
|
+------+
|
+------+
|
| a |
|
+------+
|
| 1 |
|
| 2 |
|
+------+
|
Still works afer MariaDB server (service) restart.
[elenst@localhost rpms]$ sudo systemctl restart mariadb-columnstore
|
[elenst@localhost rpms]$
|
[elenst@localhost rpms]$ sudo mysql test -e "drop table t1; select count(*) from t2; create table t3 (c varchar(8)) engine=Columnstore; insert into t3 values ('foo'),('bar'); update t3 set c = 'baz'; select * from t3"
|
+----------+
|
| count(*) |
|
+----------+
|
| 2 |
|
+----------+
|
+------+
|
| c |
|
+------+
|
| baz |
|
| baz |
|
+------+
|
[elenst@localhost rpms]$ sudo mysql test -e "show tables"
|
+----------------+
|
| Tables_in_test |
|
+----------------+
|
| t2 |
|
| t3 |
|
+----------------+
|
Works after mariadb-columnstore service restart, too.
And even after reboot:
[elenst@localhost rpms]$ sudo systemctl enable mariadb
|
Created symlink from /etc/systemd/system/multi-user.target.wants/mariadb.service to /usr/lib/systemd/system/mariadb.service.
|
[elenst@localhost rpms]$ sudo reboot now
|
[elenst@localhost ~]$ sudo mysql test -e "select count(*) from t3; create or replace table t4 (a int, c varchar(8)) engine=Columnstore; insert into t4 select seq, concat('seq',seq) from seq_1_to_1000; select count(distinct a) from t4"
|
+----------+
|
| count(*) |
|
+----------+
|
| 2 |
|
+----------+
|
ERROR 1178 (42000) at line 1: The storage engine for the table doesn't support The syntax or the data type(s) is not supported by Columnstore. Please check the Columnstore syntax guide for supported syntax or data types.
|
That's a badly formed error message. I get the idea, though.
[elenst@localhost ~]$ sudo mysql test -e "select count(*) from t3; drop table if exists t4; create table t4 (a int, c varchar(8)) engine=Columnstore; insert into t4 select seq, concat('seq',seq) from seq_1_to_1000; select count(distinct a) from t4"
|
+----------+
|
| count(*) |
|
+----------+
|
| 2 |
|
+----------+
|
+-------------------+
|
| count(distinct a) |
|
+-------------------+
|
| 1000 |
|
+-------------------+
|
Still works.
Uninstallation
[elenst@localhost ~]$ sudo mysql test -e "drop table t3; drop table t4"
|
[elenst@localhost ~]$ sudo mysql test -e "show tables"
|
|
[elenst@localhost ~]$ sudo yum remove MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64
|
Loaded plugins: fastestmirror, langpacks
|
Resolving Dependencies
|
--> Running transaction check
|
---> Package MariaDB-columnstore-engine.x86_64 0:10.5.4-1.el7.centos will be erased
|
--> Finished Dependency Resolution
|
base/7/x86_64 | 3.6 kB 00:00:00
|
extras/7/x86_64 | 2.9 kB 00:00:00
|
updates/7/x86_64 | 2.9 kB 00:00:00
|
|
Dependencies Resolved
|
|
=======================================================================================
|
Package Arch Version Repository Size
|
=======================================================================================
|
Removing:
|
MariaDB-columnstore-engine
|
x86_64 10.5.4-1.el7.centos
|
@/MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64 29 M
|
|
Transaction Summary
|
=======================================================================================
|
Remove 1 Package
|
|
Installed size: 29 M
|
Is this ok [y/N]: y
|
Downloading packages:
|
Running transaction check
|
Running transaction test
|
Transaction test succeeded
|
Running transaction
|
|
Mariab Columnstore uninstall completed
|
Erasing : MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64 1/1
|
Verifying : MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64 1/1
|
|
Removed:
|
MariaDB-columnstore-engine.x86_64 0:10.5.4-1.el7.centos
|
|
Complete!
|
[elenst@localhost ~]$ sudo mysql -e "show engines"
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Columnstore | YES | ColumnStore storage engine | YES | NO | NO |
|
Server didn't get restarted; however, uninstallation of RocksDB (as a baseline) doesn't restart it either, so I can't claim it's a Columnstore-specific problem.
[elenst@localhost rpms]$ sudo systemctl restart mariadb
|
[elenst@localhost rpms]$ sudo mysql -e "show engines"
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| CSV | YES | Stores tables as CSV files | NO | NO | NO |
|
| MRG_MyISAM | YES | Collection of identical MyISAM tables | NO | NO | NO |
|
| MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |
|
| Aria | YES | Crash-safe tables with MyISAM heritage. Used for internal temporary tables and privilege tables | NO | NO | NO |
|
| MyISAM | YES | Non-transactional engine with good performance and small data footprint | NO | NO | NO |
|
| SEQUENCE | YES | Generated tables filled with sequential values | YES | NO | YES |
|
| InnoDB | DEFAULT | Supports transactions, row-level locking, foreign keys and encryption for tables | YES | YES | YES |
|
| PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
Now it's gone.
[elenst@localhost ~]$ sudo find / -name *columnstore*
|
/etc/my.cnf.d/columnstore.cnf.rpmsave
|
/etc/columnstore
|
/etc/columnstore/Columnstore.xml.columnstoreSave
|
/var/lib/mysql/columnstore_info
|
/var/lib/columnstore
|
|
[elenst@localhost ~]$ sudo find / -name *Columnstore*
|
/etc/columnstore/Columnstore.xml-20200617
|
/etc/columnstore/Columnstore.xml.columnstoreSave
|
/etc/columnstore/Columnstore.xml.rpmsave
|
[elenst@localhost ~]$ sudo find / -name *mcs*
|
There are remains of Columnstore left, but again, since it's not a purge and I couldn't find anything specific in YUM documentation of what should be preserved and what should be removed, it might be okay.
Installation of ColumnStore on top of pre-installed server
[elenst@localhost rpms]$ rpm -qa | grep -i maria
|
MariaDB-compat-10.5.4-1.el7.centos.x86_64
|
MariaDB-common-10.5.4-1.el7.centos.x86_64
|
MariaDB-server-10.5.4-1.el7.centos.x86_64
|
MariaDB-client-10.5.4-1.el7.centos.x86_64
|
|
[elenst@localhost rpms]$ sudo systemctl status mariadb
|
● mariadb.service - MariaDB 10.5.4 database server
|
Loaded: loaded (/usr/lib/systemd/system/mariadb.service; disabled; vendor preset: disabled)
|
Drop-In: /etc/systemd/system/mariadb.service.d
|
└─migrated-from-my.cnf-settings.conf
|
Active: active (running) since Thu 2020-06-18 10:06:57 EDT; 29s ago
|
...
|
[elenst@localhost rpms]$ sudo yum install MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64.rpm MariaDB-shared-10.5.4-1.el7.centos.x86_64.rpm
|
...
|
Installing:
|
MariaDB-columnstore-engine x86_64 10.5.4-1.el7.centos /MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64
|
29 M
|
MariaDB-shared x86_64 10.5.4-1.el7.centos /MariaDB-shared-10.5.4-1.el7.centos.x86_64
|
343 k
|
Installing for dependencies:
|
boost-atomic x86_64 1.53.0-28.el7 base 35 k
|
boost-chrono x86_64 1.53.0-28.el7 base 44 k
|
boost-filesystem x86_64 1.53.0-28.el7 base 68 k
|
boost-regex x86_64 1.53.0-28.el7 base 296 k
|
expect x86_64 5.45-14.el7_1 base 262 k
|
tcl x86_64 1:8.5.13-8.el7 base 1.9 M
|
|
Transaction Summary
|
=======================================================================================
|
Install 2 Packages (+6 Dependent packages)
|
|
Total size: 32 M
|
Total download size: 2.6 M
|
Installed size: 36 M
|
Installing : MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64 8/8
|
checking for engine columnstore...
|
Columnstore YES ColumnStore storage engine YES NO NO
|
columnstore exist
|
Takes a disturbing pause upon Installing : MariaDB-columnstore-engine..., but ends all right, no errors.
[elenst@localhost rpms]$ sudo mysql -e "show engines"
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Columnstore | YES | ColumnStore storage engine | YES | NO | NO |
|
Columnstore is there right away.
[elenst@localhost rpms]$ sudo mysql -e "create database if not exists test; use test; create table t1 (a int) engine=Columnstore; insert into t1 values (1),(2); show create table t1; select * from t1"
|
+-------+--------------------------------------------------------------------------------------------+
|
| Table | Create Table |
|
+-------+--------------------------------------------------------------------------------------------+
|
| t1 | CREATE TABLE `t1` (
|
`a` int(11) DEFAULT NULL
|
) ENGINE=Columnstore DEFAULT CHARSET=latin1 |
|
+-------+--------------------------------------------------------------------------------------------+
|
+------+
|
| a |
|
+------+
|
| 1 |
|
| 2 |
|
+------+
|
It works, too.
[elenst@localhost rpms]$ sudo systemctl restart mariadb
|
[elenst@localhost rpms]$ sudo mysql test -e "update t1 set a = a+10; select * from t1"
|
+------+
|
| a |
|
+------+
|
| 11 |
|
| 12 |
|
+------+
|
[elenst@localhost rpms]$ sudo systemctl restart mariadb-columnstore
|
[elenst@localhost rpms]$ sudo mysql test -e "update t1 set a = a+10; alter table t1 add b int; select * from t1"
|
+------+------+
|
| a | b |
|
+------+------+
|
| 21 | NULL |
|
| 22 | NULL |
|
+------+------+
|
Still works after restarting server and columnstore services (separately), although first query after Columnstore restart still takes really really long.
[elenst@localhost ~]$ sudo systemctl enable mariadb
|
[elenst@localhost ~]$ sudo reboot now
|
[elenst@localhost ~]$ sudo mysql test -e "update t1 set a = a+10; create table t2 engine=Columnstore as select * from t1; select * from t2"
|
+------+------+
|
| a | b |
|
+------+------+
|
| 31 | NULL |
|
| 32 | NULL |
|
+------+------+
|
Still works.
Looks all right. I don't expect any differences in uninstallation comparing to the simultaneous installation, so I'll skip it.
Installation of ColumnStore after server upgrade
We already know that it doesn't work for debs, but RPMs are different, I want at least to see how it looks here.
[elenst@localhost ~]$ rpm -qa | grep -iE 'maria|mysql'
|
MariaDB-compat-10.3.23-1.el7.centos.x86_64
|
MariaDB-server-10.3.23-1.el7.centos.x86_64
|
MariaDB-client-10.3.23-1.el7.centos.x86_64
|
MariaDB-common-10.3.23-1.el7.centos.x86_64
|
We don't actually support major upgrade as such, so it's removal of the old server and installation of the new one.
[elenst@localhost rpms]$ sudo yum remove MariaDB-server
|
[elenst@localhost rpms]$ sudo yum install galera-4-26.4.4-1.rhel7.el7.centos.x86_64.rpm MariaDB-server-10.5.4-1.el7.centos.x86_64.rpm MariaDB-client-10.5.4-1.el7.centos.x86_64.rpm
|
[elenst@localhost rpms]$ sudo rpm -qa | grep -iE 'mysql|maria'
|
MariaDB-compat-10.3.23-1.el7.centos.x86_64
|
MariaDB-server-10.5.4-1.el7.centos.x86_64
|
MariaDB-client-10.5.4-1.el7.centos.x86_64
|
MariaDB-common-10.3.23-1.el7.centos.x86_64
|
[elenst@localhost rpms]$ sudo systemctl start mariadb
|
[elenst@localhost rpms]$ sudo mysql_upgrade -uroot
|
Phase 1/7: Checking and upgrading mysql database
|
Processing databases
|
mysql
|
mysql.column_stats OK
|
mysql.columns_priv OK
|
mysql.db OK
|
mysql.event OK
|
mysql.func OK
|
mysql.gtid_slave_pos OK
|
mysql.help_category OK
|
mysql.help_keyword OK
|
mysql.help_relation OK
|
mysql.help_topic OK
|
mysql.host OK
|
mysql.index_stats OK
|
mysql.innodb_index_stats OK
|
mysql.innodb_table_stats OK
|
mysql.plugin OK
|
mysql.proc OK
|
mysql.procs_priv OK
|
mysql.proxies_priv OK
|
mysql.roles_mapping OK
|
mysql.servers OK
|
mysql.table_stats OK
|
mysql.tables_priv OK
|
mysql.time_zone OK
|
mysql.time_zone_leap_second OK
|
mysql.time_zone_name OK
|
mysql.time_zone_transition OK
|
mysql.time_zone_transition_type OK
|
mysql.transaction_registry OK
|
mysql.user OK
|
Phase 2/7: Installing used storage engines... Skipped
|
Phase 3/7: Fixing views
|
Phase 4/7: Running 'mysql_fix_privilege_tables'
|
Phase 5/7: Fixing table and database names
|
Phase 6/7: Checking and upgrading tables
|
Processing databases
|
information_schema
|
performance_schema
|
test
|
Phase 7/7: Running 'FLUSH PRIVILEGES'
|
OK
|
[elenst@localhost rpms]$ sudo yum install MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64.rpm
|
|
Installing:
|
MariaDB-columnstore-engine x86_64 10.5.4-1.el7.centos /MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64
|
29 M
|
Installing for dependencies:
|
MariaDB-shared x86_64 10.3.23-1.el7.centos mariadb 113 k
|
boost-atomic x86_64 1.53.0-28.el7 base 35 k
|
boost-chrono x86_64 1.53.0-28.el7 base 44 k
|
boost-filesystem x86_64 1.53.0-28.el7 base 68 k
|
boost-regex x86_64 1.53.0-28.el7 base 296 k
|
expect x86_64 5.45-14.el7_1 base 262 k
|
tcl x86_64 1:8.5.13-8.el7 base 1.9 M
|
|
Transaction Summary
|
=======================================================================================
|
Install 1 Package (+7 Dependent packages)
|
|
Total size: 32 M
|
Total download size: 2.7 M
|
Installed size: 36 M
|
Installing : MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64 8/8
|
checking for engine columnstore...
|
Columnstore YES ColumnStore storage engine YES NO NO
|
columnstore exist
|
[elenst@localhost rpms]$ mysql -e "show engines"
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Engine | Support | Comment | Transactions | XA | Savepoints |
|
+--------------------+---------+-------------------------------------------------------------------------------------------------+--------------+------+------------+
|
| Columnstore | YES | ColumnStore storage engine | YES | NO | NO |
|
[elenst@localhost rpms]$ mysql -e "create database if not exists test; use test; create table t1 (a int) engine=Columnstore; insert into t1 select seq from seq_1_to_100; select max(a) from t1; show create table t1"
|
ERROR 1815 (HY000) at line 1: Internal error: CAL0009: Error while calling getSysCatDBRoot
|
Problem.
[elenst@localhost rpms]$ sudo systemctl | grep -iE 'column|maria|mcs'
|
mariadb-columnstore.service loaded active exited mariadb-columnstore
|
mariadb.service loaded active running MariaDB 10.5.4 database server
|
mcs-controllernode.service loaded active running mcs-controllernode
|
mcs-ddlproc.service loaded active running mcs-ddlproc
|
mcs-dmlproc.service loaded active running mcs-dmlproc
|
mcs-exemgr.service loaded active running mcs-exemgr
|
mcs-primproc.service loaded active running mcs-primproc
|
● mcs-storagemanager.service loaded failed failed storagemanager
|
mcs-workernode.service loaded active running mcs-workernode
|
mcs-writeengineserver.service loaded active running WriteEngineServer
|
[elenst@localhost rpms]$ sudo systemctl restart mariadb mariadb-columnstore
|
[elenst@localhost rpms]$ mysql -e "create database if not exists test; use test; create table t1 (a int) engine=Columnstore; insert into t1 select seq from seq_1_to_100; select max(a) from t1; show create table t1"
|
ERROR 1815 (HY000) at line 1: Internal error: CAL0009: Error while calling getSysCatDBRoot
|
So, it installs here upon an upgraded server, just doesn't work.
It seems different from the Debian problem, but let's see if the workaround meant for Debian magically works here, too.
[elenst@localhost rpms]$ sudo mysql -e "create user mysql@localhost identified via 'unix_socket'; grant all on *.* to mysql@localhost"
|
|
[elenst@localhost rpms]$ sudo -u mysql mysql -e "select current_user(); show grants"
|
+-----------------+
|
| current_user() |
|
+-----------------+
|
| mysql@localhost |
|
+-----------------+
|
+-------------------------------------------------------------------------------+
|
| Grants for mysql@localhost |
|
+-------------------------------------------------------------------------------+
|
| GRANT ALL PRIVILEGES ON *.* TO `mysql`@`localhost` IDENTIFIED VIA unix_socket |
|
+-------------------------------------------------------------------------------+
|
[elenst@localhost rpms]$ sudo systemctl stop mariadb mariadb-columnstore
|
[elenst@localhost rpms]$ sudo systemctl start mariadb-columnstore mariadb
|
[elenst@localhost rpms]$ mysql -e "create database if not exists test; use test; create table t1 (a int) engine=Columnstore; insert into t1 select seq from seq_1_to_100; select max(a) from t1; show create table t1"
|
ERROR 1815 (HY000) at line 1: Internal error: CAL0009: Error while calling getSysCatDBRoot
|
|
/var/log/mariadb/columnstore/debug.log
|
[elenst@localhost rpms]$ sudo tail -11 /var/log/mariadb/columnstore/debug.log
|
Jun 18 10:46:09 localhost ddlpackageproc[5209]: 09.186742 |0|0|0| E 23 CAL0009: Error while calling getSysCatDBRoot
|
Jun 18 10:50:54 localhost controllernode[5408]: 54.311081 |0|0|0| C 29 CAL0000: ExtentMap::save(): got request to save an empty BRM
|
Jun 18 10:51:04 localhost IDBFile[5493]: 04.456233 |0|0|0| D 35 CAL0002: Failed to open file: /var/lib/columnstore/data1/systemFiles/dbrm/tablelocks, exception: unable to open Buffered file
|
Jun 18 10:51:04 localhost controllernode[5493]: 04.456285 |0|0|0| D 29 CAL0000: TableLockServer::load(): could not open the save file/var/lib/columnstore/data1/systemFiles/dbrm/tablelocks
|
Jun 18 10:51:08 localhost DMLProc[5572]: 08.635409 |0|0|0| I 20 CAL0002: DMLProc starts rollbackAll.
|
Jun 18 10:51:08 localhost DMLProc[5572]: 08.666086 |0|0|0| I 20 CAL0002: DMLProc will rollback 0 tables.
|
Jun 18 10:51:08 localhost DMLProc[5572]: 08.679169 |0|0|0| I 20 CAL0002: DMLProc finished rollbackAll.
|
Jun 18 10:51:17 localhost ddlpackageproc[5573]: 17.943751 |4|3|0| D 23 CAL0041: Start SQL statement: create table t1 (a int) engine=Columnstore;|test|
|
Jun 18 10:51:17 localhost controllernode[5573]: 17.949660 |0|0|0| W 29 CAL0000: ExtentMap::getSysCatDBRoot(): OID not found: 1001
|
Jun 18 10:51:17 localhost ddlpackageproc[5573]: 17.952378 |4|3|0| D 23 CAL0042: End SQL statement
|
Jun 18 10:51:17 localhost ddlpackageproc[5573]: 17.954941 |0|0|0| E 23 CAL0009: Error while calling getSysCatDBRoot
|
We still have some old packages, maybe they cause problems for Columnstore.
[elenst@localhost rpms]$ rpm -qa | grep -iE 'maria|mysql'
|
MariaDB-compat-10.3.23-1.el7.centos.x86_64
|
MariaDB-server-10.5.4-1.el7.centos.x86_64
|
MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64
|
MariaDB-client-10.5.4-1.el7.centos.x86_64
|
MariaDB-shared-10.3.23-1.el7.centos.x86_64
|
MariaDB-common-10.3.23-1.el7.centos.x86_64
|
Let's upgrade them all
[elenst@localhost rpms]$ sudo yum install MariaDB-shared-10.5.4-1.el7.centos.x86_64.rpm MariaDB-compat-10.5.4-1.el7.centos.x86_64.rpm MariaDB-common-10.5.4-1.el7.centos.x86_64.rpm
|
...
|
|
[elenst@localhost rpms]$ sudo rpm -qa | grep -iE 'maria|mysql'
|
MariaDB-server-10.5.4-1.el7.centos.x86_64
|
MariaDB-columnstore-engine-10.5.4-1.el7.centos.x86_64
|
MariaDB-compat-10.5.4-1.el7.centos.x86_64
|
MariaDB-client-10.5.4-1.el7.centos.x86_64
|
MariaDB-common-10.5.4-1.el7.centos.x86_64
|
MariaDB-shared-10.5.4-1.el7.centos.x86_64
|
[elenst@localhost rpms]$ sudo systemctl restart mariadb mariadb-columnstore
|
[elenst@localhost rpms]$ mysql -e "create database if not exists test; use test; create table t1 (a int) engine=Columnstore; insert into t1 select seq from seq_1_to_100; select max(a) from t1; show create table t1"
|
ERROR 1815 (HY000) at line 1: Internal error: CAL0009: Error while calling getSysCatDBRoot
|
Didn't help. If there is a workaround for upgrade, it needs to be documented. It should be acceptable for a Beta.
|
|
Thanks for the latest test elenst
Upgrade issues caused by two facts:
- post-install script fails to execute SQL statements with mysql user privileges
- post-install quites early b/c it fails SQL statements and doesn't create initial syscat for MCS
We addresed these issues and changed the way we execute SQL. post-install now continues even if it fails SQL statements so initial syscat is populated in the end. The post-install script also prints a message suggesting the user to fix authentication issues and run the necessary script manually.
I've also added SQL clean up into pre-uninstall scripts to delete MCS data objects.
The next BB build should contain the mentioned fixes for both debian major upgrade scenario and the latest centos 7 scenario.
|
Basic final check before merge
bb-10.5-cs 19d09e49912 build 33657
Platforms
- Ubuntu 16.04 Xenial
- CentOS 7.8
- openSUSE 15.1
Scope of the check
- clean install
- mariadb service restart
- mariadb-columnstore service restart
- reboot
- uninstallation
- upgrade from 10.3
- mysql_upgrade
- mariadb service restart
- mariadb-columnstore service restart
Results
- Installation/upgrade and further operation at least on CentOS and openSUSE can be affected by
MCOL-4086 (CAL0060/ CAL0009) – the scope of the limitation and a workaround need to be documented;
- Installation/upgrade on Ubuntu can be affected by
MCOL-4088 (sporadic crashes of workernode or controllernode) – a workaround needs to be documented;
- Uninstallation on all platforms is affected by
MCOL-4087 (SQL syntax error) – possibly just a cosmetic issue, needs to be checked;
Other notes
MCOL-4090 - presence of columnstore plugin interferes with a local installation of a source build;MCOL-4089 - typo, small cosmetic issue;- I believe a little more complicated installation scenarios don't work quite correctly, e.g. installation of server + plugin => uninstallation of plugin => re-installation of plugin may be causing at least unexpected server shutdown. However, I don't think it makes sense to test those at this point.
|
Basic final check after the merge and fixes
bb-10.5-release 8ad9ef642a build 33657
Platforms:
- Ubuntu 16.04 Xenial
- CentOS 7.8
- openSUSE 15.1
Scope of the check: only clean simultaneous install (server + columnstore + required packages)
Ubuntu: out of two installation attempts, both failed with the symptoms of MCOL-4088 (system error during installation, followed by persistent CAL0009: Error while calling getSysCatDBRoot).
I still haven't heard of a workaround for the problem from the Columnstore team. The best I could do was remove/purge mariadb-plugin-columnstore and install it again. It helped in both cases, although I wouldn't consider it a decent workaround.
CentOS: out of three attempts, all went seemingly fine during installation, however one of three resulted in the persistent CAL0009: Error while calling getSysCatDBRoot, which could be either MCOL-4088 or MCOL-4086, or both. I found logs of crashes similar to MCOL-4088 in the trace folder. Re-installation of the Columnstore package did not help. CAL0009 persisted, even though this time there were no crashes in the trace folder. Total cleanup and re-installation of all MariaDB packages did help, but it's hardly a workaround at all.
openSUSE: single attempt, installation went and ended without any visible problems, but resulted in the persistent CAL0009: Error while calling getSysCatDBRoot error. Re-installation of Columnstore package didn't help. I didn't attempt re-installation of the whole MariaDB instance.
|
Changes in buildbot
Columnstore is now used by buildbot installation and minor upgrade tests in kvm-deb, kvm-rpm and kvm-zyp builders.
Installation tests
Installation tests install all packages built in the current build, including Columnstore.
See limitations below.
If something goes wrong, installation and minor upgrade tests on deb builders attempt to store Columnstore logs for further analysis. It is done on the best-effort basis. If succeded, the logs can be obtained from columnstore_logs file under the failed buildbot step.
After installation is finished, if Columnstore was among packages, installation tests create a Columnstore table, insert a few values, restart mariadb service, read/update the table, restart mariadb-columnstore service, read/update the table. All actions are expected to succeed.
Minor upgrade tests
The server and Columnstore packages of the last released version are installed from the official repo. After installation the test creates a Columnstore table and inserts a few values.
Then upgrade to the current packages is performed, the exact command depends on the package manager. The previously created Columnstore table is accessed and updated.
Limitations:
- due to
MDEV-14622, installation tests on deb-based systems install Columnstore separately, after the server installation is done
- due to
MCOL-4123, installation tests on deb-based x86 (32-bit) systems don't install Columnstore
- due to
MCOL-4120 and other obscure Columnstore problems, minor-upgrade-all test which installs and upgrades all available packages skips Columnstore. Columnstore upgrade test is performed separately, only mariadb-server, mariadb-columnstore and their dependencies are installed and upgraded
- due to
MCOL-4120, extra server (re-)start is performed after upgrade
At the moment there are no known false positives caused by test deficiencies. Till the last push tests were failing sporadically, either due to already reported bugs or to some other problems indistinguishable from already reported bugs. The last push into 10.5 caused upgrade test failures on all systems, from all I see it is not a test problem but a real one.
Additionally, buildbot build from source RPMs is currently broken due to MCOL-4136.
|
ColumnStore_system_problem_on_Xenial.png
columnstore_crash_report
