4.4.0-31-generic #50-Ubuntu SMP

Could not reproduce

root@ubuntu16-pm1:/var/log/mariadb/columnstore# ma getsoft
getsoftwareinfo Tue May 23 16:21:47 2017

Package: mariadb-columnstore-platform
Status: install ok installed
Priority: optional
Section: devel
Installed-Size: 11208
Maintainer: MariaDB Corporation Ab
Architecture: amd64
Version: 1.0.8
Provides: mariadb-columnstore-platform
Depends: expect, libboost-all-dev, mariadb-columnstore-libs
Description: MariaDB Columnstore: a very fast and robust SQL database server

root@ubuntu16-pm1:/var/log/mariadb/columnstore#

from pm1:

root@ubuntu16-pm1:/usr/local/mariadb/columnstore/etc# ll Columnstore.xml
rw-rr- 1 root root 20150 May 23 16:16 Columnstore.xml

mcsadmin distributeconfigfile
distributeconfigfile Tue May 23 16:19:43 2017

Successful Distribution of MariaDB ColumnStore Config File

ll Columnstore.xml
rw-rr- 1 root root 20150 May 23 16:16 Columnstore.xml

tail -10 debug.log
May 23 16:16:59 ubuntu16-pm1 ProcessManager[13813]: 59.304440 |0|0|0| D 17 CAL0000: startMgrProcessThread Exit
May 23 16:17:00 ubuntu16-pm1 ProcessMonitor[13650]: 00.467494 |0|0|0| D 18 CAL0000: statusControl: REQUEST RECEIVED: Set Process pm3/mysqld State = ACTIVE
May 23 16:17:00 ubuntu16-pm1 ProcessMonitor[13650]: 00.467594 |0|0|0| D 18 CAL0000: statusControl: Set Process pm3/mysqld State = ACTIVE PID = 4269
May 23 16:19:43 ubuntu16-pm1 ProcessManager[13813]: 43.949644 |0|0|0| I 17 CAL0000: MSG RECEIVED: Distribute Config File system/Columnstore.xml
May 23 16:19:43 ubuntu16-pm1 ProcessManager[13813]: 43.949720 |0|0|0| D 17 CAL0000: distributeConfigFile called for system file = Columnstore.xml
May 23 16:19:43 ubuntu16-pm1 ProcessManager[13813]: 43.953686 |0|0|0| D 17 CAL0000: sendMsgProcMon: Process module pm2
May 23 16:19:43 ubuntu16-pm1 ProcessManager[13813]: 43.954198 |0|0|0| D 17 CAL0000: pm2 distributeConfigFile success.
May 23 16:19:43 ubuntu16-pm1 ProcessManager[13813]: 43.957891 |0|0|0| D 17 CAL0000: sendMsgProcMon: Process module pm3
May 23 16:19:43 ubuntu16-pm1 ProcessManager[13813]: 43.958569 |0|0|0| D 17 CAL0000: pm3 distributeConfigFile success.
May 23 16:19:43 ubuntu16-pm1 ProcessManager[13813]: 43.958687 |0|0|0| I 17 CAL0000: Distribute Config File Completed system/Columnstore.xml

from pm2 before and after

root@ubuntu16-pm2:/usr/local/mariadb/columnstore/etc# ll Columnstore.xml
rw-rr- 1 root root 20150 May 23 16:16 Columnstore.xml
root@ubuntu16-pm2:/usr/local/mariadb/columnstore/etc# ll Columnstore.xml
rw-rr- 1 root root 20150 May 23 16:19 Columnstore.xml
root@ubuntu16-pm2:/usr/local/mariadb/columnstore/etc#

Environment is using linux HA (corosync/packmaker/drbd) but I was able to get this to happen on single machine. As a work around in the heartbeat agent I added a chmod on xml file before all system call that need to access the file.

#mcsadmin getsystemstatus
getsystemstatus Tue May 23 10:11:24 2017

System columnstore-1

System and Module statuses

Component Status Last Status Change
------------ -------------------------- ------------------------
System ACTIVE Mon May 22 19:24:16 2017

Module pm1 ACTIVE Mon May 22 19:24:13 2017

MariaDB ColumnStore Replication Feature is enabled

#ls -l /usr/local/mariadb/columnstore/etc/Columnstore.xml
rw-rr- 1 root root 18658 May 22 19:23 /usr/local/mariadb/columnstore/etc/Columnstore.xml

1. mcsadmin shutdownsystem y
   shutdownsystem Tue May 23 10:12:26 2017

This command stops the processing of applications on all Modules within the MariaDB ColumnStore System

Checking for active transactions

Stopping System...
Successful stop of System

Shutting Down System...
Successful shutdown of System

#ls -l /usr/local/mariadb/columnstore/etc/Columnstore.xml
rw-r---- 1 root root 18658 May 23 10:12 /usr/local/mariadb/columnstore/etc/Columnstore.xml

1. dpkg --get-selections |grep maria
   mariadb-columnstore-client install
   mariadb-columnstore-common install
   mariadb-columnstore-gssapi-client install
   mariadb-columnstore-gssapi-server install
   mariadb-columnstore-libs install
   mariadb-columnstore-platform install
   mariadb-columnstore-server install
   mariadb-columnstore-shared install
   mariadb-columnstore-storage-engine install

apt-cache showpkg mariadb-columnstore-server
Package: mariadb-columnstore-server
Versions:
1.0.8 (/var/lib/dpkg/status)
Description Language:
File: /var/lib/dpkg/status
MD5: a86517f51cc04598342e48e5300e96f6

Reverse Depends:
Dependencies:
1.0.8 - mariadb-columnstore-common (0 (null)) mariadb-columnstore-client (0 (null)) mariadb (0 (null)) mysql (0 (null)) mysql-server (0 (null)) mysql-server:i386 (0 (null)) mysql-server (0 (null)) mysql-server:i386 (0 (null)) mariadb-galera-server (0 (null)) mysql-ourdelta-server (0 (null)) mysql-server (0 (null)) mysql-server:i386 (0 (null)) mysql-server (0 (null)) mysql-server:i386 (0 (null))
Provides:
1.0.8 - mysql-server (= 1.0.8-1) mysql-server (= 1.0.8-1) mysql-server (= ) msqlormysql (= ) mysql-server (= ) mysql (= ) mariadb (= )

I will try to get time to collect more information but the first time I did the shutdown it did change and the next couple times it did not so results were inconsistent.

Hey Joe, can you check these 2 items..

1. Make sure SELINUX is disabled per the setup guide
2. Check the nmask of the install directory. This might be causing the permissions to change thing they are doing. This is an example from our system:

/usr/local # umask
0022

We did have another customer run into this same issue and it was resolved with them setting there system UMASK to the default of 022. So that is most likely your issue also, if you want to check into that further.

We also documented the setup guide for information related to this:

https://mariadb.com/kb/en/mariadb/preparing-for-columnstore-installation/#system-settings-considerations

We are all set as we added chmod in the HA agent script and it is working fine now. Didn't see the issue in the standalone environment so must be something to do with the agent code we are using. this can be closed

close pre users request. most likely related to UMASK system setting