[MCOL-787] run command to create system tables after startsystem Created: 2017-06-26  Updated: 2017-12-01  Resolved: 2017-09-13

Status: Closed
Project: MariaDB ColumnStore
Component/s: ProcMgr
Affects Version/s: 1.0.10, 1.1.0
Fix Version/s: 1.0.11, 1.1.0

Type: New Feature Priority: Minor
Reporter: David Hill (Inactive) Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Sprint: 2017-13

 Description   

A few times a system has failed to startup correctly because postConfigure didn't run to successful completion due to some install issue. When this happens, user might correct the problem then run "mcsadmin startsystem" to bring the system up. But when it done this way, the System Catalog isn't created and all create table will fail.
This enhancement will allow the system to come up in a functional state if this happens, postConfigure doesn't finished and user then does a startSystem.

If the system catalog already exist when this command is done, nothing is effected. Both ProcMgr and the script checks if the system catalog already exist and it will bypass running the script.

 Comments   
Comment by David Hill (Inactive) [ 2017-06-26 ]

how to test on a fresh install.
when postConfigure gets to the spot where it ask to start system, exit out and run mcsadmin startsystem.. then try to create tables.

===== MariaDB ColumnStore System Startup =====

System Installation is complete. If any part of the install failed,
the problem should be investigated and resolved before continuing.

Would you like to startup the MariaDB ColumnStore System? [y,n] > n

You choose not to Start the MariaDB ColumnStore Software at this time.
[root@ip-172-30-0-176 bin]# mcsadmin startsystem

Comment by David Hill (Inactive) [ 2017-06-26 ]

this will show up in the pm1 debug logs

Jun 26 14:53:42 ip-172-30-0-176 ProcessManager[9439]: 42.996536 |0|0|0| D 17 CAL0000: Set System State = ACTIVE
Jun 26 14:53:42 ip-172-30-0-176 ProcessMonitor[9287]: 42.996758 |0|0|0| D 18 CAL0000: statusControl: REQUEST RECEIVED: Set System State = ACTIVE
Jun 26 14:53:43 ip-172-30-0-176 ProcessManager[9439]: 43.001794 |0|0|0| D 17 CAL0000: sendMsgProcMon: Process module pm1
Jun 26 14:53:43 ip-172-30-0-176 ProcessMonitor[9287]: 43.001967 |0|0|0| I 18 CAL0000: MSG RECEIVED: Check and Build System Tables
Jun 26 14:53:43 ip-172-30-0-176 ProcessMonitor[9287]: 43.003988 |0|0|0| D 18 CAL0000: buildSystemTables: dbbuilder 7 Successfully Launched
Jun 26 14:53:43 ip-172-30-0-176 ProcessMonitor[9287]: 43.004046 |0|0|0| I 18 CAL0000: PROCBUILDSYSTEMTABLES: ACK back to ProcMgr, return status = 0
Jun 26 14:53:43 ip-172-30-0-176 ProcessManager[9439]: 43.004099 |0|0|0| D 17 CAL0000: startSystemThread Exit

what is logging after a future startsystem or restartsystem

Jun 26 14:56:41 ip-172-30-0-176 ProcessMonitor[9287]: 41.583212 |0|0|0| I 18 CAL0000: MSG RECEIVED: Check and Build System Tables
Jun 26 14:56:41 ip-172-30-0-176 ProcessMonitor[9287]: 41.583286 |0|0|0| E 18 CAL0000: buildSystemTables: System Tables Already Exist
Jun 26 14:56:41 ip-172-30-0-176 ProcessMonitor[9287]: 41.583319 |0|0|0| I 18 CAL0000: PROCBUILDSYSTEMTABLES: ACK back to ProcMgr, return status = 6

Comment by David Hill (Inactive) [ 2017-06-26 ]

1 line code fix in ProcMgr

fixed in 1.0.x branch

commit 3b1864621fa6cb6a04245ee62d0c2464433868c1
Author: david hill <david.hill@mariadb.com>
Date: Mon Jun 26 10:13:43 2017 -0500

MCOL-787 - add code to create system catlog on startup

procmgr/processmanager.cpp | 3 +++

fix in 1.1.0 branch

commit b424a02c2e431be6f4cf39c12f0c7fd0155c23f5
Author: david hill <david.hill@mariadb.com>
Date: Mon Jun 26 09:58:38 2017 -0500

MCOL-787 - add code to create system catlog on startup

procmgr/processmanager.cpp | 3 +++

Comment by Daniel Lee (Inactive) [ 2017-06-28 ]

Build verified: Github source for 1.0.10 and 1.1.0

Verified both root and non-root install for 1um2pm configuration.
Verified DDL, DML, query, and cpimport

1.0.10-1

[root@localhost mariadb-columnstore-server]# git show
commit e8bf04ce2b6b67ee399b3626dff6055f35d50c60
Merge: a02e745 9c0aa3c
Author: David.Hall <david.hall@mariadb.com>
Date: Thu Jun 8 10:13:15 2017 -0500

[root@localhost mariadb-columnstore-engine]# git show
commit 3b1864621fa6cb6a04245ee62d0c2464433868c1
Author: david hill <david.hill@mariadb.com>
Date: Mon Jun 26 10:13:43 2017 -0500

1.1.0-1

[root@localhost mariadb-columnstore-server]# git show
commit 60f2f261f81d994307762d6d93380873513a0be8
Author: david hill <david.hill@mariadb.com>
Date: Tue Jun 20 16:05:32 2017 -0500

[root@localhost mariadb-columnstore-engine]# git show
commit b424a02c2e431be6f4cf39c12f0c7fd0155c23f5
Author: david hill <david.hill@mariadb.com>
Date: Mon Jun 26 09:58:38 2017 -0500

Comment by David Hill (Inactive) [ 2017-07-18 ]

problem in ubuntu16 VM testing, both postConfigure and procmgr are trying to run dbbuilder 7 causing the startup to hang

root 10796 7327 0 14:37 pts/19 00:00:00 sh -c /usr/local/mariadb/columnstore/bin/dbbuilder 7 > /tmp/dbbuilder.log
root 10797 10796 0 14:37 pts/19 00:00:00 /usr/local/mariadb/columnstore/bin/dbbuilder 7
root 11000 1 0 14:37 pts/19 00:00:00 /bin/sh /usr/local/mariadb/columnstore/mysql//bin/mysqld_safe --datadir=/usr/local/mar
mysql 11188 11000 0 14:37 pts/19 00:00:00 /usr/local/mariadb/columnstore/mysql//bin/mysqld --basedir=/usr/local/mariadb/columnst
root 11683 1 0 14:38 pts/19 00:00:00 /usr/local/mariadb/columnstore/bin/dbbuilder 7

----- Starting MariaDB ColumnStore on local server -----

MariaDB ColumnStore successfully started

MariaDB ColumnStore Database Platform Starting, please wait ....... DONE

Comment by David Hill (Inactive) [ 2017-07-19 ]

fix - added check in ProcMon fucntion that builds the system catalog to not run if postConfigure or dbbuilder executibles are running. This function is called by ProcMgr at the end of the successfull startsystem.

commit 6a0bceaa9bed412912691ed775da82d54bf8a6a9
Author: david hill <david.hill@mariadb.com>
Date: Wed Jul 19 09:49:31 2017 -0500

MCOL-787 - add checks before dbbuilder 7

procmgr/processmanager.cpp | 9 ++++++++-
procmon/processmonitor.cpp | 13 ++++++++++++-
2 files changed, 20 insertions, 2 deletions

Comment by David Hill (Inactive) [ 2017-07-19 ]

This would be the debug log to show that ProcMgr did the call, but its didn't try to do the build. This return code is ALREADY_IN_PROGRESS.

Jul 19 09:45:13 virtualbox-centos7 ProcessMonitor[3937]: 13.443277 |0|0|0| I 18 CAL0000: MSG RECEIVED: Check and Build System Tables
Jul 19 09:45:13 virtualbox-centos7 ProcessMonitor[3937]: 13.455789 |0|0|0| I 18 CAL0000: PROCBUILDSYSTEMTABLES: ACK back to ProcMgr, return status = 7
Jul 19 09:45:13 virtualbox-centos7 ProcessManager[4025]: 13.455830 |0|0|0| D 17 CAL0000: System Catalog Successfully not built by ProcMgr, ret code = 7

Comment by David Hill (Inactive) [ 2017-07-19 ]

this is what you would see on startsystem when the catalog already existed

Jul 19 09:56:19 virtualbox-centos7 ProcessMonitor[8296]: 19.114912 |0|0|0| I 18 CAL0000: MSG RECEIVED: Check and Build System Tables
Jul 19 09:56:19 virtualbox-centos7 ProcessMonitor[8296]: 19.138419 |0|0|0| D 18 CAL0000: buildSystemTables: System Tables Already Exist
Jul 19 09:56:19 virtualbox-centos7 ProcessMonitor[8296]: 19.138533 |0|0|0| I 18 CAL0000: PROCBUILDSYSTEMTABLES: ACK back to ProcMgr, return status = 6
Jul 19 09:56:19 virtualbox-centos7 ProcessManager[8384]: 19.138621 |0|0|0| D 17 CAL0000: System Catalog Successfully not built by ProcMgr, ret code = 6

Comment by David Hill (Inactive) [ 2017-07-27 ]

checkins for 1.1.0

commit f17c2e1820dd4379af8b52f82519ea9bb38aaa0c
Author: David Hill <david.hill@mariadb.com>
Date: Thu Jul 27 10:26:43 2017 -0500

MCOL-787 - merged code from 1.0.10

procmgr/processmanager.cpp | 11 ++++++++---
procmon/processmonitor.cpp | 33 ++++++++++++++++++++++-----------

Comment by Daniel Lee (Inactive) [ 2017-09-13 ]

Build verified: 1.1.0-1, 1.0.11-1

Generated at Thu Feb 08 02:23:49 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.