[MCOL-2034] Select from table returns Internal error: pColStep: invalid column Created: 2018-12-17  Updated: 2022-05-18

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: 1.2.2, 6.2.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Zdravelina Sokolovska (Inactive) Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

single server


 Description   

Select from table returns Internal error: pColStep: invalid column

Select from columnstore table returns Internal error: pColStep: invalid column.
That problem was observed when table was created during cpu overload by cs process DBRMControllerNode ;

During table creation MCS system was in Active state . After restarting MCS system the DBRMControllerNode process gets normal cpu load but select from table still fails with Internal error: pColStep: invalid column.

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 
 3438 root      20   0 4826784  32904  10708 S 800.0  0.0  34995:25 controllernode
 
 1203 root      20   0 5540356  20464   8996 S   0.3  0.0  15:31.61 ProcMon
 
 9157 root      20   0  461728  20260   8536 S   0.3  0.0   3:50.27 ServerMonitor
 
    1 root      20   0  191548   3016   2040 S   0.0  0.0   0:08.78 systemd
 
    2 root      20   0       0      0      0 S   0.0  0.0   0:00.27 kthreadd
 
    3 root      20   0       0      0      0 S   0.0  0.0   0:02.03 ksoftirqd/0
 
    5 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/0:0H
 
    7 root      rt   0       0      0      0 S   0.0  0.0   0:10.26 migration/0
 
    8 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcu_bh
 
    9 root      20   0       0      0      0 S   0.0  0.0   4:55.48 rcu_sched
 
   10 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 lru-add-drain
 
   11 root      rt   0       0      0      0 S   0.0  0.0   0:10.16 watchdog/0
 
   12 root      rt   0       0      0      0 S   0.0  0.0   0:09.44 watchdog/1
 
   13 root      rt   0       0      0      0 S   0.0  0.0   0:09.91 migration/1
 
   14 root      20   0       0      0      0 S   0.0  0.0   0:06.08 ksoftirqd/1
 
   16 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/1:0H
 
   17 root      rt   0       0      0      0 S   0.0  0.0   0:09.56 watchdog/2
 
   18 root      rt   0       0      0      0 S   0.0  0.0   0:09.77 migration/2
 
   19 root      20   0       0      0      0 S   0.0  0.0   0:01.69 ksoftirqd/2
 
   21 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/2:0H
 
   22 root      rt   0       0      0      0 S   0.0  0.0   0:09.11 watchdog/3

MCS table is created successfully but select from returns internal Errors :

ERROR 1815 (HY000): Internal error: pColStep: invalid column


Dec 17 15:27:45 cps ddlpackageproc[3586]: 45.168298 |2267|63361|0| D 23 CAL0041: Start SQL statement: CREATE TABLE IF NOT EXISTS mcol1662 (a int, b varchar(200)) engine=columnstore;|foo1|
 
Dec 17 15:27:49 cps ddlpackageproc[3586]: 49.538344 |2267|63361|0| D 23 CAL0042: End SQL statement
 



MariaDB [(none)]> select * from foo1.mcol1662 ;
 
ERROR 1815 (HY000): Internal error: pColStep: invalid column
 
MariaDB [(none)]>
 



MariaDB [(none)]>  select * from foo1.mcol1662 where a=1 ;
 
ERROR 1815 (HY000): Internal error: pColStep: invalid column
 



MariaDB [(none)]>  select count(*)  from foo1.mcol1662 ;
 
ERROR 1815 (HY000): Internal error: IDB-1000: 'mcol1662' and 'mcol1662' are not joined.
 



 
 
MariaDB [(none)]> show processlist ;
 
+------+-------------+-----------+------+---------+------+--------------------------+------------------+----------+
 
| Id   | User        | Host      | db   | Command | Time | State                    | Info             | Progress |
 
+------+-------------+-----------+------+---------+------+--------------------------+------------------+----------+
 
|    1 | system user |           | NULL | Daemon  | NULL | InnoDB purge coordinator | NULL             |    0.000 |
 
|    2 | system user |           | NULL | Daemon  | NULL | InnoDB purge worker      | NULL             |    0.000 |
 
|    3 | system user |           | NULL | Daemon  | NULL | InnoDB purge worker      | NULL             |    0.000 |
 
|    4 | system user |           | NULL | Daemon  | NULL | InnoDB purge worker      | NULL             |    0.000 |
 
|    5 | system user |           | NULL | Daemon  | NULL | InnoDB shutdown handler  | NULL             |    0.000 |
 
| 2268 | root        | localhost | NULL | Query   |    0 | Init                     | show processlist |    0.000 |
 
+------+-------------+-----------+------+---------+------+--------------------------+------------------+----------+
 
6 rows in set (0.005 sec)
 
 
 
 
 
 
 
 
 



[root@cps mariadb-columnstore-tpcds]# /usr/local/mariadb/columnstore/bin/viewtablelock
 
 No tables are locked in the database.


[root@cps queries]# mcsadmin getsystemi
 
getsysteminfo   Mon Dec 17 15:43:09 2018
 
 
 
System columnstore-1
 
 
 
System and Module statuses
 
 
 
Component     Status                       Last Status Change
 
------------  --------------------------   ------------------------
 
System        ACTIVE                       Fri Dec 14 13:32:21 2018
 
 
 
Module pm1    ACTIVE                       Thu Dec 13 13:29:28 2018
 
 
 
 
 
MariaDB ColumnStore Process statuses
 
 
 
Process             Module    Status            Last Status Change        Process ID
 
------------------  ------    ---------------   ------------------------  ----------
 
ProcessMonitor      pm1       ACTIVE            Wed Dec 12 15:03:20 2018        1203
 
ProcessManager      pm1       ACTIVE            Wed Nov 28 14:36:18 2018       20607
 
DBRMControllerNode  pm1       ACTIVE            Fri Dec 14 13:32:05 2018        3438
 
ServerMonitor       pm1       ACTIVE            Thu Dec 13 13:29:08 2018        9157
 
DBRMWorkerNode      pm1       ACTIVE            Thu Dec 13 13:29:08 2018        9178
 
PrimProc            pm1       ACTIVE            Thu Dec 13 13:29:12 2018        9246
 
ExeMgr              pm1       ACTIVE            Thu Dec 13 14:05:54 2018       13141
 
WriteEngineServer   pm1       ACTIVE            Fri Dec 14 13:15:07 2018       12987
 
DDLProc             pm1       ACTIVE            Fri Dec 14 13:32:13 2018        3586
 
DMLProc             pm1       ACTIVE            Fri Dec 14 13:32:20 2018        3739
 
mysqld              pm1       ACTIVE            Thu Dec 13 13:29:04 2018        9011
 
 
 



[root@cps queries]# mcsadmin getprocessconfig
 
getprocessconfig   Mon Dec 17 15:41:47 2018
 
 
 
Process Configuration
 
 
 
Process #1 Configuration information
 
ProcessName = ProcessMonitor
 
ModuleType = ChildExtOAMModule
 
ProcessLocation = /usr/local/mariadb/columnstore/bin/ProcMon
 
BootLaunch = 0
 
LaunchID = 1
 
RunType = LOADSHARE
 
LogFile = off
 
 
 
Process #2 Configuration information
 
ProcessName = ProcessManager
 
ModuleType = ParentOAMModule
 
ProcessLocation = /usr/local/mariadb/columnstore/bin/ProcMgr
 
BootLaunch = 1
 
LaunchID = 2
 
RunType = ACTIVE_STANDBY
 
LogFile = off
 
 
 
Process #3 Configuration information
 
ProcessName = DBRMControllerNode
 
ModuleType = ParentOAMModule
 
ProcessLocation = /usr/local/mariadb/columnstore/bin/controllernode
 
ProcessArg1 = /usr/local/mariadb/columnstore/bin/controllernode
 
ProcessArg2 = fg
 
BootLaunch = 2
 
LaunchID = 4
 
DepModuleName1 = @
 
DepProcessName1 = ProcessManager
 
RunType = SIMPLEX
 
LogFile = off
 
 
 
Process #4 Configuration information
 
ProcessName = ServerMonitor
 
ModuleType = ChildOAMModule
 
ProcessLocation = /usr/local/mariadb/columnstore/bin/ServerMonitor
 
ProcessArg1 = /usr/local/mariadb/columnstore/bin/ServerMonitor
 
BootLaunch = 2
 
LaunchID = 6
 
RunType = LOADSHARE
 
LogFile = off
 
 
 
Process #5 Configuration information
 
ProcessName = DBRMWorkerNode
 
ModuleType = ChildExtOAMModule
 
ProcessLocation = /usr/local/mariadb/columnstore/bin/workernode
 
ProcessArg1 = /usr/local/mariadb/columnstore/bin/workernode
 
ProcessArg2 = DBRM_Worker
 
ProcessArg3 = fg
 
BootLaunch = 2
 
LaunchID = 7
 
RunType = LOADSHARE
 
LogFile = off
 
 
 
Process #6 Configuration information
 
ProcessName = PrimProc
 
ModuleType = pm
 
ProcessLocation = /usr/local/mariadb/columnstore/bin/PrimProc
 
BootLaunch = 2
 
LaunchID = 20
 
RunType = LOADSHARE
 
LogFile = off
 
 
 
Process #7 Configuration information
 
ProcessName = ExeMgr
 
ModuleType = pm
 
ProcessLocation = /usr/local/mariadb/columnstore/bin/ExeMgr
 
BootLaunch = 2
 
LaunchID = 30
 
DepModuleName1 = pm*
 
DepProcessName1 = PrimProc
 
RunType = LOADSHARE
 
LogFile = off
 
 
 
Process #8 Configuration information
 
ProcessName = WriteEngineServer
 
ModuleType = pm
 
ProcessLocation = /usr/local/mariadb/columnstore/bin/WriteEngineServer
 
BootLaunch = 2
 
LaunchID = 40
 
RunType = LOADSHARE
 
LogFile = off
 
 
 
Process #9 Configuration information
 
ProcessName = DDLProc
 
ModuleType = pm
 
ProcessLocation = /usr/local/mariadb/columnstore/bin/DDLProc
 
BootLaunch = 2
 
LaunchID = 50
 
DepModuleName1 = pm*
 
DepProcessName1 = WriteEngineServer
 
DepModuleName2 = @
 
DepProcessName2 = DBRMWorkerNode
 
DepModuleName3 = *
 
DepProcessName3 = ExeMgr
 
RunType = SIMPLEX
 
LogFile = off
 
 
 
Process #10 Configuration information
 
ProcessName = DMLProc
 
ModuleType = pm
 
ProcessLocation = /usr/local/mariadb/columnstore/bin/DMLProc
 
BootLaunch = 2
 
LaunchID = 51
 
DepModuleName1 = pm*
 
DepProcessName1 = WriteEngineServer
 
DepModuleName2 = @
 
DepProcessName2 = DBRMWorkerNode
 
DepModuleName3 = @
 
DepProcessName3 = DDLProc
 
RunType = SIMPLEX
 
LogFile = off
 
 
 
Process #11 Configuration information
 
ProcessName = mysqld
 
ModuleType = pm
 
ProcessLocation = /usr/local/mariadb/columnstore/mysql/libexec/mysqld
 
BootLaunch = 0
 
LaunchID = 100
 
RunType = LOADSHARE
 
LogFile = off



 Comments   
Comment by Andrew Hutchings (Inactive) [ 2018-12-17 ]

How did you get controllernode into a high CPU state?

Comment by Zdravelina Sokolovska (Inactive) [ 2018-12-21 ]

most probably controllernode get into high CPU after load with mcsimport.
the problem however is that the MCS system is in Active State and table is successfully created.
but then even if mcs is restated at some stage and there is no more high cpu usage
could not read/ writ the table .

Comment by Andrew Hutchings (Inactive) [ 2018-12-21 ]

Yes, that will be due to whatever controllernode was trying and failing to do stuck in some kind of loop. Are you able to run a poor man's profiler on the controllernode when it gets into that high CPU state? May give us some idea of what is going on: https://poormansprofiler.org/

Generated at Thu Feb 08 02:33:13 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.