[MCOL-533] table_usage() has a calculation error Created: 2017-01-25  Updated: 2017-03-22  Resolved: 2017-03-22

Status: Closed
Project: MariaDB ColumnStore
Component/s: ExeMgr
Affects Version/s: 1.0.7
Fix Version/s: 1.0.8, 1.1.0

Type: Bug Priority: Major
Reporter: Andrew Hutchings (Inactive) Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Sprint: 2017-2, 2017-3, 2017-4, 2017-5, 2017-6

 Description   

The table_usage() procedure adds entries multiple times under some scenarios which can cause significant errors. The procedure needs re-writing.



 Comments   
Comment by Andrew Hutchings (Inactive) [ 2017-01-25 ]

Recommend backporting this to 1.0 if we do another release.

Need to add a warning in the documentation that this shouldn't be run by more than one client at the same time (the new version of table_usage will have problems if you try).

Anyone who is hit with this problem can fix it on their own installation by manually piping this version of columnstore_info.sql into their mariadb client as root.

Comment by Andrew Hutchings (Inactive) [ 2017-01-25 ]

Second pull request adds safety locks as discussed with David Hall.

Comment by Daniel Lee (Inactive) [ 2017-03-02 ]

Build tested:
[root@localhost columnstore]# cd mariadb-columnstore-server/
[root@localhost mariadb-columnstore-server]# git show
commit 3da188e5c8a2630019ea810fb8c1bd3ece5e058b
Merge: 5d9686c 53c1df7
Author: Andrew Hutchings <andrew@linuxjedi.co.uk>
Date: Fri Feb 10 15:07:31 2017 +0000

Merge pull request #31 from jbfavre/fix_deb_package_dependency

MCOL-562 Fix Debian package dependencies

[root@localhost mariadb-columnstore-server]# cd mariadb-columnstore-engine/
[root@localhost mariadb-columnstore-engine]# git show
commit 16cef50caedd9ec7585b04c096996a9441bdf2d5
Author: David Hill <david.hill@mariadb.com>
Date: Wed Mar 1 10:39:11 2017 -0600

change the check for prompt back to the previous code

Tested with concurrent sessions. While session 1 is running, session 2 return the following, which is the expected behavior.

MariaDB [tpch10c]> call columnstore_info.table_usage(NULL,NULL);
ERROR 1644 (45000): Error acuqiring table_usage lock

"acquiring" spelling is incorrect.
MariaDB [tpch10c]> call columnstore_info.table_usage(NULL,NULL);
-----------------------------------------------------------------

TABLE_SCHEMA TABLE_NAME DATA_DISK_USAGE DICT_DISK_USAGE TOTAL_USAGE

-----------------------------------------------------------------

mytest lineitem 3020423168 285310976 3305734144
tpch10c customer 1879310336 1082294272 2961604608
tpch10c lineitem 4321280000 1895923712 6217203712
tpch10c nation 6324224 4218880 10543104
tpch10c orders 1644462080 1082228736 2726690816
tpch10c part 2147778560 570621952 2718400512
tpch10c partsupp 939687936 1073807360 2013495296
tpch10c region 5267456 4218880 9486336
tpch10c supplier 1220771840 33685504 1254457344

-----------------------------------------------------------------
9 rows in set (5.37 sec)

Query OK, 0 rows affected (5.37 sec)

Results are in bytes only. It should display in mb, gb.

Comment by Daniel Lee (Inactive) [ 2017-03-02 ]

Per my last comment

Comment by Andrew Hutchings (Inactive) [ 2017-03-02 ]

Fix for units and spelling error that Daniel spotted

Comment by Daniel Lee (Inactive) [ 2017-03-03 ]

Build verified: Github source

[root@localhost mariadb-columnstore-server]# git show
commit 3da188e5c8a2630019ea810fb8c1bd3ece5e058b
Merge: 5d9686c 53c1df7
Author: Andrew Hutchings <andrew@linuxjedi.co.uk>
Date: Fri Feb 10 15:07:31 2017 +0000

Merge pull request #31 from jbfavre/fix_deb_package_dependency

MCOL-562 Fix Debian package dependencies

[root@localhost mariadb-columnstore-engine]# git show
commit f19710126a9a523eccf2d65b022086cc44cde53e
Merge: 86ddc8a c6000d3
Author: dhall-InfiniDB <david.hall@mariadb.com>
Date: Thu Mar 2 14:41:51 2017 -0600

Merge pull request #129 from mariadb-corporation/MCOL-533-format

MCOL-533 fix units for table_usage

Verified the latest fixes.

Comment by David Thompson (Inactive) [ 2017-03-07 ]

re-opening to backport to 1.0.8

Comment by Andrew Hutchings (Inactive) [ 2017-03-07 ]

New pull request for backport to 1.0

Comment by Daniel Lee (Inactive) [ 2017-03-21 ]

Tested in 1.0.8-1 on a 1um2pm stack

MariaDB [(none)]> use tpch10c
Database changed
MariaDB [tpch10c]> call columnstore_info.table_usage(NULL,NULL);
ERROR 1105 (HY000): Unknown error 0% of stage done
MariaDB [tpch10c]> call columnstore_info.table_usage(NULL,NULL);
ERROR 1644 (45000): Error acquiring table_usage lock

There are crit.log and err.log files in all three nodes. Therefore, no errors have been logged.

Comment by Andrew Hutchings (Inactive) [ 2017-03-22 ]

This problem is the one in MCOL-605. I suggest retesting once that is merged.

Comment by Daniel Lee (Inactive) [ 2017-03-22 ]

Build verified: 1.0.8-1

mcsadmin> getsoftware
getsoftwareinfo Wed Mar 22 22:47:15 2017

Name : mariadb-columnstore-platform
Version : 1.0.8
Release : 1
Architecture: x86_64
Install Date: Wed 22 Mar 2017 09:37:39 PM UTC
Group : Applications/Databases
Size : 10012539
License : Copyright (c) 2016 MariaDB Corporation Ab., all rights reserved; redistributable under the terms of the GPL, see the file COPYING for details.
Signature : (none)
Source RPM : mariadb-columnstore-platform-1.0.8-1.src.rpm
Build Date : Wed 22 Mar 2017 08:41:24 PM UTC
Build Host : ip-172-30-0-125.us-west-2.compute.internal
Relocations : (not relocatable)
Vendor : MariaDB Corporation Ab
URL : http://mariadb.org
Summary : MariaDB-Columnstore software
Description :
MariaDB Columnstore: a very fast and robust SQL database server

MariaDB [(none)]> call columnstore_info.table_usage(NULL,NULL);
-----------------------------------------------------------------

TABLE_SCHEMA TABLE_NAME DATA_DISK_USAGE DICT_DISK_USAGE TOTAL_USAGE

-----------------------------------------------------------------

mytest orders 784.14 MB 260.05 MB 1.02 GB
tpch10c customer 896.12 MB 516.08 MB 1.38 GB
tpch10c lineitem 3.23 GB 1.27 GB 4.49 GB
tpch10c nation 6.03 MB 4.02 MB 10.05 MB
tpch10c orders 784.14 MB 1.00 GB 1.77 GB
tpch10c part 1.00 GB 272.09 MB 1.27 GB
tpch10c partsupp 448.08 MB 1.00 GB 1.44 GB
tpch10c region 5.02 MB 4.02 MB 9.05 MB
tpch10c supplier 396.11 MB 140.06 MB 536.17 MB

-----------------------------------------------------------------
9 rows in set (0.92 sec)

Query OK, 0 rows affected (0.92 sec)

MariaDB [(none)]> call columnstore_info.table_usage(NULL,NULL);
ERROR 1644 (45000): Error acquiring table_usage lock

On a 1um2pm stack, it is hard to see the locking mechanism working, since the query does not take long to execute. Testing with two session right after the ColumnStore has been restarted has a higher chance of see the lock.

Build verified: 1.1.0 GitHub source
[root@localhost mariadb-columnstore-server]# git show
commit 24f5065dba978ef4ec6ea8cb4bfd9728321941bd
Author: david hill <david.hill@mariadb.com>
Date: Mon Mar 20 09:57:25 2017 -0500
[root@localhost mariadb-columnstore-engine]# git show
commit ff36e8e774fd961625d42b45070f56bead83a5bc
Merge: c1d38e0 6ab434e
Author: dhall-InfiniDB <david.hall@mariadb.com>
Date: Wed Mar 22 15:16:53 2017 -0500
Merge pull request #139 from mariadb-corporation/MCOL-605-hotfix
MCOL-605 Fix multi-node "Unkown error"

MariaDB [(none)]> call columnstore_info.table_usage(NULL,NULL);
-----------------------------------------------------------------

TABLE_SCHEMA TABLE_NAME DATA_DISK_USAGE DICT_DISK_USAGE TOTAL_USAGE

-----------------------------------------------------------------

mytest orders 784.14 MB 260.05 MB 1.02 GB
tpch10c customer 896.12 MB 516.08 MB 1.38 GB
tpch10c lineitem 3.23 GB 1.27 GB 4.49 GB
tpch10c nation 6.03 MB 4.02 MB 10.05 MB
tpch10c orders 784.14 MB 1.00 GB 1.77 GB
tpch10c part 1.00 GB 272.09 MB 1.27 GB
tpch10c partsupp 448.08 MB 1.00 GB 1.44 GB
tpch10c region 5.02 MB 4.02 MB 9.05 MB
tpch10c supplier 396.11 MB 140.06 MB 536.17 MB

-----------------------------------------------------------------
9 rows in set (0.59 sec)

MariaDB [(none)]> call columnstore_info.table_usage(NULL,NULL);
ERROR 1644 (45000): Error acquiring table_usage lock

Comment by Daniel Lee (Inactive) [ 2017-03-22 ]

Closed per my last comment.

Generated at Thu Feb 08 02:21:48 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.