Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
1.2.2
-
None
-
None
-
mcsimport tool run remotely to mcs single server
-
2019-02, 2019-03
Description
High CPU usage and slow performance appears when load data with remote mcsimport
run autopilot cpimportLineitem test case group with option mcsimport .All test passed
but it's observed height cpu usage and tests finished slowly even in comparison to
maridb mysqlimport which is using SQL statement: LOAD DATA LOCAL INFILE on MCS.
how to repeat:
run remotely autopilot cpimportLineitem test case group with option mcsimport
run remotely autopilot cpimportLineitem test case group with option mysqlimport
./autopilot.sh features cpimportLineitem
| Remote Load Method | Elapsed Time [s] |
|---|---|
| MCSIMPORT | 6918 |
| MYSQLIMPORT | 2180 |
during all time of data loading with mcsimport was observed high cpu usage
# top
|
top - 14:04:09 up 53 days, 2:36, 4 users, load average: 0.83, 0.82, 0.62
|
Tasks: 180 total, 3 running, 167 sleeping, 8 stopped, 2 zombie
|
%Cpu(s): 10.3 us, 0.2 sy, 0.0 ni, 89.5 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
|
KiB Mem : 65975072 total, 454704 free, 13455012 used, 52065356 buff/cache
|
KiB Swap: 1048572 total, 745468 free, 303104 used. 49717568 avail Mem
|
|
|
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
|
21814 root 20 0 500040 170864 2524 R 83.7 0.3 9:40.07 mcsimport
|
10284 root 20 0 162004 2320 1584 R 0.3 0.0 0:12.78 top
|
17218 mysql 20 0 4911300 1.0g 17944 S 0.3 1.6 76:38.07 mysqld
|
1 root 20 0 191548 2920 1924 S 0.0 0.0 0:17.49 systemd
|
2 root 20 0 0 0 0 S 0.0 0.0 0:00.60 kthreadd
|
3 root 20 0 0 0 0 S 0.0 0.0 0:03.22 ksoftirqd/0
|
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H
|
7 root rt 0 0 0 0 S 0.0 0.0 0:22.88 migration/0
|
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh
|
|
BF Passed rowCnt=1024 actRowCnt=1024
|
BC Passed rowCnt=1025 actRowCnt=1025
|
TF Passed rowCnt=253952 actRowCnt=253952
|
TC Passed rowCnt=253953 actRowCnt=253953
|
CF Passed rowCnt=516096 actRowCnt=516096
|
CC Passed rowCnt=516097 actRowCnt=516097
|
EF Passed rowCnt=8380416 actRowCnt=8380416
|
EC Passed rowCnt=8380417 actRowCnt=8380417
|
SF Passed rowCnt=33546240 actRowCnt=33546240
|
SW Passed rowCnt=33546241 actRowCnt=33546241
|
PF Passed rowCnt=67100672 actRowCnt=67100672
|
PC Passed rowCnt=67100673 actRowCnt=67100673
|
[root@cps tests]#
|
|
trace get during the loading of EC test
# gdb -batch -ex 'thr a a bt' -p=$(pgrep mcsimport)
|
[New LWP 21818]
|
[New LWP 21817]
|
[New LWP 21816]
|
[New LWP 21815]
|
[Thread debugging using libthread_db enabled]
|
Using host libthread_db library "/lib64/libthread_db.so.1".
|
0x00007fd4e3962cc9 in ____strtod_l_internal () from /lib64/libc.so.6
|
|
|
Thread 5 (Thread 0x7fd4e250a700 (LWP 21815)):
|
#0 0x00007fd4e2730995 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
|
#1 0x00007fd4e31a4fc9 in uv_cond_wait () from /lib64/libuv.so.1
|
#2 0x00007fd4e3194136 in worker () from /lib64/libuv.so.1
|
#3 0x00007fd4e272ce25 in start_thread () from /lib64/libpthread.so.0
|
#4 0x00007fd4e3a22bad in clone () from /lib64/libc.so.6
|
|
|
Thread 4 (Thread 0x7fd4e1d09700 (LWP 21816)):
|
#0 0x00007fd4e2730995 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
|
#1 0x00007fd4e31a4fc9 in uv_cond_wait () from /lib64/libuv.so.1
|
#2 0x00007fd4e3194136 in worker () from /lib64/libuv.so.1
|
#3 0x00007fd4e272ce25 in start_thread () from /lib64/libpthread.so.0
|
#4 0x00007fd4e3a22bad in clone () from /lib64/libc.so.6
|
|
|
Thread 3 (Thread 0x7fd4e1508700 (LWP 21817)):
|
#0 0x00007fd4e2730995 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
|
#1 0x00007fd4e31a4fc9 in uv_cond_wait () from /lib64/libuv.so.1
|
#2 0x00007fd4e3194136 in worker () from /lib64/libuv.so.1
|
#3 0x00007fd4e272ce25 in start_thread () from /lib64/libpthread.so.0
|
#4 0x00007fd4e3a22bad in clone () from /lib64/libc.so.6
|
|
|
Thread 2 (Thread 0x7fd4e0d07700 (LWP 21818)):
|
#0 0x00007fd4e2730995 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
|
#1 0x00007fd4e31a4fc9 in uv_cond_wait () from /lib64/libuv.so.1
|
#2 0x00007fd4e3194136 in worker () from /lib64/libuv.so.1
|
#3 0x00007fd4e272ce25 in start_thread () from /lib64/libpthread.so.0
|
#4 0x00007fd4e3a22bad in clone () from /lib64/libc.so.6
|
|
|
Thread 1 (Thread 0x7fd4e4967740 (LWP 21814)):
|
#0 0x00007fd4e3962cc9 in ____strtod_l_internal () from /lib64/libc.so.6
|
#1 0x00007fd4e453d0bb in mcsapi::ColumnStoreDataConvert::convert (toMeta=toMeta@entry=0x7ffd3f545630, cont=0x34db038, fromValue=...) at /data/buildbot/bb-worker/centos7/mariadb-columnstore-api/src/util_dataconvert.cpp:1151
|
#2 0x00007fd4e45384bc in mcsapi::ColumnStoreBulkInsertImpl::setCharColumn (this=0xa8f1e0, columnNumber=6, value=..., status=0x7ffd3f545704) at /data/buildbot/bb-worker/centos7/mariadb-columnstore-api/src/mcsapi_bulk.cpp:481
|
#3 0x00007fd4e45387a8 in mcsapi::ColumnStoreBulkInsert::setColumn (this=0xa905e0, columnNumber=<optimized out>, value=..., status=<optimized out>) at /data/buildbot/bb-worker/centos7/mariadb-columnstore-api/src/mcsapi_bulk.cpp:75
|
#4 0x0000000000431ef2 in MCSRemoteImport::import() ()
|
#5 0x000000000042ceab in main ()
|
|