[MCOL-5300] review/finetune log collection tools Created: 2022-11-08  Updated: 2023-12-15

Status: Stalled
Project: MariaDB ColumnStore
Component/s: cmapi
Affects Version/s: 23.02
Fix Version/s: None

Type: Task Priority: Major
Reporter: David Hall (Inactive) Assignee: Leonid Fedorov
Resolution: Unresolved Votes: 0
Labels: None

Attachments: PNG File columnstore_review_1.2.1-new-features.png    
Epic Link: ColumnStore Message & Logging Improvements

 Description   

review the tool in https://github.com/mariadb-edwardstoever/columnstore_review

David's suggestion below

At the least, the following information should be contained in the reports:

  • Columnstore version
  • Build hash (hopefully)
  • Number of servers and how many are configured for accepting queries
  • Number of cores per server, RAM per server and anything else hardware we can think of.
  • Topology such as type of storage (s3, single data disk, shared nothing), aws, google, on-prem, skysql, etc.
  • Which server is currently prime.
  • Printout of Columnstore.xml
  • Print of all the files in my.cnf (or wherever .cnf files are kept)
  • If it can be found, the contents of mariadb .err file
  • All contents of /var/logs/mariadb/columnstore (or wherever logs are kept).
  • If we can figure it out, the contents of the bulk load logs.
  • Anything else we can think of that might help us figure stuff out.

  • All this and more used to be produced by ColumnstoreSupport executable (which called a bunch of shell scripts that ran some other things) That code can still be found in the develop-5 branch oamapps/columnstoreSupport for reference.

 Comments   
Comment by David Hall (Inactive) [ 2022-11-10 ]

I don't know if it's possible, but if a core file was created, including that in the same gzip could be useful.

Comment by Todd Stoffel (Inactive) [ 2022-12-13 ]

Moving to task.

Comment by alexey vorovich (Inactive) [ 2022-12-13 ]

edward to answer your question

build/commit hash is the variable that be returned as this

2022-12-13 00:20:30.401676 i_host 0 : show status like 'columnstore%';
2022-12-13 00:20:30.402592

{'Variable_name': 'Columnstore_commit_hash', 'Value': 'c14ab6d-dirty'}

2022-12-13 00:20:30.402675

{'Variable_name': 'Columnstore_version', 'Value': '22.08.4'}

I am not sure how you could obtain that easily.

Please comment on the items 1 and 2 above

Comment by Edward Stoever [ 2022-12-13 ]

Thank you alexey.vorovich. I will add commit_hash to next version of my columnstore_review script.
For anyone wanting to try out the script, please find it on github here:
columnstore_review
Thanks, – Edward

Comment by alexey vorovich (Inactive) [ 2022-12-13 ]

edward I asked the team to review and make suggestions

In my own very limited experience I can tell that the following folders pop-up in conversations here and there

'/var/lib/columnstore/local'
'/etc/columnstore'
'/etc/my.cnf.d'

and for sky this one may be relevant as well

/mnt/skysql/podinfo

Comment by alexey vorovich (Inactive) [ 2022-12-14 ]

edward
I will copy here the questions that engineers ask about customers on various channels. Maybe we could add that info to your tool if it is not already there.

here is the recent one

If they are are on prem what kind of storage do they use: S3, shared FS, shared block device, standalone block devices?

Comment by Edward Stoever [ 2022-12-14 ]

alexey.vorovich I just release 1.2.1. Regarding new features, I have added in a report of the namespace and podname if they are found. If not, then it must be on-prem or other hosting and it reports nothing.

I added Columnstore_commit_hash.

Both of these features can be seen in the attached image.

All of the other requests regarding "S3, shared FS, shared block device, standalone block devices" can be derived from things already reported in the script such as Topology, Storage type and mount points. If someone wants something more specific, let me know.

Comment by alexey vorovich (Inactive) [ 2022-12-14 ]

edward I saw new additions
thnks

1. the hash is good and informative for our dev build.. I will see why the prod build shows "source" . It may be that the version for prod build is enough
2. for sky , this is the content files in /mnt/sky, correct ?. It may be good to list the file names as well.. just to educate people like me

thnks

Comment by alexey vorovich (Inactive) [ 2023-03-17 ]

leonid.fedorov here is the log collection ticket
Based on yesterday exchange we may want to edward to add info such as disk IOPS for our main folders .

Comment by JiraAutomate [ 2023-12-15 ]

Automated message:
----------------------------
Since this issue has not been updated since 6 weeks, it's time to move it back to Stalled.

Generated at Thu Feb 08 02:56:51 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.