[MDBF-276] Apply changes to the MSAN builder Created: 2021-10-05  Updated: 2022-03-03  Resolved: 2022-03-03

Status: Closed
Project: MariaDB Foundation Development
Component/s: None
Affects Version/s: N/A
Fix Version/s: N/A

Type: Task Priority: Major
Reporter: Vlad Bogolin Assignee: Vlad Bogolin
Resolution: Fixed Votes: 0
Labels: buildbot
Remaining Estimate: 0d
Time Spent: 1d 1.5h
Original Estimate: Not Specified

Attachments: File build-msan.sh    
Issue Links:
PartOf
includes MDBF-229 Instrumented libcrack for MSAN builder Closed
is part of MDBF-41 Milestone 5: Desirable fixes Open
Relates
relates to MDEV-26757 Compression-related tests fail with M... Open
relates to MDEV-27964 The function ENCRYPT() causes SIGSEGV... Open

 Description   

Apply the new changes to the MSAN builder as described in the updated MDEV-20377 and MDEV-26758.

 Comments   
Comment by Marko Mäkelä [ 2022-02-16 ]

Either all libraries must be instrumented, or uninstrumented libraries must be unavailable for the MSAN build. An example of a rare dependency is this:

CURRENT_TEST: plugins.two_password_validations
 
mysqltest: At line 21: query 'grant select on *.* to foobar identified by 'q$%^&*R1234ty'' failed: <Unknown> (2013): Lost connection to server during query
 
…
 
Uninitialized bytes in __interceptor_strchr at offset 0 inside [0x7f54277492e0, 1)
 
==267325==WARNING: MemorySanitizer: use-of-uninitialized-value
 
    #0 0x7f54247d1f43  (/usr/lib/x86_64-linux-gnu/libcrack.so.2+0x3f43)
 
    #1 0x7f54247d2356 in FascistCheckUser (/usr/lib/x86_64-linux-gnu/libcrack.so.2+0x4356)
 
    #2 0x7f5424ff00c5 in crackme /buildbot/amd64-ubuntu-1804-msan/build/plugin/cracklib_password_check/cracklib_password_check.c:39:13
 
    #3 0x56518621b228 in do_validate(THD*, st_plugin_int**, void*) /buildbot/amd64-ubuntu-1804-msan/build/sql/sql_acl.cc:2178:10

Comment by Marko Mäkelä [ 2022-02-23 ]

Thank you, this is much better now. Some work remains to be done:

  • The ENCRYPT() function is broken in some tests.
  • some compression libraries need to be instrumented, or removed from the test run altogether
  • libcrack needs to be instrumented (as noted in my previous comment)
  • some tests are too slow or big for MSAN and should be tweaked somehow, possibly by extending the have_sanitizer variable and skipping tests on MSAN
Comment by Marko Mäkelä [ 2022-02-23 ]

The problems related to compression libraries has been filed as MDEV-26757. I had posted some commands to build one compression library there.

Either all compression libraries should be built with MSAN instrumentation, or (my preference, because at least when it comes to InnoDB, I do not expect anything to be caught) the compression libraries should be made unavailable during MSAN builds.

Comment by Vlad Bogolin [ 2022-02-23 ]

I have disabled compression tests. Hopefully, 

--skip-test=.*compression.*

is the way to do it and that doesn't miss anything. Now I am trying to see why instrumented libcrack doesn't find the dictionary.

Comment by Vlad Bogolin [ 2022-02-24 ]

All changes should now be added to the new MSAN builder https://buildbot.mariadb.org/#/builders/348.

I managed to solve the libcrack dictionary problem (see MDBF-229 for details). The updated build-msan.sh script is attached.

Comment by Vlad Bogolin [ 2022-02-28 ]

Updated to clang-12 as discussed on Slack. All changes should be live.

Comment by Marko Mäkelä [ 2022-03-01 ]

After I worked around MDEV-27964 in the tests, the MSAN builds should now be green, for all main branches 10.5 to 10.9.

Generated at Thu Feb 08 03:36:42 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.