buildbot.mariadb.org (MDEV-8244)

[MDEV-11924] use asan/tsan/ubsan in buildbot Created: 2017-01-27  Updated: 2022-04-01

Status: Open
Project: MariaDB Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Technical task Priority: Major
Reporter: Sergei Golubchik Assignee: Vicențiu Ciorbaru
Resolution: Unresolved Votes: 0
Labels: None

Attachments: PNG File Screen Shot 2018-04-19 at 10.20.07 AM.png     HTML File sample_tsan_output    
Issue Links:
Blocks
is blocked by CONC-369 libmariadb doesn't compile with WITH_... Closed
is blocked by MDEV-17703 Add WITH_UBSAN switch to CMake simila... Closed

 Description   

For https://buildbot.mariadb.org



 Comments   
Comment by Sergei Golubchik [ 2017-04-06 ]

asan is enabled on fulltest-big builder

Comment by Sergei Golubchik [ 2017-04-07 ]

… and it failed completely, because fulltest VM is precise and has gcc-4.6. No asan.

Comment by Daniel Black [ 2017-04-07 ]

https://travis-ci.org/grooverdan/mariadb-server/builds/219519182 is what to expect. I hadn't merged the Darwin fixes at that stage however its a fairly recent 10.2 tree.

Comment by Sergei Golubchik [ 2017-04-07 ]

moved fulltest and fulltest-big builders to xenial, let's see how it'll work out

Comment by Daniel Black [ 2017-04-09 ]

FYI you may need to -mysqld=-thread_stack=

{something larger}

I saw a few tests running out of stack (on Debug builds):
https://travis-ci.org/grooverdan/mariadb-server/jobs/220189059#L5986

Comment by Teodor Mircea Ionita (Inactive) [ 2018-04-13 ]

serg Hi, do you remember what state was this left in? Any builders configured to use it?

Comment by Sergei Golubchik [ 2018-04-14 ]

you can grep maria-master.cfg for the string "SAN", and you'll see that fulltest-big builds with ASAN. And that no other sanitizers are used anywhere.

Comment by Teodor Mircea Ionita (Inactive) [ 2018-04-18 ]

I noticed that, however, it doesn't seem be effective since the verbose compiler lines don't have -fsanitize=address. While this is a 16.04 GCC machine, I think the option has the same syntax as clang.

https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html
https://buildbot.askmonty.org/buildbot/builders/kvm-fulltest-big/builds/1713/steps/compile/logs/stdio

Comment by Sergei Golubchik [ 2018-04-19 ]

strange, I think I can see -fsanitize=address in the log that you linked to above.

Comment by Teodor Mircea Ionita (Inactive) [ 2018-04-19 ]

Safari lost me on this one. Disappointing... =)

Comment by Teodor Mircea Ionita (Inactive) [ 2018-10-15 ]

serg In the context of buildbot.mariadb.org, do we wish to compile with SAN for *most builders, or just big-test ones? This will impact the quick builders performance which currently run at about 30-35 minutes to completion (including tests).

Comment by Sergei Golubchik [ 2018-10-15 ]

Looking at the current buildbot, most builders build and test packages. We surely cannot enable sanitizers on those builders.

So, no, not on most builders. Just on a few. Not necessarily on bigtest ones, though.

Comment by Teodor Mircea Ionita (Inactive) [ 2018-12-05 ]

Reminder to self:

MDEV-11924 - ASAN in Buildbot - Runtime Address Sanitizer for bounds checking
Undefined behavior testor:
Easy win in CI environment

Thread Safety Analysis (http://clang.llvm.org/docs/ThreadSafetyAnalysis.html)
Needs to mark variables with mutexs required (attribute and annotate mutex functions)
C

Unknown macro: {,XX}

FLAGS=-Wthread-safety
Doesn’t seem that difficult or intrusive

Comment by Eugene Kosov (Inactive) [ 2018-12-18 ]

Here is a simple script to use TSAN.

#!/bin/bash
 
git clone --depth=1 --recurse-submodules git@github.com:MariaDB/server.git server
 
mkdir debug_build
cd debug_build
 
COMPILER_FLAGS="-fdiagnostics-color -fno-omit-frame-pointer -gsplit-dwarf"
cmake \
    -G Ninja \
    -DWITH_TSAN=1 \
    -DCMAKE_BUILD_TYPE=Debug \
    -DCMAKE_C_FLAGS=$COMPILER_FLAGS \
    -DCMAKE_CXX_FLAGS=$COMPILER_FLAGS \
    -DWITH_SAFEMALLOC=0 \
    -DWITH_WSREP=0 \
    -DWITH_UNIT_TESTS=0 \
    -DWITH_UNITTEST=0 \
    -DWITH_CLIENT=0 \
    -DWITH_ZLIB=system \
    -DWITHOUT_ROCKSDB_STORAGE_ENGINE=1 \
    -DWITHOUT_MROONGA_STORAGE_ENGINE=1 \
    -DWITHOUT_PARTITION_STORAGE_ENGINE=1 \
    -DWITHOUT_CONNECT_STORAGE_ENGINE=1 \
    -DWITHOUT_SPHINX_STORAGE_ENGINE=1 \
    -DWITHOUT_TOKUDB_STORAGE_ENGINE=1 \
    -DWITHOUT_SPIDER_STORAGE_ENGINE=1 \
    -DCMAKE_EXPORT_COMPILE_COMMANDS=1 \
    ../server
 
ninja -j4
 
cd mysql-test
./mtr -mem -par=auto

I also attach a sample output. 91 warnings total. sample_tsan_output

Comment by Eugene Kosov (Inactive) [ 2018-12-18 ]

And I have a recent gcc:
gcc (Ubuntu 8.2.0-7ubuntu1) 8.2.0

Generated at Thu Feb 08 07:53:42 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.