[MDEV-7144] Warnings "bytes lost" during server shutdown after running connect.part_file test in buildbot Created: 2014-11-20  Updated: 2015-01-15  Resolved: 2015-01-15

Status: Closed
Project: MariaDB Server
Component/s: Data Manipulation - Update
Affects Version/s: 10.0
Fix Version/s: 10.0.16

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Sergei Golubchik
Resolution: Fixed Votes: 0
Labels: buildbot, tests

Issue Links:
Relates
relates to MDEV-7069 Fix buildbot failures in main server ... Stalled

 Description   

Buildbot example: http://buildbot.askmonty.org/buildbot/builders/kvm-dgcov-jaunty-i386/builds/4449/steps/test_2/logs/stdio

I can reproduce the problem by running the test with valgrind on a debug build. The valgrind report goes OK, but the warnings are generated on shutdown. It happens almost every time when I run the test:

perl ./mtr connect.part_file  --valgrind-mysqld
Logging: ./mtr  connect.part_file --valgrind-mysqld
...
 
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
connect.part_file                        [ pass ]   4482
***Warnings generated in error logs during shutdown after running tests: connect.part_file
 
Warning: 65536 bytes lost at 0xfaea070, allocated by T@0 at 
valgrind_report                          [ pass ]       
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 4.482 of 31 seconds executing testcases
 
Warnings in log: All 2 tests were successful.
 
Errors/warnings were found in logfiles during server shutdown after running the
following sequence(s) of tests:
    connect.part_file
mysql-test-run: *** ERROR: There where errors/warnings in server logs after running test cases.



 Comments   
Comment by Olivier Bertrand [ 2014-11-20 ]

Helena,
Your almost seems to me sort of an overstatement.
As a matter of facts, this case seems very strange. Indeed, the first time I ran a valgrind test on part_file, I got some valgrind warnings about lost memories (there were 3 or 4 of them) Unfortunately I did not save them but I remember that they where all about UPDATE and that in the pile of called functions (terminated by malloc, my_malloc and sometimes strdup) there were no CONNECT functions involved.
So what I did is to comment out the unique successful UPDATE statement of part_file and run it again. No more Valgrind warnings.
Then I made a new small test doing nothing all than this UPDATE to be sure it was the culprit but this one also was successful with no warnings from Valgrind.
Then I re-test the original part_file test but could not have Valgrind warnings anymore. Even after restarting ubuntu all other attempts I made to get these warnings were unsuccessful.
Looks like an almost "cannot reproduce" case.

Comment by Elena Stepanova [ 2014-11-20 ]

Olivier,

My almost was not generic, it meant exactly what it said: when I ran the test case, it happened almost every time. It doesn't mean it is so for everyone else.
More specifically, before filing the report, I ran it 6 times, 1st time it failed, then it didn't, then it failed 4 times. I think it qualifies as 'almost'.
The ratio still holds today, see at the end of this message the unabridged output from my console created while I was writing this – I just re-run the test over and over again. 4 times it failed, then it didn't, then it failed again.

The situation in buildbot is similar, out of 6 last runs of kvm-dgcov-jaunty-i386 on the main 10.0 tree, it happened 4 times. We didn't notice it before, because there were other failing tests, but now when we've cleaned the buildbot a bit, the warnings became apparent.
I can also see them on 10.0-connect tree, see for example http://buildbot.askmonty.org/buildbot/builders/kvm-dgcov-jaunty-i386/builds/4406/steps/test_3/logs/stdio
(search for connect.part_file).

Even regardless my local experiments, it certainly cannot be qualified as "cannot reproduce" case – not only does it happen in buildbot, but you have even seen it yourself (I'm surprised to hear about 3 or 4 though, for me it's always a single warning).

If you can't reproduce it, please try the following:

  • run perl ./mtr connect.part_table connect.part_file --valgrind-mysqld --noreorder – that's the sequence that initially triggers the problem in buildbot. Maybe it will work for you, too;
  • if you are running on a valgrind build, try a non-valgrind one (-DWITH_VALGRIND=OFF). That's what both buildbot and I are using.

elenst@wheezy-64:~/bzr/10.0/mysql-test$ perl ./mtr connect.part_file  --valgrind-mysqld 
Logging: ./mtr  connect.part_file --valgrind-mysqld
Turning on valgrind for mysqld(s) only
Running valgrind with options " --show-reachable=yes --quiet "
vardir: /data/repo/bzr/10.0/mysql-test/var
Checking leftover processes...
Removing old var directory...
Creating var directory '/data/repo/bzr/10.0/mysql-test/var'...
Checking supported features...
MariaDB Version 10.0.15-MariaDB-debug
 - SSL connections supported
 - binaries are debug compiled
Collecting tests...
Installing system database...
 
==============================================================================
 
TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------
 
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
connect.part_file                        [ pass ]   4525
***Warnings generated in error logs during shutdown after running tests: connect.part_file
 
Warning: 65536 bytes lost at 0xfaea070, allocated by T@0 at 
valgrind_report                          [ pass ]       
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 4.525 of 32 seconds executing testcases
 
Warnings in log: All 2 tests were successful.
 
Errors/warnings were found in logfiles during server shutdown after running the
following sequence(s) of tests:
    connect.part_file
mysql-test-run: *** ERROR: There where errors/warnings in server logs after running test cases.
elenst@wheezy-64:~/bzr/10.0/mysql-test$ perl ./mtr connect.part_file  --valgrind-mysqld 
Logging: ./mtr  connect.part_file --valgrind-mysqld
Turning on valgrind for mysqld(s) only
Running valgrind with options " --show-reachable=yes --quiet "
vardir: /data/repo/bzr/10.0/mysql-test/var
Checking leftover processes...
Removing old var directory...
Creating var directory '/data/repo/bzr/10.0/mysql-test/var'...
Checking supported features...
MariaDB Version 10.0.15-MariaDB-debug
 - SSL connections supported
 - binaries are debug compiled
Collecting tests...
Installing system database...
 
==============================================================================
 
TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------
 
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
connect.part_file                        [ pass ]   4431
***Warnings generated in error logs during shutdown after running tests: connect.part_file
 
Warning: 65536 bytes lost at 0xfaea070, allocated by T@0 at 
valgrind_report                          [ pass ]       
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 4.431 of 31 seconds executing testcases
 
Warnings in log: All 2 tests were successful.
 
Errors/warnings were found in logfiles during server shutdown after running the
following sequence(s) of tests:
    connect.part_file
mysql-test-run: *** ERROR: There where errors/warnings in server logs after running test cases.
elenst@wheezy-64:~/bzr/10.0/mysql-test$ perl ./mtr connect.part_file  --valgrind-mysqld 
Logging: ./mtr  connect.part_file --valgrind-mysqld
Turning on valgrind for mysqld(s) only
Running valgrind with options " --show-reachable=yes --quiet "
vardir: /data/repo/bzr/10.0/mysql-test/var
Checking leftover processes...
Removing old var directory...
Creating var directory '/data/repo/bzr/10.0/mysql-test/var'...
Checking supported features...
MariaDB Version 10.0.15-MariaDB-debug
 - SSL connections supported
 - binaries are debug compiled
Collecting tests...
Installing system database...
 
==============================================================================
 
TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------
 
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
connect.part_file                        [ pass ]   4373
***Warnings generated in error logs during shutdown after running tests: connect.part_file
 
Warning: 65536 bytes lost at 0xfaea070, allocated by T@0 at 
valgrind_report                          [ pass ]       
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 4.373 of 30 seconds executing testcases
 
Warnings in log: All 2 tests were successful.
 
Errors/warnings were found in logfiles during server shutdown after running the
following sequence(s) of tests:
    connect.part_file
mysql-test-run: *** ERROR: There where errors/warnings in server logs after running test cases.
elenst@wheezy-64:~/bzr/10.0/mysql-test$ perl ./mtr connect.part_file  --valgrind-mysqld 
Logging: ./mtr  connect.part_file --valgrind-mysqld
Turning on valgrind for mysqld(s) only
Running valgrind with options " --show-reachable=yes --quiet "
vardir: /data/repo/bzr/10.0/mysql-test/var
Checking leftover processes...
Removing old var directory...
Creating var directory '/data/repo/bzr/10.0/mysql-test/var'...
Checking supported features...
MariaDB Version 10.0.15-MariaDB-debug
 - SSL connections supported
 - binaries are debug compiled
Collecting tests...
Installing system database...
 
==============================================================================
 
TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------
 
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
connect.part_file                        [ pass ]   4372
***Warnings generated in error logs during shutdown after running tests: connect.part_file
 
Warning: 65536 bytes lost at 0xfaea070, allocated by T@0 at 
valgrind_report                          [ pass ]       
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 4.372 of 29 seconds executing testcases
 
Warnings in log: All 2 tests were successful.
 
Errors/warnings were found in logfiles during server shutdown after running the
following sequence(s) of tests:
    connect.part_file
mysql-test-run: *** ERROR: There where errors/warnings in server logs after running test cases.
elenst@wheezy-64:~/bzr/10.0/mysql-test$ perl ./mtr connect.part_file  --valgrind-mysqld 
Logging: ./mtr  connect.part_file --valgrind-mysqld
Turning on valgrind for mysqld(s) only
Running valgrind with options " --show-reachable=yes --quiet "
vardir: /data/repo/bzr/10.0/mysql-test/var
Checking leftover processes...
Removing old var directory...
Creating var directory '/data/repo/bzr/10.0/mysql-test/var'...
Checking supported features...
MariaDB Version 10.0.15-MariaDB-debug
 - SSL connections supported
 - binaries are debug compiled
Collecting tests...
Installing system database...
 
==============================================================================
 
TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------
 
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
connect.part_file                        [ pass ]   4429
valgrind_report                          [ pass ]       
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 4.429 of 30 seconds executing testcases
 
Completed: All 2 tests were successful.
 
elenst@wheezy-64:~/bzr/10.0/mysql-test$ perl ./mtr connect.part_file  --valgrind-mysqld 
Logging: ./mtr  connect.part_file --valgrind-mysqld
Turning on valgrind for mysqld(s) only
Running valgrind with options " --show-reachable=yes --quiet "
vardir: /data/repo/bzr/10.0/mysql-test/var
Checking leftover processes...
Removing old var directory...
Creating var directory '/data/repo/bzr/10.0/mysql-test/var'...
Checking supported features...
MariaDB Version 10.0.15-MariaDB-debug
 - SSL connections supported
 - binaries are debug compiled
Collecting tests...
Installing system database...
 
==============================================================================
 
TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------
 
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
connect.part_file                        [ pass ]   4354
***Warnings generated in error logs during shutdown after running tests: connect.part_file
 
Warning: 65536 bytes lost at 0xfaea070, allocated by T@0 at 
valgrind_report                          [ pass ]       
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 4.354 of 30 seconds executing testcases
 
Warnings in log: All 2 tests were successful.
 
Errors/warnings were found in logfiles during server shutdown after running the
following sequence(s) of tests:
    connect.part_file
mysql-test-run: *** ERROR: There where errors/warnings in server logs after running test cases.

Comment by Olivier Bertrand [ 2014-11-20 ]

Helena,

I did what you said and here is the result:

buggynours@UBUNTU:~/repos/10.0.14/10.0-connect/mysql-test$ perl ./mtr connect.part_table connect.part_file --valgrind-mysqld --noreorder
Logging: ./mtr  connect.part_table connect.part_file --valgrind-mysqld --noreorder
Turning on valgrind for mysqld(s) only
Running valgrind with options " --show-reachable=yes --quiet "
vardir: /home/buggynours/repos/10.0.14/10.0-connect/mysql-test/var
Checking leftover processes...
Removing old var directory...
Creating var directory '/home/buggynours/repos/10.0.14/10.0-connect/mysql-test/var'...
Checking supported features...
MariaDB Version 10.0.15-MariaDB
 - SSL connections supported
Collecting tests...
Installing system database...
 
==============================================================================
 
TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------
 
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
connect.part_table                       [ pass ]   2709
connect.part_file                        [ pass ]   1079
valgrind_report                          [ pass ]       
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 3.788 of 21 seconds executing testcases
 
Completed: All 3 tests were successful.
 
buggynours@UBUNTU:~/repos/10.0.14/10.0-connect/mysql-test$

I did it ten times with the same result. However, I admit there is a problem somewhere that comes up only randomly depending on I don't know what. The problem is that without any more information, it will be very difficult to find out the cause, except of a trivial and visible error.

On my machine, Valgrind warnings/errors are logged with a description of the pile of called functions before the error occurs. It is too bad I could not record this the one and only time I got this warning. Can you try to get this information? If it is not directly printed on screen, it may be found in a log file.

It is also strange that the lost memory were not the same on my machine and on yours. This can depends on the environment or on the compiler. I am using ubuntu and compiling with gcc.

Comment by Elena Stepanova [ 2014-11-20 ]

Olivier,

You didn't really miss much by not saving results of that run, this line that we see in the output is all that appears in the error log. It looks like this (again, this is a full unabridged error log, see the last line):

$ cat var/log/mysqld.1.err
CURRENT_TEST: connect.part_file
141120 20:10:10 [Note] Plugin 'InnoDB' is disabled.
141120 20:10:10 [Note] Plugin 'XTRADB_READ_VIEW' is disabled.
141120 20:10:10 [Note] Plugin 'XTRADB_INTERNAL_HASH_TABLES' is disabled.
141120 20:10:10 [Note] Plugin 'XTRADB_RSEG' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_TRX' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_LOCK_WAITS' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_CMP_RESET' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_CMPMEM' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_CMPMEM_RESET' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_CMP_PER_INDEX_RESET' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_BUFFER_PAGE' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_BUFFER_PAGE_LRU' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_BUFFER_POOL_STATS' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_METRICS' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_FT_DEFAULT_STOPWORD' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_FT_DELETED' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_FT_BEING_DELETED' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_FT_CONFIG' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_FT_INDEX_CACHE' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_FT_INDEX_TABLE' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_SYS_TABLES' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_SYS_TABLESTATS' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_SYS_INDEXES' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_SYS_COLUMNS' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_SYS_FIELDS' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_SYS_FOREIGN' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_SYS_FOREIGN_COLS' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_SYS_TABLESPACES' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_SYS_DATAFILES' is disabled.
141120 20:10:10 [Note] Plugin 'INNODB_CHANGED_PAGES' is disabled.
141120 20:10:11 [Note] CONNECT: Version 1.03.0003 Nov 14 2014 04:30:18
141120 20:10:11 [Warning] /data/repo/bzr/10.0/sql/mysqld: unknown variable 'loose-feedback-user-info=mysql-test'
141120 20:10:13 [Note] Server socket created on IP: '::'.
141120 20:10:14 [Note] Event Scheduler: Loaded 0 events
141120 20:10:14 [Note] /data/repo/bzr/10.0/sql/mysqld: ready for connections.
Version: '10.0.15-MariaDB-debug-log'  socket: '/data/repo/bzr/10.0/mysql-test/var/tmp/mysqld.1.sock'  port: 16000  Source distribution
CURRENT_TEST: connect.part_table
141120 20:10:34 [Note] /data/repo/bzr/10.0/sql/mysqld: Normal shutdown
 
141120 20:10:34 [Note] Event Scheduler: Purging the queue. 0 events
141120 20:10:35 [Note] Debug sync points hit:                   7394
141120 20:10:35 [Note] Debug sync points executed:              0
141120 20:10:35 [Note] Debug sync points max active per thread: 0
141120 20:10:35 [Note] /data/repo/bzr/10.0/sql/mysqld: Shutdown complete
 
Warning: 65536 bytes lost at 0xfaea070, allocated by T@0 at 

I am on Debian Wheezy 64 bit, valgrind 3.10.0, my full build parameters are here:

-- Running cmake version 2.8.9
-- MariaDB 10.0.15
-- Packaging as: mariadb-10.0.15-Linux-x86_64
-- suffixes <.so;.a>
-- OPENSSL_INCLUDE_DIR = /usr/include
-- OPENSSL_LIBRARIES = /usr/lib/x86_64-linux-gnu/libssl.so
-- CRYPTO_LIBRARY = /usr/lib/x86_64-linux-gnu/libcrypto.so
-- OPENSSL_MAJOR_VERSION = 1
-- SSL_LIBRARIES = /usr/lib/x86_64-linux-gnu/libssl.so;/usr/lib/x86_64-linux-gnu/libcrypto.so;dl
-- Could NOT find PkgConfig (missing:  PKG_CONFIG_EXECUTABLE) 
-- Configuring OQGraph
-- Boost version: 1.49.0
-- OQGraph OK
-- CONNECT: GCC: Some warnings disabled
-- Configuring done
-- Generating done
-- Build files have been written to: /home/elenst/bzr/10.0
-- Cache values
// path to the executable
ACLOCAL_EXECUTABLE:FILEPATH=/usr/bin/aclocal
 
// Path to a library.
AIO_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libaio.so
 
// path to the executable
AUTOCONF_EXECUTABLE:FILEPATH=/usr/bin/autoconf
 
// path to the executable
AUTOHEADER_EXECUTABLE:FILEPATH=/usr/bin/autoheader
 
// path to the executable
AUTOMAKE_EXECUTABLE:FILEPATH=/usr/bin/automake
 
// 
BACKUP_TEST:BOOL=OFF
 
// path to the bison executable
BISON_EXECUTABLE:FILEPATH=/usr/bin/bison
 
// CTest build name
BUILDNAME:STRING=ft-index Debug Linux-3.2.0-4-amd64 x86_64 GNU c++ 4.7.2
 
// Build the testing tree.
BUILD_TESTING:BOOL=OFF
 
// path to the executable
BZR_EXECUTABLE:FILEPATH=/usr/bin/bzr
 
// The directory containing a CMake configuration file for Boost.
Boost_DIR:PATH=Boost_DIR-NOTFOUND
 
// Path to a file.
Boost_INCLUDE_DIR:PATH=/usr/include
 
// Boost library directory
Boost_LIBRARY_DIRS:FILEPATH=/usr/lib
 
// path to the executable
CAT_EXECUTABLE:FILEPATH=/bin/cat
 
// Path to a program.
CMAKE_AR:FILEPATH=/usr/bin/ar
 
// Choose the type of build, options are: None(CMAKE_CXX_FLAGS or CMAKE_C_FLAGS used) Debug Release RelWithDebInfo MinSizeRel
CMAKE_BUILD_TYPE:STRING=Debug
 
// Enable/Disable color output during build.
CMAKE_COLOR_MAKEFILE:BOOL=ON
 
// CXX compiler.
CMAKE_CXX_COMPILER:FILEPATH=/usr/bin/c++
 
// Flags used by the compiler during all build types.
CMAKE_CXX_FLAGS:STRING=
 
// Flags used by the compiler during debug builds.
CMAKE_CXX_FLAGS_DEBUG:STRING=-g
 
// Flags used by the compiler during release minsize builds.
CMAKE_CXX_FLAGS_MINSIZEREL:STRING=-Os -DNDEBUG
 
// Flags used by the compiler during release builds (/MD /Ob1 /Oi /Ot /Oy /Gs will produce slightly less optimized but smaller files).
CMAKE_CXX_FLAGS_RELEASE:STRING=-O3 -DNDEBUG
 
// Flags used by the compiler during Release with Debug Info builds.
CMAKE_CXX_FLAGS_RELWITHDEBINFO:STRING=-O2 -g
 
// C compiler.
CMAKE_C_COMPILER:FILEPATH=/usr/bin/gcc
 
// Flags used by the compiler during all build types.
CMAKE_C_FLAGS:STRING=
 
// Flags used by the compiler during debug builds.
CMAKE_C_FLAGS_DEBUG:STRING=-g
 
// Flags used by the compiler during release minsize builds.
CMAKE_C_FLAGS_MINSIZEREL:STRING=-Os -DNDEBUG
 
// Flags used by the compiler during release builds (/MD /Ob1 /Oi /Ot /Oy /Gs will produce slightly less optimized but smaller files).
CMAKE_C_FLAGS_RELEASE:STRING=-O3 -DNDEBUG
 
// Flags used by the compiler during Release with Debug Info builds.
CMAKE_C_FLAGS_RELWITHDEBINFO:STRING=-O2 -g
 
// Flags used by the linker.
CMAKE_EXE_LINKER_FLAGS:STRING= 
 
// Flags used by the linker during debug builds.
CMAKE_EXE_LINKER_FLAGS_DEBUG:STRING=
 
// Flags used by the linker during release minsize builds.
CMAKE_EXE_LINKER_FLAGS_MINSIZEREL:STRING=
 
// Flags used by the linker during release builds.
CMAKE_EXE_LINKER_FLAGS_RELEASE:STRING=
 
// Flags used by the linker during Release with Debug Info builds.
CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO:STRING=
 
// Enable/Disable output of compile commands during generation.
CMAKE_EXPORT_COMPILE_COMMANDS:BOOL=OFF
 
// install prefix
CMAKE_INSTALL_PREFIX:PATH=/usr/local/mysql
 
// Path to a program.
CMAKE_LINKER:FILEPATH=/usr/bin/ld
 
// Path to a program.
CMAKE_MAKE_PROGRAM:FILEPATH=/usr/bin/make
 
// Flags used by the linker during the creation of modules.
CMAKE_MODULE_LINKER_FLAGS:STRING= 
 
// Flags used by the linker during debug builds.
CMAKE_MODULE_LINKER_FLAGS_DEBUG:STRING=
 
// Flags used by the linker during release minsize builds.
CMAKE_MODULE_LINKER_FLAGS_MINSIZEREL:STRING=
 
// Flags used by the linker during release builds.
CMAKE_MODULE_LINKER_FLAGS_RELEASE:STRING=
 
// Flags used by the linker during Release with Debug Info builds.
CMAKE_MODULE_LINKER_FLAGS_RELWITHDEBINFO:STRING=
 
// Path to a program.
CMAKE_NM:FILEPATH=/usr/bin/nm
 
// Path to a program.
CMAKE_OBJCOPY:FILEPATH=/usr/bin/objcopy
 
// Path to a program.
CMAKE_OBJDUMP:FILEPATH=/usr/bin/objdump
 
// Path to a program.
CMAKE_RANLIB:FILEPATH=/usr/bin/ranlib
 
// Flags used by the linker during the creation of dll's.
CMAKE_SHARED_LINKER_FLAGS:STRING= 
 
// Flags used by the linker during debug builds.
CMAKE_SHARED_LINKER_FLAGS_DEBUG:STRING=
 
// Flags used by the linker during release minsize builds.
CMAKE_SHARED_LINKER_FLAGS_MINSIZEREL:STRING=
 
// Flags used by the linker during release builds.
CMAKE_SHARED_LINKER_FLAGS_RELEASE:STRING=
 
// Flags used by the linker during Release with Debug Info builds.
CMAKE_SHARED_LINKER_FLAGS_RELWITHDEBINFO:STRING=
 
// If set, runtime paths are not added when installing shared libraries, but are added when building.
CMAKE_SKIP_INSTALL_RPATH:BOOL=NO
 
// If set, runtime paths are not added when using shared libraries.
CMAKE_SKIP_RPATH:BOOL=NO
 
// Path to a program.
CMAKE_STRIP:FILEPATH=/usr/bin/strip
 
// Revision of tokudb.
CMAKE_TOKUDB_REVISION:STRING=0
 
// If true, cmake will use relative paths in makefiles and projects.
CMAKE_USE_RELATIVE_PATHS:BOOL=OFF
 
// If this value is on, makefiles will be generated without the .SILENT directive, and all commands will be echoed to the console during the make.  This is useful for debugging only. With Visual Studio IDE projects all commands are done without /nologo.
CMAKE_VERBOSE_MAKEFILE:BOOL=FALSE
 
// Set to true if this is a community build
COMMUNITY_BUILD:BOOL=ON
 
// Compile CONNECT storage engine with LIBXML2 support
CONNECT_WITH_LIBXML2:BOOL=ON
 
// Compile CONNECT storage engine with remote MySQL connection support
CONNECT_WITH_MYSQL:BOOL=ON
 
// Compile CONNECT storage engine with ODBC support
CONNECT_WITH_ODBC:BOOL=ON
 
// Compile CONNECT storage engine with index file mapping support
CONNECT_WITH_XMAP:BOOL=ON
 
// Path to a library.
CRYPTO_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libcrypto.so
 
// Path to a file.
CURSES_CURSES_H_PATH:PATH=/usr/include
 
// Path to a library.
CURSES_CURSES_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libcurses.so
 
// Path to a library.
CURSES_EXTRA_LIBRARY:FILEPATH=CURSES_EXTRA_LIBRARY-NOTFOUND
 
// Path to a library.
CURSES_FORM_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libform.so
 
// Path to a file.
CURSES_HAVE_CURSES_H:FILEPATH=/usr/include/curses.h
 
// The curses include path
CURSES_INCLUDE_PATH:FILEPATH=/usr/include
 
// The curses library
CURSES_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libcurses.so
 
// Path to a library.
CURSES_NCURSES_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libncurses.so
 
// 
CYBOZU:BOOL=OFF
 
// Don't build shared libraries, compile code as position-dependent
DISABLE_SHARED:BOOL=OFF
 
// Path to a program.
DTRACE:FILEPATH=DTRACE-NOTFOUND
 
// If we should should enable LOAD DATA LOCAL by default
ENABLED_LOCAL_INFILE:BOOL=OFF
 
// Enable profiling
ENABLED_PROFILING:BOOL=ON
 
// Enable debug sync (debug builds only)
ENABLE_DEBUG_SYNC:BOOL=ON
 
// Enable gcov (debug, Linux builds only)
ENABLE_GCOV:BOOL=OFF
 
// Path to a library.
EVENT_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libevent.so
 
// The curses form library
FORM_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libform.so
 
// Path to a program.
GETCONF:FILEPATH=/usr/bin/getconf
 
// path to the executable
GLIBTOOLIZE_EXECUTABLE:FILEPATH=GLIBTOOLIZE_EXECUTABLE-NOTFOUND
 
// groonga default document root
GRN_DEFAULT_DOCUMENT_ROOT:PATH=/usr/local/mysql/share/groonga/html/admin
 
// groonga default document root base path
GRN_DEFAULT_DOCUMENT_ROOT_BASE:PATH=html/admin
 
// groonga default match escalation threshold
GRN_DEFAULT_MATCH_ESCALATION_THRESHOLD:STRING=0
 
// groonga default relative document root
GRN_DEFAULT_RELATIVE_DOCUMENT_ROOT:PATH=share/groonga/html/admin
 
// timeout to acquire a lock.
GRN_LOCK_TIMEOUT:STRING=10000000
 
// wait time in nanosecond to acquire a lock.
GRN_LOCK_WAIT_TIME_NANOSECOND:STRING=1000000
 
// log file path
GRN_LOG_PATH:FILEPATH=/usr/local/mysql/var/log/groonga/groonga.log
 
// mecab-config path
GRN_MECAB_CONFIG:FILEPATH=mecab-config
 
// Path to a program.
GRN_MECAB_CONFIG_ABSOLUTE_PATH:FILEPATH=GRN_MECAB_CONFIG_ABSOLUTE_PATH-NOTFOUND
 
// DANGER!!! groonga stack size. Normarlly, you should not change this variable.
GRN_STACK_SIZE:STRING=1024
 
// enable debug build.
GRN_WITH_DEBUG:BOOL=OFF
 
// use KyTea for morphological analysis
GRN_WITH_KYTEA:STRING=auto
 
// use libevent for suggestion
GRN_WITH_LIBEVENT:STRING=auto
 
// use LZO for data compression.
GRN_WITH_LZO:BOOL=OFF
 
// use MeCab for morphological analysis
GRN_WITH_MECAB:STRING=auto
 
// use MessagePack for suggestion
GRN_WITH_MESSAGE_PACK:STRING=auto
 
// use mruby
GRN_WITH_MRUBY:BOOL=OFF
 
// use NFKC based UTF8 normalization.
GRN_WITH_NFKC:BOOL=ON
 
// use ZeroMQ for suggestion
GRN_WITH_ZEROMQ:STRING=auto
 
// use zlib for data compression.
GRN_WITH_ZLIB:BOOL=OFF
 
// Path to a program.
GROFF:FILEPATH=/usr/bin/groff
 
// path to the executable
GTAR_EXECUTABLE:FILEPATH=GTAR_EXECUTABLE-NOTFOUND
 
// BIN installation directory
INSTALL_BINDIR:STRING=bin
 
// DOC installation directory
INSTALL_DOCDIR:STRING=docs
 
// DOCREADME installation directory
INSTALL_DOCREADMEDIR:STRING=.
 
// INCLUDE installation directory
INSTALL_INCLUDEDIR:STRING=include/mysql
 
// INFO installation directory
INSTALL_INFODIR:STRING=docs
 
// Installation directory layout. Options are: STANDALONE (as in zip or tar.gz installer) RPM DEB SVR4
INSTALL_LAYOUT:STRING=STANDALONE
 
// LIB installation directory
INSTALL_LIBDIR:STRING=lib
 
// MAN installation directory
INSTALL_MANDIR:STRING=man
 
// MYSQLDATA installation directory
INSTALL_MYSQLDATADIR:STRING=data
 
// MYSQLSHARE installation directory
INSTALL_MYSQLSHAREDIR:STRING=share
 
// MYSQLTEST installation directory
INSTALL_MYSQLTESTDIR:STRING=mysql-test
 
// PLUGIN installation directory
INSTALL_PLUGINDIR:STRING=lib/plugin
 
// SBIN installation directory
INSTALL_SBINDIR:STRING=bin
 
// SCRIPT installation directory
INSTALL_SCRIPTDIR:STRING=scripts
 
// SHARE installation directory
INSTALL_SHAREDIR:STRING=share
 
// SQLBENCH installation directory
INSTALL_SQLBENCHDIR:STRING=.
 
// SUPPORTFILES installation directory
INSTALL_SUPPORTFILESDIR:STRING=support-files
 
// SYSCONF2 installation directory
INSTALL_SYSCONF2DIR:STRING=
 
// SYSCONF installation directory
INSTALL_SYSCONFDIR:STRING=
 
// UNIX_ADDR installation directory
INSTALL_UNIX_ADDRDIR:STRING=/tmp/mysql.sock
 
// Path to a file.
Judy_INCLUDE_DIR:PATH=/usr/include
 
// Path to a library.
Judy_LIBRARIES:FILEPATH=/usr/lib/libJudy.so
 
// Name of libtokufractaltree.so
LIBTOKUDB:STRING=tokufractaltree
 
// Name of libtokuportability.so
LIBTOKUPORTABILITY:STRING=tokuportability
 
// path to the executable
LIBTOOLIZE_EXECUTABLE:FILEPATH=/usr/bin/libtoolize
 
// Path to a file.
LIBXML2_INCLUDE_DIR:PATH=/usr/include/libxml2
 
// Path to a library.
LIBXML2_LIBRARIES:FILEPATH=/usr/lib/x86_64-linux-gnu/libxml2.so
 
// Path to a program.
LIBXML2_XMLLINT_EXECUTABLE:FILEPATH=LIBXML2_XMLLINT_EXECUTABLE-NOTFOUND
 
// Set the entity that appears as the manufacturer of packages that support a manufacturer field.
MANUFACTURER:STRING=Built from Source
 
// The default fulltext parser
MRN_DEFAULT_PARSER:STRING=TokenBigram
 
// default MySQL data directory
MYSQL_DATADIR:PATH=/usr/local/mysql/data
 
// MySQL maintainer-specific development environment. Options are: ON OFF AUTO.
MYSQL_MAINTAINER_MODE:STRING=OFF
 
// MySQL project name
MYSQL_PROJECT_NAME:STRING=MySQL
 
// Allow linking with GPLv2-incompatible system libraries. Only set it you never plan to distribute the resulting binaries
NOT_FOR_DISTRIBUTION:BOOL=OFF
 
// No need to use alarm to implement timeout
NO_ALARM:BOOL=1
 
// Path to a program.
NROFF:FILEPATH=/usr/bin/nroff
 
// Specify the directory containing sql.h.
ODBC_INCLUDE_DIR:PATH=/usr/include
 
// Specify the ODBC driver manager library here.
ODBC_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libodbc.so
 
// Path to a file.
OPENSSL_INCLUDE_DIR:PATH=/usr/include
 
// Path to a library.
OPENSSL_LIBRARIES:FILEPATH=/usr/lib/x86_64-linux-gnu/libssl.so
 
// Path to a file.
OPENSSL_ROOT_DIR:PATH=/usr
 
// Buffer size parameter for pcregrep. See PCREGREP_BUFSIZE in config.h.in for details.
PCREGREP_BUFSIZE:STRING=20480
 
// Build pcregrep
PCRE_BUILD_PCREGREP:BOOL=ON
 
// Build the tests
PCRE_BUILD_TESTS:BOOL=ON
 
// Internal link size (2, 3 or 4 allowed). See LINK_SIZE in config.h.in for details.
PCRE_LINK_SIZE:STRING=2
 
// Default limit on internal looping. See MATCH_LIMIT in config.h.in for details.
PCRE_MATCH_LIMIT:STRING=10000000
 
// Default limit on internal recursion. See MATCH_LIMIT_RECURSION in config.h.in for details.
PCRE_MATCH_LIMIT_RECURSION:STRING=MATCH_LIMIT
 
// What to recognize as a newline (one of CR, LF, CRLF, ANY, ANYCRLF).
PCRE_NEWLINE:STRING=LF
 
// If ON, then don't use stack recursion when matching. See NO_RECURSE in config.h.in for details.
PCRE_NO_RECURSE:BOOL=ON
 
// Default nested parentheses limit. See PARENS_NEST_LIMIT in config.h.in for details.
PCRE_PARENS_NEST_LIMIT:STRING=250
 
// Threshold for malloc() usage. See POSIX_MALLOC_THRESHOLD in config.h.in for details.
PCRE_POSIX_MALLOC_THRESHOLD:STRING=10
 
// Show the final configuration report
PCRE_SHOW_REPORT:BOOL=OFF
 
// ON=Backslash-R matches only LF CR and CRLF, OFF=Backslash-R matches all Unicode Linebreaks
PCRE_SUPPORT_BSR_ANYCRLF:BOOL=OFF
 
// Unicode properties
PCRE_SUPPORT_UNICODE_PROPERTIES:BOOL=ON
 
// pkg-config executable
PKG_CONFIG_EXECUTABLE:FILEPATH=PKG_CONFIG_EXECUTABLE-NOTFOUND
 
// Path to a file.
READLINE_INCLUDE_DIR:PATH=/usr/include/readline
 
// Path to a library.
READLINE_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libreadline.so
 
// path to the executable
TAR_EXECUTABLE:FILEPATH=/bin/tar
 
// PATH to MySQL TMP dir. Defaults to the P_tmpdir macro in <stdio.h>
TMPDIR:PATH=
 
// Path to data files for tests
TOKUDB_DATA:FILEPATH=/home/elenst/bzr/10.0/storage/tokudb/ft-index/../tokudb.data
 
// Enable paranoid asserts.
TOKU_DEBUG_PARANOID:BOOL=OFF
 
// Path to a file.
Thrift_INCLUDE_DIRS:PATH=Thrift_INCLUDE_DIRS-NOTFOUND
 
// Path to a library.
Thrift_LIBS:FILEPATH=Thrift_LIBS-NOTFOUND
 
// Use Aria for temporary tables
USE_ARIA_FOR_TMP_TABLES:BOOL=ON
 
// 
USE_BDB:BOOL=OFF
 
// Use gcov for test coverage.
USE_GCOV:BOOL=OFF
 
// Build to run safely under valgrind (often slower).
USE_VALGRIND:BOOL=OFF
 
// OFF
WITHOUT_SERVER:BOOL=OFF
 
// Link ARCHIVE statically to the server
WITH_ARCHIVE_STORAGE_ENGINE:BOOL=OFF
 
// Link ARIA statically to the server
WITH_ARIA_STORAGE_ENGINE:BOOL=ON
 
// Enable address sanitizer
WITH_ASAN:BOOL=OFF
 
// Implement atomic operations using pthread rwlocks (rwlocks); or atomic CPU
instructions for multi-processor (smp) or uniprocessor (up)
configuration. By default gcc built-in sync functions are used,
if available and 'smp' configuration otherwise.
WITH_ATOMIC_OPS:STRING=
 
// Link BLACKHOLE statically to the server
WITH_BLACKHOLE_STORAGE_ENGINE:BOOL=OFF
 
// Link CONNECT statically to the server
WITH_CONNECT_STORAGE_ENGINE:BOOL=OFF
 
// Compile MySQL with embedded server
WITH_EMBEDDED_SERVER:BOOL=OFF
 
// Options are: none complex all
WITH_EXTRA_CHARSETS:STRING=all
 
// Compile with fast mutexes
WITH_FAST_MUTEXES:BOOL=OFF
 
// Link FEDERATEDX statically to the server
WITH_FEDERATEDX_STORAGE_ENGINE:BOOL=OFF
 
// Link FEEDBACK statically to the server
WITH_FEEDBACK:BOOL=OFF
 
// Build with jemalloc. Possible values are 'yes', 'no', 'auto'
WITH_JEMALLOC:STRING=auto
 
// Compile with tcp wrappers support
WITH_LIBWRAP:BOOL=OFF
 
// Link LOCALES statically to the server
WITH_LOCALES:BOOL=OFF
 
// Link METADATA_LOCK_INFO statically to the server
WITH_METADATA_LOCK_INFO:BOOL=OFF
 
// Additional linker flags for mysqld
WITH_MYSQLD_LDFLAGS:STRING=
 
// Link PARTITION statically to the server
WITH_PARTITION_STORAGE_ENGINE:BOOL=ON
 
// Which pcre to use (possible values are 'bundled', 'system', or 'auto')
WITH_PCRE:STRING=auto
 
// Link PERFSCHEMA statically to the server
WITH_PERFSCHEMA_STORAGE_ENGINE:BOOL=ON
 
// Generate PIC objects
WITH_PIC:BOOL=ON
 
// Link QUERY_CACHE_INFO statically to the server
WITH_QUERY_CACHE_INFO:BOOL=OFF
 
// Link QUERY_RESPONSE_TIME statically to the server
WITH_QUERY_RESPONSE_TIME:BOOL=OFF
 
// Use bundled readline
WITH_READLINE:BOOL=OFF
 
// Use safemalloc memory debugger. Will result in slower execution. Options are: ON OFF AUTO.
WITH_SAFEMALLOC:STRING=AUTO
 
// Link SEMISYNC_MASTER statically to the server
WITH_SEMISYNC_MASTER:BOOL=OFF
 
// Link SEMISYNC_SLAVE statically to the server
WITH_SEMISYNC_SLAVE:BOOL=OFF
 
// Link SEQUENCE statically to the server
WITH_SEQUENCE_STORAGE_ENGINE:BOOL=OFF
 
// Link SPHINX statically to the server
WITH_SPHINX_STORAGE_ENGINE:BOOL=OFF
 
// Link TEST_SQL_DISCOVERY statically to the server
WITH_TEST_SQL_DISCOVERY_STORAGE_ENGINE:BOOL=OFF
 
// Compile MySQL with unit tests
WITH_UNIT_TESTS:BOOL=ON
 
// Valgrind instrumentation
WITH_VALGRIND:BOOL=OFF
 
// Link XTRADB statically to the server
WITH_XTRADB_STORAGE_ENGINE:BOOL=ON
 
// Which zlib to use (possible values are 'bundled' or 'system')
WITH_ZLIB:STRING=system
 
// Where to find sources for xz (lzma).
XZ_SOURCE_DIR:FILEPATH=/home/elenst/bzr/10.0/storage/tokudb/ft-index/third_party/xz-4.999.9beta
 
// Path to a file.
ZLIB_INCLUDE_DIR:PATH=/usr/include
 
// Path to a library.
ZLIB_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libz.so

Since it's reproducible relatively well on my machine, I can be your hands for the initial steps of investigation. Tell me what you'd want to do/try, I will try it and paste the results.

Comment by Olivier Bertrand [ 2014-11-20 ]

Hmmm... things are going stranger and stranger!

Firstly, to have an example of Valgring error output, I voluntarily added an error in TDBCSV::PrepareWriting

  char *valtest = (char*)malloc(4321);

This memory is never freed. However, when running the test Valgrind did not complain...
Then I added more to tilt it:

  char *valtest = (char*)malloc(4321);
  printf("valtest %s\n", valtest);

Hurrah! this time Valgrind awaked:

buggynours@UBUNTU:~/repos/10.0.14/10.0-connect/mysql-test$ ./mtr --valgrind connect.part_valg
Logging: ./mtr  --valgrind connect.part_valg
Turning on valgrind for all executables
Running valgrind with options " --show-reachable=yes --quiet "
vardir: /home/buggynours/repos/10.0.14/10.0-connect/mysql-test/var
Checking leftover processes...
Removing old var directory...
Creating var directory '/home/buggynours/repos/10.0.14/10.0-connect/mysql-test/var'...
Checking supported features...
MariaDB Version 10.0.15-MariaDB
 - SSL connections supported
Collecting tests...
Installing system database...
 
==============================================================================
 
TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------
 
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
set @@global.connect_exact_info=ON;
#
# Testing partitioning on inward table
#
CREATE TABLE t1 (
id INT NOT NULL,
msg VARCHAR(32)
) ENGINE=CONNECT TABLE_TYPE=CSV AVG_ROW_LENGTH=10
PARTITION BY RANGE(id) (
PARTITION first VALUES LESS THAN(10),
PARTITION middle VALUES LESS THAN(50),
PARTITION last VALUES LESS THAN(MAXVALUE));
INSERT INTO t1 VALUES(4, 'four'),(24, 'twenty four');
INSERT INTO t1 VALUES(7,'seven'),(10,'ten'),(40,'forty'),(60,'sixty'),(81,'eighty one');
SELECT * FROM t1;
id	msg
4	four
7	seven
24	twenty four
10	ten
40	forty
60	sixty
81	eighty one
UPDATE t1 set msg = 'quatre' WHERE id = 4;
SELECT * FROM t1 WHERE id = 4;
id	msg
4	quatre
DROP TABLE t1;
set @@global.connect_exact_info=OFF;
connect.part_valg                        [ fail ]  Found warnings/errors in server log file!
        Test ended at 2014-11-20 19:14:28
line
==12897== Thread 4:
==12897== Conditional jump or move depends on uninitialised value(s)
==12897==    at 0x425231F: vfprintf (vfprintf.c:1661)
==12897==    by 0x4304D8F: __printf_chk (printf_chk.c:35)
==12897==    by 0x7E07744: TDBCSV::PrepareWriting(_global*) (stdio2.h:104)
==12897==    by 0x7E0841B: TDBCSV::WriteDB(_global*) (tabfmt.cpp:1025)
==12897==    by 0x7DD77E2: CntWriteRow(_global*, TDB*) (connect.cc:506)
==12897==    by 0x7DCE574: ha_connect::write_row(unsigned char*) (ha_connect.cc:3003)
==12897==    by 0x8329D66: handler::ha_write_row(unsigned char*) (handler.cc:5961)
==12897==    by 0x87013A7: ha_partition::write_row(unsigned char*) (ha_partition.cc:4165)
==12897==    by 0x8329CF0: handler::ha_write_row(unsigned char*) (handler.cc:5961)
==12897==    by 0x81C7983: write_record(THD*, TABLE*, st_copy_info*) (sql_insert.cc:1835)
==12897==    by 0x81CAE81: mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool) (sql_insert.cc:960)
==12897==    by 0x81DF1D8: mysql_execute_command(THD*) (sql_parse.cc:3431)
==12897==    by 0x81E58F7: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6406)
==12897==    by 0x81E7B51: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1299)
==12897==    by 0x81E87F7: do_command(THD*) (sql_parse.cc:996)
==12897==    by 0x8298570: do_handle_one_connection(THD*) (sql_connect.cc:1379)
^ Found warnings in /home/buggynours/repos/10.0.14/10.0-connect/mysql-test/var/log/mysqld.1.err
ok
 
 - saving '/home/buggynours/repos/10.0.14/10.0-connect/mysql-test/var/log/connect.part_valg/' to '/home/buggynours/repos/10.0.14/10.0-connect/mysql-test/var/log/connect.part_valg/'
***Warnings generated in error logs during shutdown after running tests: connect.part_valg
 
==12897== Thread 1:
==12897== 4,321 bytes in 1 blocks are definitely lost in loss record 4 of 5
==12897==    at 0x402A17C: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==12897==    by 0x7E0772A: ???
==12897==    by 0x7E0841B: ???
==12897==    by 0x7DD77E2: ???
==12897==    by 0x7DD78A7: ???
==12897==    by 0x7DCE694: ???
==12897==    by 0x8329EDA: handler::ha_update_row(unsigned char const*, unsigned char*) (handler.cc:5994)
==12897==    by 0x8701781: ha_partition::update_row(unsigned char const*, unsigned char*) (ha_partition.cc:4255)
==12897==    by 0x8329E5A: handler::ha_update_row(unsigned char const*, unsigned char*) (handler.cc:5994)
==12897==    by 0x8269AA0: mysql_update(THD*, TABLE_LIST*, List<Item>&, List<Item>&, Item*, unsigned int, st_order*, unsigned long long, enum_duplicates, bool, unsigned long long*, unsigned long long*) (sql_update.cc:803)
==12897==    by 0x81DF30A: mysql_execute_command(THD*) (sql_parse.cc:3294)
==12897==    by 0x81E58F7: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6406)
==12897==    by 0x81E7B51: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1299)
==12897==    by 0x81E87F7: do_command(THD*) (sql_parse.cc:996)
==12897==    by 0x8298570: do_handle_one_connection(THD*) (sql_connect.cc:1379)
==12897==    by 0x82985D5: handle_one_connection (sql_connect.cc:1293)
==12897== 30,247 bytes in 7 blocks are definitely lost in loss record 5 of 5
==12897==    at 0x402A17C: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==12897==    by 0x7E0772A: ???
==12897==    by 0x7E0841B: ???
==12897==    by 0x7DD77E2: ???
==12897==    by 0x7DCE574: ???
==12897==    by 0x8329D66: handler::ha_write_row(unsigned char*) (handler.cc:5961)
==12897==    by 0x87013A7: ha_partition::write_row(unsigned char*) (ha_partition.cc:4165)
==12897==    by 0x8329CF0: handler::ha_write_row(unsigned char*) (handler.cc:5961)
==12897==    by 0x81C7983: write_record(THD*, TABLE*, st_copy_info*) (sql_insert.cc:1835)
==12897==    by 0x81CAE81: mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool) (sql_insert.cc:960)
==12897==    by 0x81DF1D8: mysql_execute_command(THD*) (sql_parse.cc:3431)
==12897==    by 0x81E58F7: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6406)
==12897==    by 0x81E7B51: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1299)
==12897==    by 0x81E87F7: do_command(THD*) (sql_parse.cc:996)
==12897==    by 0x8298570: do_handle_one_connection(THD*) (sql_connect.cc:1379)
==12897==    by 0x82985D5: handle_one_connection (sql_connect.cc:1293)
 
valgrind_report                          [ pass ]       
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 0.000 of 19 seconds executing testcases
mysql-test-run: WARNING: Got errors/warnings while running tests, please examine '/home/buggynours/repos/10.0.14/10.0-connect/mysql-test/var/log/warnings' for details.
 
Warnings in log: Failed 1/2 tests, 50.00% were successful.
 
Failing test(s): connect.part_valg
 
The log files in var/log may give you some hint of what went wrong.
 
If you want to report this error, please read first the documentation
at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html
 
Errors/warnings were found in logfiles during server shutdown after running the
following sequence(s) of tests:
    connect.part_valg
mysql-test-run: *** ERROR: there were failing test cases

Several things to remark:

  1. As you see, Valgring prints the full call stack enable to see where in the code the error occurs. It is what interest me to see in the test you made. I don't know why you do not get it but there may be some parameters to set in order to get it printed.
  2. Valgrind correctly reported the two errors: using an uninitialised variable and 4321 bytes of lost memory.
  3. An additional lost of 30247 bytes is reported that is not due to to this code addition. It may be the one that was not reported previously (remember the added one was not before adding the printf) Unfortunately the calls stack is interrupted by ??? preventing to see where the allocation was done.

Therefore can we really trust Valgrind? Nevertheless let's try to imagine what could have happen. It can be easily verified by grep that malloc is not used in the CONNECT functions prone to be executed by this test. Besides, in the first printout I had seen, I am almost sure that the call stack (that did not have these ???) did not include any CONNECT functions.

Here is the little test I used:

--source include/have_partition.inc
let $MYSQLD_DATADIR= `select @@datadir`;
 
set @@global.connect_exact_info=ON;
 
--echo #
--echo # Testing partitioning on inward table
--echo #
CREATE TABLE t1 (
  id INT NOT NULL,
  msg VARCHAR(32)
) ENGINE=CONNECT TABLE_TYPE=CSV AVG_ROW_LENGTH=10
PARTITION BY RANGE(id) (
PARTITION first VALUES LESS THAN(10),
PARTITION middle VALUES LESS THAN(50),
PARTITION last VALUES LESS THAN(MAXVALUE));
INSERT INTO t1 VALUES(4, 'four'),(24, 'twenty four');
INSERT INTO t1 VALUES(7,'seven'),(10,'ten'),(40,'forty'),(60,'sixty'),(81,'eighty one');
SELECT * FROM t1;
UPDATE t1 set msg = 'quatre' WHERE id = 4;
SELECT * FROM t1 WHERE id = 4;
 
DROP TABLE t1;
 
set @@global.connect_exact_info=OFF;

An interesting thing you could do is to check if it does raise Valgrind warnings, then to modify it to change the table engine to another engine. This to be sure that CONNECT is involved. Indeed there is still a possibility that the partition engine or some other progam be involved.

Sorry, I cannot do much more for the moment.

Comment by Elena Stepanova [ 2014-11-21 ]

Hi Olivier,

As you see, Valgring prints the full call stack enable to see where in the code the error occurs. It is what interest me to see in the test you made. I don't know why you do not get it but there may be some parameters to set in order to get it printed.

That's probably because what I (and buildbot) are getting is not a valgrind warning. I think you misunderstood it a bit.
I am getting it when I run the test with valgrind, but it doesn't necessarily mean that valgrind produces it. If you look at the buildbot page by the initial link you'll see that it runs without valgrind at all, and still gets the very same warning.
Apparently, whatever it is, valgrind doesn't catch it.

An interesting thing you could do is to check if it does raise Valgrind warnings

If you mean that this test should be run with the mentioned change in TDBCSV::PrepareWriting, the answer is yes and no. I'm getting the valgrind warning "Conditional jump or move", similar to yours (see below), and I'm also getting "Syscall param write(buf) points to uninitialised byte", but not "blocks lost". It might be the matter of different valgrind versions.

==15916== Conditional jump or move depends on uninitialised value(s)
==15916==    at 0x6B26D8A: vfprintf (vfprintf.c:1623)
==15916==    by 0x6B2F549: printf (printf.c:35)
==15916==    by 0xE0DB830: TDBCSV::PrepareWriting(_global*) (tabfmt.cpp:942)
==15916==    by 0xE0DBDA1: TDBCSV::WriteDB(_global*) (tabfmt.cpp:1022)
==15916==    by 0xE094021: CntWriteRow(_global*, TDB*) (connect.cc:514)
==15916==    by 0xE086FE3: ha_connect::write_row(unsigned char*) (ha_connect.cc:2808)
==15916==    by 0x883844: handler::ha_write_row(unsigned char*) (handler.cc:5953)
==15916==    by 0xE576D5: ha_partition::write_row(unsigned char*) (ha_partition.cc:4165)
==15916==    by 0x8837F5: handler::ha_write_row(unsigned char*) (handler.cc:5953)
==15916==    by 0x662C74: write_record(THD*, TABLE*, st_copy_info*) (sql_insert.cc:1835)
==15916==    by 0x660997: mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool) (sql_insert.cc:960)
==15916==    by 0x680DAF: mysql_execute_command(THD*) (sql_parse.cc:3432)
==15916==    by 0x68900A: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6407)
==15916==    by 0x67BDE3: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1299)
==15916==    by 0x67B188: do_command(THD*) (sql_parse.cc:996)
==15916==    by 0x79D91B: do_handle_one_connection(THD*) (sql_connect.cc:1379)
==15916== 

==15916== Thread 1:
==15916== Syscall param write(buf) points to uninitialised byte(s)
==15916==    at 0x6BB129D: ??? (syscall-template.S:82)
==15916==    by 0x6B53F22: _IO_file_write@@GLIBC_2.2.5 (fileops.c:1276)
==15916==    by 0x6B53B99: new_do_write (fileops.c:530)
==15916==    by 0x6B53EC4: _IO_do_write@@GLIBC_2.2.5 (fileops.c:503)
==15916==    by 0x6B55D06: _IO_flush_all_lockp (genops.c:849)
==15916==    by 0x6B56B14: _IO_cleanup (genops.c:1010)
==15916==    by 0x6B17E5D: __run_exit_handlers (exit.c:91)
==15916==    by 0x6B17F14: exit (exit.c:100)
==15916==    by 0x5BD98D: mysqld_exit(int) (mysqld.cc:1977)
==15916==    by 0x5C41A7: mysqld_main(int, char**) (mysqld.cc:5580)
==15916==    by 0x5BA27B: main (main.cc:25)
==15916==  Address 0x402304c is not stack'd, malloc'd or (recently) free'd

Comment by Olivier Bertrand [ 2014-11-21 ]

Elena,

If you mean that this test should be run with the mentioned change in TDBCSV::PrepareWriting

I didn't mean that. Running the small test without the change is just to check whether the warning is raised because of that particular UPDATE command. What you could do is to split the original test in parts to check which command(s) are causing the warning.

And also checking with the partition table made with another engine.

Thanks and regards,
Olivier

Comment by Elena Stepanova [ 2014-11-21 ]

Hi Olivier,

No, I am not getting the warnings on your test case. Here is the minimal one which triggers "bytes lost" warnings for me (they look not quite the same as the initial one, but I suppose it's just because there are more of them; the first warning from the three could be the same as the initial unfinished one):

--source include/have_partition.inc
CREATE TABLE t1 (
  id INT NOT NULL,
  msg VARCHAR(32)
) ENGINE=CONNECT TABLE_TYPE=CSV AVG_ROW_LENGTH=10
PARTITION BY RANGE(id) (
PARTITION first VALUES LESS THAN(10),
PARTITION middle VALUES LESS THAN(50),
PARTITION last VALUES LESS THAN(MAXVALUE));
--error ER_GET_ERRMSG
UPDATE t1 set id = 41 WHERE msg = 'four';

perl ./mtr part_file2  --valgrind-mysqld --mysqld=--plugin-load-add=ha_connect
 
...
CREATE TABLE t1 (
id INT NOT NULL,
msg VARCHAR(32)
) ENGINE=CONNECT TABLE_TYPE=CSV AVG_ROW_LENGTH=10
PARTITION BY RANGE(id) (
PARTITION first VALUES LESS THAN(10),
PARTITION middle VALUES LESS THAN(50),
PARTITION last VALUES LESS THAN(MAXVALUE));
UPDATE t1 set id = 41 WHERE msg = 'four';
ERROR HY000: Got error 174 'Cannot update column id because it is used for partitioning' from CONNECT
***Warnings generated in error logs during shutdown after running tests: main.part_file2
 
Warning: 65536 bytes lost at 0xfaa3070, allocated by T@0 at mysys/mf_iocache.c:231, mysys/mf_cache.c:69, sql/sql_update.cc:580, sql/sql_parse.cc:3295, sql/sql_parse.cc:6407, sql/sql_parse.cc:1301, sql/sql_parse.cc:996, sql/sql_connect.cc:1379
Warning:    8 bytes lost at 0xf850b70, allocated by T@0 at mysys/my_malloc.c:239, mysys/mf_cache.c:65, sql/sql_update.cc:580, sql/sql_parse.cc:3295, sql/sql_parse.cc:6407, sql/sql_parse.cc:1301, sql/sql_parse.cc:996, sql/sql_connect.cc:1379
Warning:   48 bytes lost at 0xf84ffb0, allocated by T@0 at mysys/my_malloc.c:239, mysys/mf_cache.c:64, sql/sql_update.cc:580, sql/sql_parse.cc:3295, sql/sql_parse.cc:6407, sql/sql_parse.cc:1301, sql/sql_parse.cc:996, sql/sql_connect.cc:1379
 
mysql-test-run: *** ERROR: Test suite aborted

I tried the same set with MyISAM, InnoDB, and Archive engines (the latter is a dynamic library, like Connect), no warnings there.
Of course, the other difference is that neither of them return an error on the update, it's Connect-specific, maybe the warnings are somehow related to it.

Comment by Olivier Bertrand [ 2014-11-21 ]

Helena,

Great! Thanks to this information I have a better idea of what is happening.
It begins in mysql_update (in sql_update.cc):

int mysql_update:
...
  We are doing a search on a key that is updated. In this case
  we go trough the matching rows, save a pointer to them and
  update these in a separate loop based on the pointer.
      IO_CACHE tempfile;
      if (open_cached_file(&tempfile, mysql_tmpdir,TEMP_PREFIX,
         DISK_BUFFER_SIZE, MYF(MY_WME)))
  goto err;
 
      /* If quick select is used, initialize it before retrieving rows. */
      if (select && select->quick && select->quick->reset())
      {
        close_cached_file(&tempfile);
        goto err;
      }
      table->file->try_semi_consistent_read(1);
 
      /*
        When we get here, we have one of the following options:
        A. query_plan.index == MAX_KEY
           This means we should use full table scan, and start it with
           init_read_record call
        B. query_plan.index != MAX_KEY
           B.1 quick select is used, start the scan with init_read_record
           B.2 quick select is not used, this is full index scan (with LIMIT)
               Full index scan must be started with init_read_record_idx
      */
 
      if (query_plan.index == MAX_KEY || (select && select->quick))
      {
        if (init_read_record(&info, thd, table, select, 0, 1, FALSE))
          goto err;
      }
      else
        init_read_record_idx(&info, thd, table, 1, query_plan.index, reverse);

It calls open_cached_file that calls init_io_cache

int init_io_cache:
...
      if ((info->buffer= (uchar*) my_malloc(buffer_block, flags)) != 0)
...

This is where the buffer of 65536 bytes is allocated. When updating is done,
for instance for a MyISAM table, there is a call to close_cached_file

void close_cached_file(IO_CACHE *cache)
{
  DBUG_ENTER("close_cached_file");
  if (my_b_inited(cache))
  {
    File file=cache->file;
    cache->file= -1;        /* Don't flush data */
    (void) end_io_cache(cache);
    if (file >= 0)
    {
      (void) my_close(file,MYF(0));
#ifdef CANT_DELETE_OPEN_FILES
      if (cache->file_name)
      {
  (void) my_delete(cache->file_name,MYF(MY_WME | ME_NOINPUT));
  my_free(cache->file_name);
      }
#endif
    }
    my_free(cache->dir);
    my_free(cache->prefix);
  }
  DBUG_VOID_RETURN;
}

and in end_io_cache

int end_io_cache(IO_CACHE *info)
...
    my_free(info->buffer);
...

the allocated memory is freed.

However, in the case of a CONNECT table, when calling init_read_record
different functions are called until ha_connect::rnd_init

int ha_connect::rnd_init(bool scan)
...
  if (OpenTable(g, xmod == MODE_DELETE))
    DBUG_RETURN(HA_ERR_INITIALIZATION);

and in ha_connect::OpenTable

int ha_connect::OpenTable(PGLOBAL g, bool del)
...
          if (part_id && bitmap_is_set(part_id, fp->field_index)) {
            // Trying to update a column used for partitioning
            // This cannot be currently done because it may require
            // a row to be moved in another partition.
            sprintf(g->Message,
              "Cannot update column %s because it is used for partitioning",
              p);
            return HA_ERR_INTERNAL_ERROR;
            } // endif part_id

This causes in mysql_update the statement:

        if (init_read_record(&info, thd, table, select, 0, 1, FALSE))
          goto err;
...
err:
  delete select;
  free_underlaid_joins(thd, select_lex);
  table->disable_keyread();
  thd->abort_on_warning= 0;
  DBUG_RETURN(1);

to return in error. Actually in delete select the function close_cached_file
is called but the cache parameter passed to it is not the one that have been
allocated previously and this function does nothing (my_b_inited(cache) is false)
This explains the warnings, memory being lost indeed.

Is this a CONNECT bug or a MariaDB bug? I can't tell but this could happen with
any engine returning an error in this place.

Perhaps, adding to CONNECT the flag HA_NOT_DELETE_WITH_CACHE could remove this problem
but this would be masking it instead of really fix it.

Comment by Elena Stepanova [ 2014-11-21 ]

Is this a CONNECT bug or a MariaDB bug?

It's probably best to ask serg about it.

Comment by Sergei Golubchik [ 2014-11-21 ]

Looks like a MariaDB bug to me. Another goto err few lines above is correctly accompanied with the close_cached_file(&tempfile).

I'll fix it. Thanks for pointing out exactly what line to fix!

Generated at Thu Feb 08 07:17:21 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.