[MDEV-12341] MariaRocks: rocksdb.persistent_cache test fails on some platforms Created: 2017-03-23  Updated: 2018-05-11

Status: Open
Project: MariaDB Server
Component/s: Storage Engine - RocksDB
Affects Version/s: 10.2.6
Fix Version/s: 10.4

Type: Bug Priority: Minor
Reporter: Sergei Petrunia Assignee: Sergei Petrunia
Resolution: Unresolved Votes: 0
Labels: upstream

Issue Links:
PartOf
is part of MDEV-9658 Make MyRocks in MariaDB stable Closed

 Description   

rocksdb.persistent_cache fails on some platforms.

rocksdb.persistent_cache                 [ fail ]
        Test ended at 2017-03-23 11:52:37
 
CURRENT_TEST: rocksdb.persistent_cache
mysqltest: At line 24: query 'set global rocksdb_force_flush_memtable_now=1' failed: 2013: Lost connection to MySQL server during query

...

2017-03-23 11:52:31 139947375835008 [Note] Server socket created on IP: '127.0.0.1'.
2017-03-23 11:52:31 139947375835008 [Note] /home/psergey/dev-git/10.2-mariarocks/sql/mysqld: ready for connections.
Version: '10.2.5-MariaDB-debug-log'  socket: '/home/psergey/dev-git/10.2-mariarocks/mysql-test/var/tmp/mysqld.1.sock'  port: 16000  Source distribution
2017-03-23 11:52:35 139947375219456 [Note] RocksDB: Manual memtable flush
 
mysqld: /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/table/block_based_table_reader.cc:634: static rocksdb::Slice rocksdb::BlockBasedTable::GetCacheKey(const char*, size_t, const rocksdb::BlockHandle&, char*): Assertion `cache_key_prefix_size != 0' failed.
170323 11:52:35 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

...

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/home/psergey/dev-git/10.2-mariarocks/sql/mysqld --defaults-group-suffix=.1 --d'.
Program terminated with signal SIGABRT, Aborted.
#0  __pthread_kill (threadid=<optimized out>, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:62
62      ../sysdeps/unix/sysv/linux/pthread_kill.c: No such file or directory.
#0  __pthread_kill (threadid=<optimized out>, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:62
#1  0x00007f48091d41c2 in my_write_core (sig=6) at /home/psergey/dev-git/10.2-mariarocks/mysys/stacktrace.c:477
#2  0x00007f4808a581f7 in handle_fatal_signal (sig=6) at /home/psergey/dev-git/10.2-mariarocks/sql/signal_handler.cc:296
#3  <signal handler called>
#4  0x00007f4805b54267 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
#5  0x00007f4805b55eca in __GI_abort () at abort.c:89
#6  0x00007f4805b4d03d in __assert_fail_base (fmt=0x7f4805caf028 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x7f47fef98885 "cache_key_prefix_size != 0", file=file@entry=0x7f47fef98810 "/home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/table/block_based_table_reader.cc", line=line@entry=634, function=function@entry=0x7f47fef9a500 <rocksdb::BlockBasedTable::GetCacheKey(char const*, unsigned long, rocksdb::BlockHandle const&, char*)::__PRETTY_FUNCTION__> "static rocksdb::Slice rocksdb::BlockBasedTable::GetCacheKey(const char*, size_t, const rocksdb::BlockHandle&, char*)") at assert.c:92
#7  0x00007f4805b4d0f2 in __GI___assert_fail (assertion=0x7f47fef98885 "cache_key_prefix_size != 0", file=0x7f47fef98810 "/home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/table/block_based_table_reader.cc", line=634, function=0x7f47fef9a500 <rocksdb::BlockBasedTable::GetCacheKey(char const*, unsigned long, rocksdb::BlockHandle const&, char*)::__PRETTY_FUNCTION__> "static rocksdb::Slice rocksdb::BlockBasedTable::GetCacheKey(const char*, size_t, const rocksdb::BlockHandle&, char*)") at assert.c:101
#8  0x00007f47fed630ac in rocksdb::BlockBasedTable::GetCacheKey (cache_key_prefix=0x7f4806715678 <std::string::_Rep::_S_empty_rep_storage+24> "", cache_key_prefix_size=0, handle=..., cache_key=0x7f47cb3f1550 "") at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/table/block_based_table_reader.cc:634
#9  0x00007f47fedabcca in rocksdb::PersistentCacheHelper::LookupRawPage (cache_options=..., handle=..., raw_data=0x7f47cb3f1610, raw_data_size=37) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/table/persistent_cache_helper.cc:57
#10 0x00007f47fed79d53 in rocksdb::ReadBlockContents (file=0x7f47cd019160, footer=..., read_options=..., handle=..., contents=0x7f47cb3f2af0, ioptions=..., decompression_requested=true, compression_dict=..., cache_options=...) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/table/format.cc:344
#11 0x00007f47fed626d9 in rocksdb::(anonymous namespace)::ReadBlockFromFile (file=0x7f47cd019160, footer=..., options=..., handle=..., result=0x7f47cb3f2b80, ioptions=..., do_uncompress=true, compression_dict=..., cache_options=..., global_seqno=18446744073709551615, read_amp_bytes_per_bit=0) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/table/block_based_table_reader.cc:77
#12 0x00007f47fed64791 in rocksdb::BlockBasedTable::ReadMetaBlock (rep=0x7f47cd0661c0, meta_block=0x7f47cb3f2c90, iter=0x7f47cb3f2ca0) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/table/block_based_table_reader.cc:917
#13 0x00007f47fed634e3 in rocksdb::BlockBasedTable::Open(rocksdb::ImmutableCFOptions const&, rocksdb::EnvOptions const&, rocksdb::BlockBasedTableOptions const&, rocksdb::InternalKeyComparator const&, std::unique_ptr<rocksdb::RandomAccessFileReader, std::default_delete<rocksdb::RandomAccessFileReader> >&&, unsigned long, std::unique_ptr<rocksdb::TableReader, std::default_delete<rocksdb::TableReader> >*, bool, bool, int) (ioptions=..., env_options=..., table_options=..., internal_comparator=..., file=<unknown type in /home/psergey/dev-git/10.2-mariarocks/mysql-test/var/plugins/ha_rocksdb.so, CU 0xebb7c8, DIE 0xf0c575>, file_size=1017, table_reader=0x7f47cb3f30e0, prefetch_index_and_filter_in_cache=true, skip_filters=false, level=0) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/table/block_based_table_reader.cc:693
#14 0x00007f47fed5f43e in rocksdb::BlockBasedTableFactory::NewTableReader(rocksdb::TableReaderOptions const&, std::unique_ptr<rocksdb::RandomAccessFileReader, std::default_delete<rocksdb::RandomAccessFileReader> >&&, unsigned long, std::unique_ptr<rocksdb::TableReader, std::default_delete<rocksdb::TableReader> >*, bool) const (this=0x7f48006b1160, table_reader_options=..., file=<unknown type in /home/psergey/dev-git/10.2-mariarocks/mysql-test/var/plugins/ha_rocksdb.so, CU 0xe90a21, DIE 0xeb8d6c>, file_size=1017, table_reader=0x7f47cb3f30e0, prefetch_index_and_filter_in_cache=true) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/table/block_based_table_factory.cc:59
#15 0x00007f47fed0ab75 in rocksdb::TableCache::GetTableReader (this=0x7f48007834a0, env_options=..., internal_comparator=..., fd=..., sequential_mode=false, readahead=0, record_read_stats=true, file_read_hist=0x7f4800797000, table_reader=0x7f47cb3f30e0, skip_filters=false, level=0, prefetch_index_and_filter_in_cache=true) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/db/table_cache.cc:111
#16 0x00007f47fed0af5f in rocksdb::TableCache::FindTable (this=0x7f48007834a0, env_options=..., internal_comparator=..., fd=..., handle=0x7f47cb3f31f0, no_io=false, record_read_stats=true, file_read_hist=0x7f4800797000, skip_filters=false, level=0, prefetch_index_and_filter_in_cache=true) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/db/table_cache.cc:148
#17 0x00007f47fed0b3b7 in rocksdb::TableCache::NewIterator (this=0x7f48007834a0, options=..., env_options=..., icomparator=..., fd=..., range_del_agg=0x0, table_reader_ptr=0x0, file_read_hist=0x7f4800797000, for_compaction=false, arena=0x0, skip_filters=false, level=0) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/db/table_cache.cc:208
#18 0x00007f47fee93227 in rocksdb::BuildTable (dbname="./.rocksdb", env=0x7f47ff3fde80 <rocksdb::Env::Default()::default_env>, ioptions=..., mutable_cf_options=..., env_options=..., table_cache=0x7f48007834a0, iter=0x7f47cb3f3c68, range_del_iter=std::unique_ptr<rocksdb::InternalIterator> containing 0x0, meta=0x7f47cb3f4a98, internal_comparator=..., int_tbl_prop_collector_factories=0x7f4800658940, column_family_id=0, column_family_name="default", snapshots=std::vector of length 0, capacity 0, earliest_write_conflict_snapshot=72057594037927935, compression=<incomplete type>, compression_opts=..., paranoid_file_checks=false, internal_stats=0x7f480504d580, reason=rocksdb::TableFileCreationReason::kFlush, event_logger=0x7f4800b6ced8, job_id=3, io_priority=rocksdb::Env::IO_HIGH, table_properties=0x7f47cb3f4978, level=0) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/db/builder.cc:188
#19 0x00007f47fecc2187 in rocksdb::FlushJob::WriteLevel0Table (this=0x7f47cb3f48e0) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/db/flush_job.cc:295
#20 0x00007f47fecc1261 in rocksdb::FlushJob::Run (this=0x7f47cb3f48e0, file_meta=0x7f47cb3f4740) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/db/flush_job.cc:182
#21 0x00007f47fec32c18 in rocksdb::DBImpl::FlushMemTableToOutputFile (this=0x7f4800b6c400, cfd=0x7f4800658900, mutable_cf_options=..., made_progress=0x7f47cb3f5137, job_context=0x7f47cb3f5190, log_buffer=0x7f47cb3f5390) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/db/db_impl.cc:1934
#22 0x00007f47fec3a10b in rocksdb::DBImpl::BackgroundFlush (this=0x7f4800b6c400, made_progress=0x7f47cb3f5137, job_context=0x7f47cb3f5190, log_buffer=0x7f47cb3f5390) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/db/db_impl.cc:3245
#23 0x00007f47fec3a37c in rocksdb::DBImpl::BackgroundCallFlush (this=0x7f4800b6c400) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/db/db_impl.cc:3270
#24 0x00007f47fec39775 in rocksdb::DBImpl::BGWorkFlush (db=0x7f4800b6c400) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/db/db_impl.cc:3136
#25 0x00007f47fef1f6b6 in rocksdb::ThreadPoolImpl::<lambda()>::operator()(void) const (__closure=0x7f47ccc57120) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/util/threadpool_imp.cc:414
#26 0x00007f47fef1fb51 in std::_Function_handler<void(), rocksdb::ThreadPoolImpl::Schedule(void (*)(void*), void*, void*, void (*)(void*))::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/4.9/functional:2039
#27 0x00007f47fee4b3b4 in std::function<void ()>::operator()() const (this=0x7f47cb3f5d90) at /usr/include/c++/4.9/functional:2439
#28 0x00007f47fef1eadb in rocksdb::ThreadPoolImpl::Impl::BGThread (this=0x7f4805079640, thread_id=0) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/util/threadpool_imp.cc:236
#29 0x00007f47fef1ebc2 in rocksdb::ThreadPoolImpl::Impl::BGThreadWrapper (arg=0x7f47ccc57130) at /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/rocksdb/util/threadpool_imp.cc:260
#30 0x00007f47fef255a6 in std::_Bind_simple<void* (*(rocksdb::BGThreadMetadata*))(void*)>::_M_invoke<0ul>(std::_Index_tuple<0ul>) (this=0x7f47ccdc31e8) at /usr/include/c++/4.9/functional:1700
#31 0x00007f47fef2549f in std::_Bind_simple<void* (*(rocksdb::BGThreadMetadata*))(void*)>::operator()() (this=0x7f47ccdc31e8) at /usr/include/c++/4.9/functional:1688
#32 0x00007f47fef2541c in std::thread::_Impl<std::_Bind_simple<void* (*(rocksdb::BGThreadMetadata*))(void*)> >::_M_run() (this=0x7f47ccdc31d0) at /usr/include/c++/4.9/thread:115
#33 0x00007f48064c1e30 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#34 0x00007f480671d6aa in start_thread (arg=0x7f47cb3f6700) at pthread_create.c:333
#35 0x00007f4805c25eed in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109



 Comments   
Comment by Sergei Petrunia [ 2017-03-23 ]

Fails on debug build, psergey-desktop, Ubuntu 15.04
Doesn't fail on aws ec2, Ubuntu 16.04.1 LTS, debug build.

Comment by Sergei Petrunia [ 2017-03-23 ]

Seems like an upstream issue: https://github.com/facebook/mysql-5.6/issues/579

Generated at Thu Feb 08 07:56:57 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.