Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-13852

MyRocks/Windows: SyncWAL() is not supported for this implementation of WAL file

Details

    Description

      After the merge of recent changes from the upstream, the following error appeared on Windows:

      http://buildbot.askmonty.org/buildbot/builders/winx64-packages/builds/4936/steps/test/logs/stdio

      2017-09-19 20:35:52 3272 [ERROR] RocksDB: Failed to write to WAL (dictionary), Status Code: 3, Status: Not implemented: SyncWAL() is not supported for this implementation of WAL file
      

      This didn't happen before. Need to investigate.

      • main 10.2 branch has MyRocks without this error
      • bb-10.2-mariarocks-merge branch has newer MyRocks where the error occurs.

      Attachments

        Issue Links

          Activity

            varun, can you investigate please? Elena and Wlad know how to get to Windows boxes.

            psergei Sergei Petrunia added a comment - varun , can you investigate please? Elena and Wlad know how to get to Windows boxes.

            See also MDEV-12427 . Does current MariaDB-10.2 ever call SyncWAL ? (I think yes). What happens when it is called on Windows?

            It looks rather unlikely that write_sync.test was the only test that would suffer from an absent SyncWAL implementation although I would not rule this out. Please check.

            psergei Sergei Petrunia added a comment - See also MDEV-12427 . Does current MariaDB-10.2 ever call SyncWAL ? (I think yes). What happens when it is called on Windows? It looks rather unlikely that write_sync.test was the only test that would suffer from an absent SyncWAL implementation although I would not rule this out. Please check.

            The error comes from here

            >	ha_rocksdb.dll!rocksdb::DBImpl::SyncWAL() Line 661	C++	Symbols loaded.
             	ha_rocksdb.dll!rocksdb::DBImpl::FlushWAL(bool sync) Line 638	C++	Symbols loaded.
             	ha_rocksdb.dll!rocksdb::DBImpl::WriteImpl(const rocksdb::WriteOptions & write_options, rocksdb::WriteBatch * my_batch, rocksdb::WriteCallback * callback, unsigned __int64 * log_used, unsigned __int64 log_ref, bool disable_memtable, unsigned __int64 * seq_used) Line 304	C++	Symbols loaded.
             	ha_rocksdb.dll!rocksdb::DBImpl::Write(const rocksdb::WriteOptions & write_options, rocksdb::WriteBatch * my_batch) Line 49	C++	Symbols loaded.
             	ha_rocksdb.dll!myrocks::Rdb_dict_manager::commit(rocksdb::WriteBatch * const batch, const bool & sync) Line 4511	C++	Symbols loaded.
             	ha_rocksdb.dll!myrocks::Rdb_dict_manager::rollback_ongoing_index_creation() Line 4966	C++	Symbols loaded.
             	ha_rocksdb.dll!myrocks::Rdb_dict_manager::init(rocksdb::DB * const rdb_dict, myrocks::Rdb_cf_manager * const cf_manager) Line 4465	C++	Symbols loaded.
             	ha_rocksdb.dll!myrocks::rocksdb_init_func(void * const p) Line 4123	C++	Symbols loaded.
             	mysqld.exe!ha_initialize_handlerton(st_plugin_int * plugin) Line 520	C++	Symbols loaded.
             	mysqld.exe!plugin_initialize(st_mem_root * tmp_root, st_plugin_int * plugin, int * argc, char * * argv, bool options_only) Line 1413	C++	Symbols loaded.
             	mysqld.exe!plugin_init(int * argc, char * * argv, int flags) Line 1694	C++	Symbols loaded.
            

            psergei Sergei Petrunia added a comment - The error comes from here > ha_rocksdb.dll!rocksdb::DBImpl::SyncWAL() Line 661 C++ Symbols loaded. ha_rocksdb.dll!rocksdb::DBImpl::FlushWAL(bool sync) Line 638 C++ Symbols loaded. ha_rocksdb.dll!rocksdb::DBImpl::WriteImpl(const rocksdb::WriteOptions & write_options, rocksdb::WriteBatch * my_batch, rocksdb::WriteCallback * callback, unsigned __int64 * log_used, unsigned __int64 log_ref, bool disable_memtable, unsigned __int64 * seq_used) Line 304 C++ Symbols loaded. ha_rocksdb.dll!rocksdb::DBImpl::Write(const rocksdb::WriteOptions & write_options, rocksdb::WriteBatch * my_batch) Line 49 C++ Symbols loaded. ha_rocksdb.dll!myrocks::Rdb_dict_manager::commit(rocksdb::WriteBatch * const batch, const bool & sync) Line 4511 C++ Symbols loaded. ha_rocksdb.dll!myrocks::Rdb_dict_manager::rollback_ongoing_index_creation() Line 4966 C++ Symbols loaded. ha_rocksdb.dll!myrocks::Rdb_dict_manager::init(rocksdb::DB * const rdb_dict, myrocks::Rdb_cf_manager * const cf_manager) Line 4465 C++ Symbols loaded. ha_rocksdb.dll!myrocks::rocksdb_init_func(void * const p) Line 4123 C++ Symbols loaded. mysqld.exe!ha_initialize_handlerton(st_plugin_int * plugin) Line 520 C++ Symbols loaded. mysqld.exe!plugin_initialize(st_mem_root * tmp_root, st_plugin_int * plugin, int * argc, char * * argv, bool options_only) Line 1413 C++ Symbols loaded. mysqld.exe!plugin_init(int * argc, char * * argv, int flags) Line 1694 C++ Symbols loaded.
            psergei Sergei Petrunia added a comment - - edited

            The new code sets DBOptions::concurrent_prepare to TRUE (see e.g. rdb_init_rocksdb_db_options).

            Changing it to FALSE removes the assert there, but then I get the same "SyncWAL() is not supported for this implementation of WAL file" error here:

            >	ha_rocksdb.dll!rocksdb::DBImpl::SyncWAL() Line 662	C++	Symbols loaded.
             	ha_rocksdb.dll!rocksdb::DBImpl::FlushWAL(bool sync) Line 638	C++	Symbols loaded.
             	ha_rocksdb.dll!rocksdb::StackableDB::FlushWAL(bool sync) Line 273	C++	Symbols loaded.
             	ha_rocksdb.dll!myrocks::Rdb_background_thread::run() Line 11730	C++	Symbols loaded.
             	ha_rocksdb.dll!myrocks::Rdb_thread::thread_func(void * const thread_ptr) Line 35	C++	Symbols loaded.
            

            psergei Sergei Petrunia added a comment - - edited The new code sets DBOptions::concurrent_prepare to TRUE (see e.g. rdb_init_rocksdb_db_options). Changing it to FALSE removes the assert there, but then I get the same "SyncWAL() is not supported for this implementation of WAL file" error here: > ha_rocksdb.dll!rocksdb::DBImpl::SyncWAL() Line 662 C++ Symbols loaded. ha_rocksdb.dll!rocksdb::DBImpl::FlushWAL(bool sync) Line 638 C++ Symbols loaded. ha_rocksdb.dll!rocksdb::StackableDB::FlushWAL(bool sync) Line 273 C++ Symbols loaded. ha_rocksdb.dll!myrocks::Rdb_background_thread::run() Line 11730 C++ Symbols loaded. ha_rocksdb.dll!myrocks::Rdb_thread::thread_func(void * const thread_ptr) Line 35 C++ Symbols loaded.

            It doesn't seem to be possible to run on Windows currently.

            void Rdb_background_thread::run() at some point calls

                  const rocksdb::Status s = rdb->FlushWAL(true);
            

            which calls DBImpl::FlushWAL(sync=true)
            which calls DBImpl::SyncWAL()

            where it makes this check

              !it->writer->file()->writable_file()->IsSyncThreadSafe()
            

            On Linux, PosixWritableFile::IsSyncThreadSafe() returns true.

            For Windows, I was not able to find any implementation in port/win that would have IsSyncThreadSafe() return true.

            psergei Sergei Petrunia added a comment - It doesn't seem to be possible to run on Windows currently. void Rdb_background_thread::run() at some point calls const rocksdb::Status s = rdb->FlushWAL(true); which calls DBImpl::FlushWAL(sync=true) which calls DBImpl::SyncWAL() where it makes this check !it->writer->file()->writable_file()->IsSyncThreadSafe() On Linux, PosixWritableFile::IsSyncThreadSafe() returns true. For Windows, I was not able to find any implementation in port/win that would have IsSyncThreadSafe() return true.
            wlad Vladislav Vaintroub added a comment - - edited

            psergey, are you going to file this bug in rocksdb bug tracker? If not , I can do that.

            I implemented a hackish workaround for now, by using patched header file for win_io.h where IsSyncThreadSafe returns true, in https://github.com/mariadb/server/commit/689168be1240cf61d0ea17529e79660e98d15b46

            wlad Vladislav Vaintroub added a comment - - edited psergey , are you going to file this bug in rocksdb bug tracker? If not , I can do that. I implemented a hackish workaround for now, by using patched header file for win_io.h where IsSyncThreadSafe returns true, in https://github.com/mariadb/server/commit/689168be1240cf61d0ea17529e79660e98d15b46

            Should be fixed in the recent merge

            psergei Sergei Petrunia added a comment - Should be fixed in the recent merge

            Closing as upstream has fixed this and we merged the fix. (i've also removed our workaround)

            psergei Sergei Petrunia added a comment - Closing as upstream has fixed this and we merged the fix. (i've also removed our workaround)

            People

              psergei Sergei Petrunia
              psergei Sergei Petrunia
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.