Details
-
Task
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
None
Description
MyRocks has a scripts/myrocks_hotbackup script, and rocksdb_hotbackup testsuite to test it.
At the moment myrocks_hotbackup is not included in MariaDB (it was outside of storage/rocksdb/ so we missed it).
I also experienced difficulties when running rocksdb_hotbackup tests: https://github.com/facebook/mysql-5.6/issues/577 . Some of the tests will not run on [current] MariaDB (as they require "slocket" feature or MySQL-style gtid or something else), but it would be nice to have at least one test passing.
Dependencies:
- MySQL-python
- Either tar or wdt tool (https://github.com/facebook/wdt) must be in the PATH.
Attachments
Issue Links
- is part of
-
-
- Closed
-
- relates to
-
-
- Closed
-
Activity
Take aways from discussion with cvicentiu:
Python version
- Does myrocks_hotbackup need python2 or python3? Not clear.
Packaging options
- make a new package for myrocks_hotbackup, with proper dependencies.
- put myrocks_hotbackup into the same package as myrocks itself. Add dependencies as "recommended". it is acceptable some things in a package do not work when recommended dependencies are not met.
About MySQL dependency
- We (MariaDB) do not maintain a python connector.
- There are certain issues with the MySQLDB connector (didn't take note what they are exactly
). This connector is based on the C client library.
- There is also a pure python implementation, https://github.com/PyMySQL/PyMySQL (package python-pymysql or python3-pymysql)
- myrocks_hotbackup only needs two commands: select @@datadir and SET GLOBAL rocksdb_create_checkpoint=param. If dependency on mysql connector is too
hard, we could just invoke /usr/bin/mysql manually.
myrocks_hotbackup only works with python 2. It makes use of the commands module which is only available in python 2
Pushed cvicentiu's fix into main 10.2, and I see myrocks_hotbackup in the debian package:
mariadb-plugin-rocksdb_10.2.13+maria-artful_amd64.deb:
|
new debian package, version 2.0.
|
size 2236152 bytes: control archive= 836 bytes.
|
38 bytes, 1 lines conffiles
|
690 bytes, 15 lines control
|
397 bytes, 6 lines md5sums
|
Package: mariadb-plugin-rocksdb
|
Source: mariadb-10.2
|
Version: 10.2.13+maria~artful
|
Architecture: amd64
|
Maintainer: MariaDB Developers <maria-developers@lists.launchpad.net>
|
Installed-Size: 13223
|
Depends: mariadb-server-10.2 (= 10.2.13+maria~artful), libc6 (>= 2.17), liblz4-1 (>= 0.0~r130), libsnappy1v5, libstdc++6 (>= 7), zlib1g (>= 1:1.1.4)
|
Recommends: python-mysqldb
|
Section: database
|
Priority: optional
|
Homepage: http://mariadb.org/
|
Description: RocksDB storage engine for MariaDB
|
The RocksDB storage engine is a high performance storage engine, aimed
|
at maximising storage efficiency while maintaining InnoDB-like performance.
|
This package contains the RocksDB plugin for MariaDB.
|
|
|
*** Contents:
|
drwxr-xr-x root/root 0 2018-02-09 13:51 ./
|
drwxr-xr-x root/root 0 2018-02-09 13:51 ./etc/
|
drwxr-xr-x root/root 0 2018-02-09 13:51 ./etc/mysql/
|
drwxr-xr-x root/root 0 2018-02-09 13:51 ./etc/mysql/mariadb.conf.d/
|
-rw-r--r-- root/root 40 2018-02-09 13:51 ./etc/mysql/mariadb.conf.d/rocksdb.cnf
|
drwxr-xr-x root/root 0 2018-02-09 13:51 ./usr/
|
drwxr-xr-x root/root 0 2018-02-09 13:51 ./usr/bin/
|
-rwxr-xr-x root/root 24704 2018-02-09 13:47 ./usr/bin/myrocks_hotbackup
|
...
|
So, we include myrocks_backup script into at least some packages, but not a single test from rocksdb_hotbackup test suite works. This is quite bad. I suggest either to deprecate and remove the script if it's not supported, or add tests for it.
Current rocksdb_hotbackup script will not backup ibdata and ib_logfile, but will backup .ibd files.
Basically, it works for MyRocks-only instances, but it's difficult to guarantee anything if other storage engines are used.