https://github.com/MariaDB/server/commit/a7bb9e8fdbdd5a3a01910117ef54c0fe09374de2
has been fixed very very recently. Also merged into 10.2 however it was after the release.
Are you sure about that patch?
I made it worked by this patch, where the first part is, where it needs to be patched, but the second part (commented out), where it needs to be untouched (!!).
And it seems to me, that you swapped those 2 places by mistake.
--- mariadb-10.2.5/storage/innobase/fil/fil0crypt.cc 2017-04-04 21:12:06.000000000 +0200 |
+++ mariadb-10.2.5/storage/innobase/fil/fil0crypt.cc_patched 2017-04-07 14:12:53.208687050 +0200 |
@@ -1767,7 +1767,7 @@ fil_crypt_rotate_page(
|
return; |
}
|
|
- if (space == TRX_SYS_SPACE && offset == TRX_SYS_PAGE_NO) { |
+ if (space_id == TRX_SYS_SPACE && offset == TRX_SYS_PAGE_NO) { |
/* don't encrypt this as it contains address to dblwr buffer */
|
return; |
}
|
#@@ -1937,7 +1937,7 @@ fil_crypt_rotate_pages(
|
# * real pages, they will be updated anyway when the
|
# * real page is updated
|
# */
|
#- if (space == TRX_SYS_SPACE &&
|
#+ if (space_id == TRX_SYS_SPACE &&
|
# buf_dblwr_page_inside(state->offset)) {
|
# continue;
|
# } |
Another issue emerged, related to RocksDB
/builddir/build/BUILD/mariadb-10.2.5/storage/rocksdb/rdb_datadic.cc: In function 'int myrocks::rdb_unpack_floating_point(uchar*, myrocks::Rdb_string_reader*, const size_t&, const int&, const uchar*, const uchar*, void (*)(uchar*, const uchar*))': |
/builddir/build/BUILD/mariadb-10.2.5/storage/rocksdb/rdb_datadic.cc:1306:3: error: non-constant condition for static assertion |
static_assert(swap_func == nullptr, "Assuming that no swapping is needed."); |
^~~~~~~~~~~~~
|
/builddir/build/BUILD/mariadb-10.2.5/storage/rocksdb/rdb_datadic.cc:1306:3: error: 'swap_func' is not a constant expression |
/builddir/build/BUILD/mariadb-10.2.5/storage/rocksdb/rdb_datadic.cc: In member function 'bool myrocks::Rdb_ddl_manager::init(myrocks::Rdb_dict_manager*, myrocks::Rdb_cf_manager*, const uint32_t&)': |
/builddir/build/BUILD/mariadb-10.2.5/storage/rocksdb/rdb_datadic.cc:3039:50: warning: '*' in boolean context, suggest '&&' instead [-Wint-in-bool-context] |
if (real_val_size % Rdb_key_def::PACKED_SIZE * 2) { |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
|
Can you please re-open this bug report and leave it open until I'll go successfuly through all platforms?
Reassigning to the RocksDB maintainer.
mschorm, the InnoDB patch that danblack contributed was against 10.1 and merged by me to 10.2 shortly after 10.2.5 was released. The differences between 10.1 and 10.2 could explain why the 10.1 patch did not apply cleanly to 10.2. By the way, storage/xtradb is not compiled in MariaDB Server 10.2.
The patch in comment 2 I wrote just today for the 10.2.5 version only.
Fedora 25, GCC6, i686, debug build:
In file included from /builddir/build/BUILD/mariadb-10.2.5/include/my_global.h:1051:0, |
from /builddir/build/BUILD/mariadb-10.2.5/storage/innobase/include/univ.i:83, |
from /builddir/build/BUILD/mariadb-10.2.5/storage/innobase/handler/ha_innodb.cc:38: |
/builddir/build/BUILD/mariadb-10.2.5/storage/innobase/handler/ha_innodb.cc: In function 'int wsrep_innobase_kill_one_trx(void*, const trx_t*, trx_t*, ulint)': |
/builddir/build/BUILD/mariadb-10.2.5/storage/innobase/handler/ha_innodb.cc:20332:75: error: 'debug_sync_set_action' was not declared in this scope |
STRING_WITH_LEN(act)));
|
^
|
/builddir/build/BUILD/mariadb-10.2.5/include/my_dbug.h:88:50: note: in definition of macro 'DBUG_EXECUTE_IF' |
do {if (_db_keyword_(0, (keyword), 1)) { a1 }} while(0) |
^~
|
/builddir/build/BUILD/mariadb-10.2.5/storage/innobase/handler/ha_innodb.cc:20331:20: note: in expansion of macro 'DBUG_ASSERT' |
DBUG_ASSERT(!debug_sync_set_action(bf_thd,
|
^
|
storage/innobase/CMakeFiles/innobase.dir/build.make:1313: recipe for target 'storage/innobase/CMakeFiles/innobase.dir/handler/ha_innodb.cc.o' failed |
Fedora 25, GCC6, x86_64, debug build:
/builddir/build/BUILD/mariadb-10.2.5/storage/connect/ha_connect.cc: In member function 'CONDFIL* ha_connect::CheckCond(PGLOBAL, PCFIL, const Item*)': |
/builddir/build/BUILD/mariadb-10.2.5/storage/connect/ha_connect.cc:2744:10: warning: 'ph2' may be used uninitialized in this function [-Wmaybe-uninitialized] |
ph1= ph2 + strlen(ph2);
|
~~~^~~~~~~~~~~~~~~~~~~
|
/builddir/build/BUILD/mariadb-10.2.5/storage/connect/ha_connect.cc:2768:10: warning: 'ph0' may be used uninitialized in this function [-Wmaybe-uninitialized] |
*ph0 = 0;
|
~~~~~^~~
|
In file included from /builddir/build/BUILD/mariadb-10.2.5/include/my_global.h:1051:0, |
from /builddir/build/BUILD/mariadb-10.2.5/sql/sql_plugin.h:28, |
from /builddir/build/BUILD/mariadb-10.2.5/sql/item_func.cc:28: |
/builddir/build/BUILD/mariadb-10.2.5/sql/item_func.cc: In member function 'virtual longlong Item_func_sleep::val_int()': |
/builddir/build/BUILD/mariadb-10.2.5/sql/item_func.cc:4567:72: error: 'debug_sync_set_action' was not declared in this scope |
(thd, STRING_WITH_LEN("dispatch_command_end SIGNAL query_done")); |
^
|
/builddir/build/BUILD/mariadb-10.2.5/include/my_dbug.h:88:50: note: in definition of macro 'DBUG_EXECUTE_IF' |
do {if (_db_keyword_(0, (keyword), 1)) { a1 }} while(0) |
^~
|
libmysqld/CMakeFiles/sql_embedded.dir/build.make:761: recipe for target 'libmysqld/CMakeFiles/sql_embedded.dir/__/sql/item_func.cc.o' failed |
and
In file included from /builddir/build/BUILD/mariadb-10.2.5/include/my_global.h:1051:0, |
from /builddir/build/BUILD/mariadb-10.2.5/storage/innobase/include/univ.i:83, |
from /builddir/build/BUILD/mariadb-10.2.5/storage/innobase/handler/ha_innodb.cc:38: |
/builddir/build/BUILD/mariadb-10.2.5/storage/innobase/handler/ha_innodb.cc: In function 'int wsrep_innobase_kill_one_trx(void*, const trx_t*, trx_t*, ulint)': |
/builddir/build/BUILD/mariadb-10.2.5/storage/innobase/handler/ha_innodb.cc:20332:75: error: 'debug_sync_set_action' was not declared in this scope |
STRING_WITH_LEN(act)));
|
^
|
/builddir/build/BUILD/mariadb-10.2.5/include/my_dbug.h:88:50: note: in definition of macro 'DBUG_EXECUTE_IF' |
do {if (_db_keyword_(0, (keyword), 1)) { a1 }} while(0) |
^~
|
/builddir/build/BUILD/mariadb-10.2.5/storage/innobase/handler/ha_innodb.cc:20331:20: note: in expansion of macro 'DBUG_ASSERT' |
DBUG_ASSERT(!debug_sync_set_action(bf_thd,
|
^
|
Fedora 27, GCC7, all architectures
/builddir/build/BUILD/mariadb-10.2.5/vio/viosslfactories.c: In function 'get_dh2048': |
/builddir/build/BUILD/mariadb-10.2.5/vio/viosslfactories.c:62:5: error: dereferencing pointer to incomplete type 'DH {aka struct dh_st}' |
dh->p=BN_bin2bn(dh2048_p,sizeof(dh2048_p),NULL);
|
^~
|
the last viossl compile error above is due to openssl-1.1 and has a fix in progress in MDEV-10332
For the avoidance of doubt about the correctness of first patch: fil_crypt_rotate_pages ulint space = state->space->id. In fil_crypt_rotate_page, space is fil_space_t*space = state->space. Merged patch is correct.
Daniel, thanks for the check of the patch.
I'll leave the OpenSSL issues out of this bug.
---------
I did some analysis, which lead me to small amount of unstable tests in addition to yours:
# Those fails as expected
|
main.ssl_7937, main.ssl_crl_clients, main.ssl_cert_verify
|
# Not expected, long term problem
|
main.userstat, main.multi_update
|
# Not expected, long term problem
|
perfschema.nesting, perfschema.socket_summary_by_event_name_func, perfschema.socket_summary_by_instance_func
|
|
|
# Newly failing
|
main.non_blocking_api, main.mysql_client_test_nonblock
|
# Newly failing
|
plugins.show_all_plugins
|
# Newly failing - rocksdb
|
rocksdb.shutdown |
rocksdb.drop_table
|
rocksdb.type_varchar
|
rocksdb.use_direct_reads_writes
|
rocksdb.bloomfilter
|
|
|
# Only on ARM, Long term problem
|
encryption.innodb-encryption-alter, encryption.innodb_encryption_discard_import
|
# Only on ARM, newly failing - rocksdb
|
rocksdb.commit_in_the_middle_ddl
|
rocksdb.concurrent_alter
|
rocksdb.type_varchar
|
rocksdb.add_index_inplace
|
rocksdb.drop_index_inplace
|
rocksdb_sys_vars.rocksdb_merge_buf_size_basic
|
rocksdb_sys_vars.rocksdb_merge_combine_read_size_basic
|
---------
With the list above, both F25 and F27 on all architectures build fine.
Except PPC64 and PPC64p7, with following error:
(UPDATE: fails both with GCC6 and GCC7, seems like only big-endian issue)
/builddir/build/BUILD/mariadb-10.2.5/storage/rocksdb/rdb_datadic.cc: In function 'int myrocks::rdb_unpack_floating_point(uchar*, myrocks::Rdb_string_reader*, const size_t&, const int&, const uchar*, const uchar*, void (*)(uchar*, const uchar*))': |
/builddir/build/BUILD/mariadb-10.2.5/storage/rocksdb/rdb_datadic.cc:1306:3: error: non-constant condition for static assertion |
static_assert(swap_func == nullptr, "Assuming that no swapping is needed."); |
^~~~~~~~~~~~~
|
/builddir/build/BUILD/mariadb-10.2.5/storage/rocksdb/rdb_datadic.cc:1306:3: error: 'swap_func' is not a constant expression |
/builddir/build/BUILD/mariadb-10.2.5/storage/rocksdb/rdb_datadic.cc: In member function 'bool myrocks::Rdb_ddl_manager::init(myrocks::Rdb_dict_manager*, myrocks::Rdb_cf_manager*, const uint32_t&)': |
/builddir/build/BUILD/mariadb-10.2.5/storage/rocksdb/rdb_datadic.cc:3039:50: warning: '*' in boolean context, suggest '&&' instead [-Wint-in-bool-context] |
if (real_val_size % Rdb_key_def::PACKED_SIZE * 2) { |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
|
---------
I'm far from finishing all the tests, but this is the current state so far.
- The big issue here is the PPC one.
- Aside from that, I would like to see those rocksdb test pasing, because current state means, something's wrong. Ask for any log or help you need, however all debug builds have compile time errors and cannot be build.
One build log, that containts the failing tests and could help is here (x86_64, GCC7), another one here (ARMv7hl, GCC7)(still not finished, as I write this, can take couple of hours).
PPC error: if you can remove the line of the error - the code is predictable enough that it won't assert there - a proper fix will probably be a debug_assert of some form.
PPC warnings: could take my upstream patch
rocks test some recently disabled: https://github.com/MariaDB/server/commit/8c9cd26c06ac3ccf667af17249478b6afae6f3aa . I agree that a higher passing rate is desirable.
ssl tests - I suspect there are openssl-1.1 having different defaults/errors/strictness
encryption.innodb_encryption_discard_import -> MDEV-11218
show_all_plugins - is it like MDEV-8664?
perfschema.socket* - MDEV-10622
perfschema.nesting - see MDEV-8446
PPC error:
I made a patch that comments out that particular line on PPC64 and PPC64p7 adn it compiles fine.
However, it has a bit different problem now,
Normally, around 10 tests is failing. But 70 ? Something must be wrong here.
Build log is here.
Completed: Failed 70/2761 tests, 97.46% were successful. |
Failing test(s): rocksdb.rocksdb_parts rocksdb.add_index_inplace rocksdb.partition rocksdb.rocksdb_cf_per_partition rocksdb.bloomfilter_skip rocksdb.rocksdb_qcache rocksdb.insert_optimized_config rocksdb.compact_deletes rocksdb.bloomfilter3 rocksdb.rocksdb_range rocksdb.rocksdb_cf_reverse rocksdb.type_float_indexes rocksdb.type_int_indexes rocksdb.update_ignore rocksdb.rocksdb_icp rocksdb.rocksdb_icp_rev rocksdb.bloomfilter4 rocksdb.singledelete rocksdb.bloomfilter2 rocksdb.optimize_table rocksdb.issue100_delete rocksdb.rocksdb_deadlock_detect_rc rocksdb.allow_no_primary_key_with_sk rocksdb.apply_changes_iter rocksdb.autoinc_secondary rocksdb.autoinc_vars rocksdb.autoinc_vars_thread_2 rocksdb.check_table rocksdb.checksum_table rocksdb.checksum_table_live rocksdb.col_opt_default rocksdb.col_opt_not_null rocksdb.col_opt_null rocksdb.col_opt_unsigned rocksdb.col_opt_zerofill rocksdb.cons_snapshot_read_committed rocksdb.cons_snapshot_repeatable_read rocksdb.create_table rocksdb.deadlock rocksdb.delete rocksdb.delete_ignore rocksdb.delete_quick rocksdb.delete_with_keys rocksdb.drop_index_inplace rocksdb.dup_key_update rocksdb.duplicate_table rocksdb.handler_basic rocksdb.hermitage rocksdb.insert_with_keys rocksdb.issue100 rocksdb.issue111 rocksdb.level_read_committed rocksdb.level_repeatable_read rocksdb.lock_info rocksdb.locking_issues rocksdb.perf_context rocksdb.rocksdb_deadlock_detect_rr rocksdb.rocksdb_row_stats rocksdb.select rocksdb.tbl_opt_ai rocksdb.transaction rocksdb.truncate_table rocksdb.unique_sec_rev_cf rocksdb.update rocksdb.update_multi rocksdb.update_with_keys rocksdb.rollback_savepoint rocksdb_sys_vars.rocksdb_flush_memtable_on_analyze_basic unit.conc_features-10_2 unit.conc_misc |
|
I've added pull requests here and for facebook upstream to fix the compile error. Yikes - totally agree with the test failures - seems to be a lot of big endian mistakes in the rocksdb code. I'll see if I can get an intern to correct the code related to these later in the week.
On the packaging side:
- Where is the source for your build (spec and patches)?
- Is there a reason cpack -DRPM isn't suitable? I suspect there are lots of macros used
- mysql-wait-ready and mysql-wait-stop aren't needed with systemd Type=notify
- Is there selinux differences between what you have and ./support-files/policy/selinux?
- logrotate - I made some changes in
MDEV-11610. I see logrotate has had a painful history in Fedora/Redhat. Is creating a logrotate user with RELOAD privs and pointing the mysqladmin to its credentials acceptable. With systemd and no error-log settings in [mysqld] the error will go to the journal. I'll write an enhancement to support-files/mysql-log-rotate so it is an upstream maintained way.
Hello again, sorry for a delay.
- I moved my current work to GitHub
- We ship updated SELinux rules, but only for Galera. Can be found in 'mariadb-server-galera.te' file
- About Log rotation:
I investigated the current state a month back and I was told, everything should work fine for quite a time in Fedora. I also looked at another projects and their way of dealing with it. Conclusion is, that I plan to enable log rotation for 10.2 version.
Thanks for exposing this mschorm. cnf_examples.patch -> MDEV-12346 . Looking the other patches I think many of these should be upstream here. Are you willing to submit pull requests for these?
Logrotate: started along this lines: https://lists.launchpad.net/maria-discuss/msg04453.html though I'm tempted to got along the lines of a USR1 signal. /var/log for log files is hard as the server run as the mysql user. The mysql log itself is going to the journal. Perhaps other log file types should go there too. Is the systemd ExecStopPost aimed at preventing timeout on shutdown? Happy to continue discussion of reasons for any differences but perhaps on a different JIRA issue.
I'm working on those things again.
- I prepared myself a patch (for fedora) and sent it for review - it should remove those "mysql-wait-*" scripts as you suggested
- the cpack -DRPM is beeing used in our SPECfile. Yeah, there's one hell of a macro, but it translates to OS+version ("fedora27" for example)
- I was talking here about the GCC issues previously (and in other tickets too). They mostly seems to be solved but one. Currently, I'm talking about MariaDB 10.2.6 on Fedora Rawhide (27) If I make a debug build, all architectures pass, but x86_64, with following error:
[ 15%] Building C object mysys/CMakeFiles/mysys.dir/list.c.o
cd /builddir/build/BUILD/mariadb-10.2.6/mysys && /usr/bin/cc -DHAVE_CONFIG_H -DHAVE_SYSTEMD -D_FILE_OFFSET_BITS=64 -I/builddir/build/BUILD/mariadb-10.2.6/include -I/builddir/build/BUILD/mariadb-10.2.6/mysys -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -fPIC -pie -fPIC -Wl,-z,relro,-z,now -fstack-protector --param=ssp-buffer-size=4 -DWITH_INNODB_DISALLOW_WRITES -fPIC -O -g -static-libgcc -fno-omit-frame-pointer -fno-strict-aliasing -Wno-uninitialized -DENABLED_DEBUG_SYNC -ggdb3 -DSAFE_MUTEX -DSAFEMALLOC -Wall -Wextra -Wformat-security -Wno-init-self -Wvla -Wwrite-strings -Wdeclaration-after-statement -o CMakeFiles/mysys.dir/list.c.o -c /builddir/build/BUILD/mariadb-10.2.6/mysys/list.c
/builddir/build/BUILD/mariadb-10.2.6/storage/tokudb/PerconaFT/util/dbt.cc: In function 'int toku_dbt_set(uint32_t, const void*, DBT*, simple_dbt*)':
/builddir/build/BUILD/mariadb-10.2.6/storage/tokudb/PerconaFT/util/dbt.cc:201:21: error: this statement may fall through [-Werror=implicit-fallthrough=]
d->ulen = 0;~~~~~~~~^~~/builddir/build/BUILD/mariadb-10.2.6/storage/tokudb/PerconaFT/util/dbt.cc:203:9: note: here
case (DB_DBT_REALLOC):
^~~~cc1plus: all warnings being treated as errors
- The logrotate is still in my TODO list.
I just saw MariaDB 10.2.7 is available, so i tried it!
Unfortunatelly, a build error occured on all architectures 
:
make -f libmariadb/libmariadb/CMakeFiles/SYM_libmysqlclient.so.dir/build.make libmariadb/libmariadb/CMakeFiles/SYM_libmysqlclient.so.dir/build |
/builddir/build/BUILD/mariadb-10.2.7/mysys_ssl/openssl.c: In function 'coc_malloc': |
/builddir/build/BUILD/mariadb-10.2.7/mysys_ssl/openssl.c:37:1: error: parameter name omitted |
static void *coc_malloc(size_t size, const char *, int)
|
^~~~~~
|
/builddir/build/BUILD/mariadb-10.2.7/mysys_ssl/openssl.c:37:1: error: parameter name omitted |
https://github.com/MariaDB/server/commit/7fc75c420a47fd29a399468b920641ebb43fa75c
Thanks, that patch worked just fine, no more FTBFS!
Now I can confirm, that the issue with the debug build on X86_64, I described in this comment, remains in 10.2.7 too.
Described in what comment? I suspect you have the wrong link. The comment you've mentioned was already after 10.2.7 is released (you say there "I just saw MariaDB 10.2.7 is available, so i tried it").
Oh, now I see. I thought I built last time 10.2.6, but it looks like I built with 10.2.7 both times.
Ok, let's just jake it as an confirmation 
Just tested with gcc-7.1.0 — all compiles fine. There are warnings, but I'll fix them in MDEV-13459 (needs to be done starting from 5.5)
In fact, only 10.1 compiles, 10.2 doesn't.
Fails when compiling AWS SDK — it enables -Werror and fails on "may fall-through". AWS SDK is not part of the source tree, it's downloaded as needed. I guess, we need to switch to a newer version of AWS SDK where this is fixed. This failure doesn't affect Fedora, as you don't download AWS SDK during the build.
Fixed AWS SDK version