[MDEV-27956] hardware lock elision on s390x / ppc64 detection improvements Created: 2022-02-28  Updated: 2022-08-18  Resolved: 2022-03-11

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Fix Version/s: 10.6.8, 10.7.4, 10.8.3

Type: Task Priority: Minor
Reporter: Daniel Black Assignee: Daniel Black
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-28137 Some memory transactions are unnecess... Closed
relates to MDEV-29324 Compile error in debug mode on s390x ... Closed

 Description   

As requested in MDEV-27936, s390x for transactional memory of s390 was requested.

In going far to much in the fix for MDEV-27936 the branch containing this was developed.

The first commit is probably a debian compiler downstream issue and will be omitted.

marko, feedback on continuing aspects of this implementation requested please:

For the rest of the s390x changes, target version 10.9?

In making a common s390x/ppc64le htm detection I used a SIGILL based detection modeled of openssl. sigsetjmp not fully understood, seems to avoid SIGILL during testing, but scrutiny requested.

Problems is solves are:

  • compatible with non-linux (BSDs)
  • compatible with s390x
  • The innodb implementation uses transactions without syscalls (please validate this closely that no syscall gets called from under transactional_shared_lock_guard), the feature detection of the suspend mode (not explicitly used, and please don't) and operation system call interface, is overly strict.
  • the s390x tm begin implementation defaults to no floats. I assume because I haven't looked at the ISA, that this means that any use of floats under HTM won't get restored on tabort. Is there consequences here?

If all these are moderately on track, what version should the transactional_lock_enabled go into?

 Comments   
Comment by Marko Mäkelä [ 2022-03-08 ]

I think that it should be fine to introduce the s390x lock elision in 10.6.

The critical sections of lock elision should be very small, and I don’t think that they should include any floating-point operations. There should be no system calls either, except maybe for debug assertions that might call write() and abort(), should they hold. I tested the Intel TSX-NI implementation and paid attention to the tx-abort event counts reported by perf, and removed some too optimistic lock elision sections until I got it to a reasonable level. I think that on Intel TSX-NI, any system call would lead to memory transaction abort.

Comment by Daniel Black [ 2022-03-09 ]

bb-10.6-danielblack-MDEV-27956-htm-s390x

  • Debian 11 - detects TM (worker p9-rhel8-bbw1-docker-debian-11, logs), passes all tests
  • Debian sid - no TM (worker p9-db-bbw1-docker-debian-sid, logs), passes same tests as 10.6 branch (something infrastructure related is wrong- MDBF-351)
  • ubuntu-21.10 queued

s390x

Comment by Daniel Black [ 2022-03-09 ]

first confirmation - https://buildbot.mariadb.org/#/builders/309/builds/2839

 
CURRENT_TEST: binlog_encryption.encrypted_master
 
$ /home/buildbot/s390x-ubuntu-2004/build/sql/mariadbd --defaults-group-suffix=.1 --defaults-file=/home/buildbot/s390x-ubuntu-2004/build/mysql-test/var/1/my.cnf --log-output=file --binlog-format=mixed --loose-file-key-management-encryption-algorithm=aes_cbc --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --loose-enable-partition --log-bin=master-bin --core-file --loose-debug-sync-timeout=300
 
2022-03-09 19:25:55 0 [Note] /home/buildbot/s390x-ubuntu-2004/build/sql/mariadbd (server 10.6.8-MariaDB-log) starting as process 17435 ...
 
2022-03-09 19:25:55 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 32190)
 
2022-03-09 19:25:55 0 [Warning] Changed limits: max_open_files: 1024  max_connections: 151 (was 151)  table_cache: 421 (was 2000)
 
2022-03-09 19:25:55 0 [Note] Plugin 'SEQUENCE' is disabled.
 
2022-03-09 19:25:55 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
 
2022-03-09 19:25:55 0 [Note] InnoDB: Using transactional memory
 
...

Comment by Daniel Black [ 2022-03-10 ]

sles-15 happy - from https://ci.mariadb.org/23090/logs/s390x-sles-15/mysqld.1.err.1

 
CURRENT_TEST: binlog_encryption.rpl_packet
 
$ /buildbot/s390x-sles-15/build/sql/mariadbd --defaults-group-suffix=.1 --defaults-file=/buildbot/s390x-sles-15/build/mysql-test/var/1/my.cnf --log-output=file --log-bin=master-bin --binlog-format=row --log-bin=master-bin --core-file --loose-debug-sync-timeout=300
 
2022-03-09 21:56:07 0 [Note] /buildbot/s390x-sles-15/build/sql/mariadbd (server 10.6.8-MariaDB-log) starting as process 25615 ...
 
2022-03-09 21:56:07 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 32190)
 
2022-03-09 21:56:07 0 [Warning] Changed limits: max_open_files: 1024  max_connections: 151 (was 151)  table_cache: 421 (was 2000)
 
2022-03-09 21:56:07 0 [Note] Plugin 'partition' is disabled.
 
2022-03-09 21:56:07 0 [Note] Plugin 'SEQUENCE' is disabled.
 
2022-03-09 21:56:07 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
 
2022-03-09 21:56:07 0 [Note] InnoDB: Using transactional memory

Generated at Thu Feb 08 09:56:55 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.