Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-11203

Test with atomic_writes (FusionIO, shannon ssd)

Details

    • 10.2.4-1

    Description

      Use O_DIRECT and innodb-trim=1, innodb-doublewrite=0 and
      create normal tables, encrypted tables, compressed tables, compressed+encrypted tables
      insert rows
      restart
      select rows

      Attachments

        Issue Links

          Activity

            elenst Elena Stepanova added a comment - - edited

            Test 3

            I've modified "Test 1"to hopefully get more chance for trim. The tables now have this structure:

            CREATE TABLE `t1` (
              `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
              `k` int(10) unsigned NOT NULL DEFAULT 0,
              `c` char(255) NOT NULL DEFAULT '',
              `pad` varchar(2048) NOT NULL DEFAULT '',
              PRIMARY KEY (`id`),
              KEY `k` (`k`),
              KEY `pad` (`pad`(128))
            )
            

            And INSERTs do this:

            INSERT INTO `t1` SELECT NULL, seq%1000, REPEAT('c',255), CONCAT('pad',seq%100,REPEAT('*',1024)) FROM seq_1_to_100000;
            

            The config was also modified a bit:

            [mysqld]
            innodb-use-atomic-writes
            innodb_use_fallocate
            innodb_flush_method=O_DIRECT
            innodb-use-trim
            innodb_doublewrite=0
            plugin-load-add=file_key_management.so
            file-key-management-filename=/home/elenst/10.2/mysql-test/std_data/keys.txt
            innodb-encryption-threads=4
            

            And I'm still getting no trim at all on 10.2:

            10.2 status

            MariaDB [(none)]> show global status like '%compress%';
            +------------------------------------------+------------+
            | Variable_name                            | Value      |
            +------------------------------------------+------------+
            | Compression                              | OFF        |
            | Innodb_page_compression_saved            | 1437753344 |
            | Innodb_page_compression_trim_sect512     | 0          |
            | Innodb_page_compression_trim_sect1024    | 0          |
            | Innodb_page_compression_trim_sect2048    | 0          |
            | Innodb_page_compression_trim_sect4096    | 0          |
            | Innodb_page_compression_trim_sect8192    | 0          |
            | Innodb_page_compression_trim_sect16384   | 0          |
            | Innodb_page_compression_trim_sect32768   | 0          |
            | Innodb_num_pages_page_compressed         | 119847     |
            | Innodb_num_page_compressed_trim_op       | 0          |
            | Innodb_num_page_compressed_trim_op_saved | 0          |
            | Innodb_num_pages_page_decompressed       | 238921     |
            | Innodb_num_pages_page_compression_error  | 26827      |
            | Innodb_defragment_compression_failures   | 0          |
            +------------------------------------------+------------+
            

            On 10.1, the exact same test with the same config (with additional innodb_file_format=Barracuda, which is default in 10.2) clearly causes trimming:

            10.1 status

            MariaDB [(none)]> show global status like '%compress%';
            +------------------------------------------+------------+
            | Variable_name                            | Value      |
            +------------------------------------------+------------+
            | Compression                              | OFF        |
            | Innodb_page_compression_saved            | 1449443328 |
            | Innodb_page_compression_trim_sect512     | 0          |
            | Innodb_page_compression_trim_sect1024    | 0          |
            | Innodb_page_compression_trim_sect2048    | 0          |
            | Innodb_page_compression_trim_sect4096    | 302313     |
            | Innodb_page_compression_trim_sect8192    | 0          |
            | Innodb_page_compression_trim_sect16384   | 0          |
            | Innodb_page_compression_trim_sect32768   | 0          |
            | Innodb_num_pages_page_compressed         | 120871     |
            | Innodb_num_page_compressed_trim_op       | 101120     |
            | Innodb_num_page_compressed_trim_op_saved | 33582      |
            | Innodb_num_pages_page_decompressed       | 240934     |
            | Innodb_num_pages_page_compression_error  | 27398      |
            | Innodb_defragment_compression_failures   | 0          |
            +------------------------------------------+------------+
            15 rows in set (0.01 sec)
            

            So, it seems the core problem is that trim does not work at all on 10.2?

            elenst Elena Stepanova added a comment - - edited Test 3 I've modified "Test 1"to hopefully get more chance for trim. The tables now have this structure: CREATE TABLE `t1` ( `id` int (10) unsigned NOT NULL AUTO_INCREMENT, `k` int (10) unsigned NOT NULL DEFAULT 0, `c` char (255) NOT NULL DEFAULT '' , `pad` varchar (2048) NOT NULL DEFAULT '' , PRIMARY KEY (`id`), KEY `k` (`k`), KEY `pad` (`pad`(128)) ) And INSERTs do this: INSERT INTO `t1` SELECT NULL , seq%1000, REPEAT( 'c' ,255), CONCAT( 'pad' ,seq%100,REPEAT( '*' ,1024)) FROM seq_1_to_100000; The config was also modified a bit: [mysqld] innodb-use-atomic-writes innodb_use_fallocate innodb_flush_method=O_DIRECT innodb-use-trim innodb_doublewrite=0 plugin-load-add=file_key_management.so file-key-management-filename=/home/elenst/10.2/mysql-test/std_data/keys.txt innodb-encryption-threads=4 And I'm still getting no trim at all on 10.2: 10.2 status MariaDB [(none)]> show global status like '%compress%'; +------------------------------------------+------------+ | Variable_name | Value | +------------------------------------------+------------+ | Compression | OFF | | Innodb_page_compression_saved | 1437753344 | | Innodb_page_compression_trim_sect512 | 0 | | Innodb_page_compression_trim_sect1024 | 0 | | Innodb_page_compression_trim_sect2048 | 0 | | Innodb_page_compression_trim_sect4096 | 0 | | Innodb_page_compression_trim_sect8192 | 0 | | Innodb_page_compression_trim_sect16384 | 0 | | Innodb_page_compression_trim_sect32768 | 0 | | Innodb_num_pages_page_compressed | 119847 | | Innodb_num_page_compressed_trim_op | 0 | | Innodb_num_page_compressed_trim_op_saved | 0 | | Innodb_num_pages_page_decompressed | 238921 | | Innodb_num_pages_page_compression_error | 26827 | | Innodb_defragment_compression_failures | 0 | +------------------------------------------+------------+ On 10.1, the exact same test with the same config (with additional innodb_file_format=Barracuda , which is default in 10.2) clearly causes trimming: 10.1 status MariaDB [(none)]> show global status like '%compress%'; +------------------------------------------+------------+ | Variable_name | Value | +------------------------------------------+------------+ | Compression | OFF | | Innodb_page_compression_saved | 1449443328 | | Innodb_page_compression_trim_sect512 | 0 | | Innodb_page_compression_trim_sect1024 | 0 | | Innodb_page_compression_trim_sect2048 | 0 | | Innodb_page_compression_trim_sect4096 | 302313 | | Innodb_page_compression_trim_sect8192 | 0 | | Innodb_page_compression_trim_sect16384 | 0 | | Innodb_page_compression_trim_sect32768 | 0 | | Innodb_num_pages_page_compressed | 120871 | | Innodb_num_page_compressed_trim_op | 101120 | | Innodb_num_page_compressed_trim_op_saved | 33582 | | Innodb_num_pages_page_decompressed | 240934 | | Innodb_num_pages_page_compression_error | 27398 | | Innodb_defragment_compression_failures | 0 | +------------------------------------------+------------+ 15 rows in set (0.01 sec) So, it seems the core problem is that trim does not work at all on 10.2?

            jplindst, marko, do you feel like it's time to resume the tests? And given the overall status of the feature, is it to be done by 10.2 GA?
            They were stalled due to MDEV-11254, which has now been fixed, but I see that MDEV-11232 which is also linked to this task is still open.

            elenst Elena Stepanova added a comment - jplindst , marko , do you feel like it's time to resume the tests? And given the overall status of the feature, is it to be done by 10.2 GA? They were stalled due to MDEV-11254 , which has now been fixed, but I see that MDEV-11232 which is also linked to this task is still open.

            elenst, sorry, I had missed your question back then. I encountered this ticket while searching for something else.

            The logic to detect atomic writes should now be automatic and actually work correctly (MDEV-26042). But, special hardware is still needed for testing that, and additional access permissions directly to the block device are needed as well, for ioctl() to work. A sysfs interface similar to /sys/block/*/queue/rotational (MDEV-17380) would be much nicer.

            I last tested this related to MDEV-26029 and MDEV-26192 some months ago.

            Should we acquire and use suitable hardware in some of our internal stress tests?

            marko Marko Mäkelä added a comment - elenst , sorry, I had missed your question back then. I encountered this ticket while searching for something else. The logic to detect atomic writes should now be automatic and actually work correctly ( MDEV-26042 ). But, special hardware is still needed for testing that, and additional access permissions directly to the block device are needed as well, for ioctl() to work. A sysfs interface similar to /sys/block/*/queue/rotational ( MDEV-17380 ) would be much nicer. I last tested this related to MDEV-26029 and MDEV-26192 some months ago. Should we acquire and use suitable hardware in some of our internal stress tests?

            marko, mleich, maybe it would better belong to the InnoDB team then?
            Matthias, do you want to take over and request the needed hardware to use in your internal stress tests?

            elenst Elena Stepanova added a comment - marko , mleich , maybe it would better belong to the InnoDB team then? Matthias, do you want to take over and request the needed hardware to use in your internal stress tests?

            elenst, yes, I think that this belongs to our team. I think that it suffices to run a one-time test campaign. I think that the hardware will be most useful in an environment that is primarily used for performance tests.

            marko Marko Mäkelä added a comment - elenst , yes, I think that this belongs to our team. I think that it suffices to run a one-time test campaign. I think that the hardware will be most useful in an environment that is primarily used for performance tests.

            People

              mleich Matthias Leich
              jplindst Jan Lindström (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.