Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-33817

Implement AVX512BW and VPCLMULQDQ based CRC-32 algorithms

Details

    Description

      MDEV-22749 introduced faster checksum calculation on AMD64 by means of the 128-bit carry-less multiplication (pclmul) of the AVX ISA extension. Some recent ISA extensions include wider vpclmulqdq as well as vmovdqu8, which allows unaligned loads of up to 64 bytes at a time. Making use of these instructions could greatly improve performance.

      Some reference implementations exist in NASM format in https://github.com/intel/intel-ipsec-mb/ and https://github.com/intel/isa-l/ under the BSD 3-clause license. CRC-32 with the ISO 3309 polynomical is called gzip, and the Castagnoli polynomial is referred to as SCTP or iSCSI.

      Attachments

        Issue Links

          Activity

            marko Marko Mäkelä created issue -
            marko Marko Mäkelä made changes -
            Field Original Value New Value
            marko Marko Mäkelä made changes -
            Description MDEV-22749 introduced faster checksum calculation on AMD64 by means of the 128-bit carry-less multiplication ({{pclmul}}). The AVX-512 ISA extension introduced a wider variant {{vpmsum}}, which could greatly improve performance.

            Some reference implementations exist in https://github.com/intel/intel-ipsec-mb/ and https://github.com/intel/isa-l/ under the BSD 3-clause license. CRC-32 with the ISO 3309 polynomical is called {{gzip}}, and the Castagnoli polynomial is referred to as SCTP or iSCSI.
            MDEV-22749 introduced faster checksum calculation on AMD64 by means of the 128-bit carry-less multiplication ({{pclmul}}). The AVX-512 ISA extension introduced a wider variant {{vpmsum}}, which could greatly improve performance.

            Some reference implementations exist in NASM format in https://github.com/intel/intel-ipsec-mb/ and https://github.com/intel/isa-l/ under the BSD 3-clause license. CRC-32 with the ISO 3309 polynomical is called {{gzip}}, and the Castagnoli polynomial is referred to as SCTP or iSCSI.

            The header file {{<immintrin.h>}} should define {{_mm512_clmulepi64_epi128()}} in recent enough compilers.
            In recent enough versions of {{clang}} there is {{__builtin_ia32_pclmulqdq512()}}.
            Summary Implement AVX-512 vpclmul based checksums Implement AVX-512 vpclmulqdq based checksums
            marko Marko Mäkelä made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            marko Marko Mäkelä made changes -
            Summary Implement AVX-512 vpclmulqdq based checksums Implement AVX512BW and VPCLMULQDQ based CRC-32 algorithms
            marko Marko Mäkelä made changes -
            Description MDEV-22749 introduced faster checksum calculation on AMD64 by means of the 128-bit carry-less multiplication ({{pclmul}}). The AVX-512 ISA extension introduced a wider variant {{vpmsum}}, which could greatly improve performance.

            Some reference implementations exist in NASM format in https://github.com/intel/intel-ipsec-mb/ and https://github.com/intel/isa-l/ under the BSD 3-clause license. CRC-32 with the ISO 3309 polynomical is called {{gzip}}, and the Castagnoli polynomial is referred to as SCTP or iSCSI.

            The header file {{<immintrin.h>}} should define {{_mm512_clmulepi64_epi128()}} in recent enough compilers.
            In recent enough versions of {{clang}} there is {{__builtin_ia32_pclmulqdq512()}}.
            MDEV-22749 introduced faster checksum calculation on AMD64 by means of the 128-bit carry-less multiplication ({{pclmul}}) of the AVX ISA extension. Variants of the AVX512 ISA extension include wider {{vpclmulqdq}} as well as

            , which could greatly improve performance.

            Some reference implementations exist in NASM format in https://github.com/intel/intel-ipsec-mb/ and https://github.com/intel/isa-l/ under the BSD 3-clause license. CRC-32 with the ISO 3309 polynomical is called {{gzip}}, and the Castagnoli polynomial is referred to as SCTP or iSCSI.

            The header file {{<immintrin.h>}} should define {{_mm512_clmulepi64_epi128()}} in recent enough compilers.
            In recent enough versions of {{clang}} there is {{__builtin_ia32_pclmulqdq512()}}.
            marko Marko Mäkelä made changes -
            Description MDEV-22749 introduced faster checksum calculation on AMD64 by means of the 128-bit carry-less multiplication ({{pclmul}}) of the AVX ISA extension. Variants of the AVX512 ISA extension include wider {{vpclmulqdq}} as well as

            , which could greatly improve performance.

            Some reference implementations exist in NASM format in https://github.com/intel/intel-ipsec-mb/ and https://github.com/intel/isa-l/ under the BSD 3-clause license. CRC-32 with the ISO 3309 polynomical is called {{gzip}}, and the Castagnoli polynomial is referred to as SCTP or iSCSI.

            The header file {{<immintrin.h>}} should define {{_mm512_clmulepi64_epi128()}} in recent enough compilers.
            In recent enough versions of {{clang}} there is {{__builtin_ia32_pclmulqdq512()}}.
            MDEV-22749 introduced faster checksum calculation on AMD64 by means of the 128-bit carry-less multiplication ({{pclmul}}) of the AVX ISA extension. Some recent ISA extensions include wider {{vpclmulqdq}} as well as {{vmovdqu8}}, which allows unaligned loads of up to 64 bytes at a time. Making use of these instructions could greatly improve performance.

            Some reference implementations exist in NASM format in https://github.com/intel/intel-ipsec-mb/ and https://github.com/intel/isa-l/ under the BSD 3-clause license. CRC-32 with the ISO 3309 polynomical is called {{gzip}}, and the Castagnoli polynomial is referred to as SCTP or iSCSI.
            marko Marko Mäkelä made changes -
            Assignee Marko Mäkelä [ marko ] Vladislav Vaintroub [ wlad ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            marko Marko Mäkelä made changes -
            wlad Vladislav Vaintroub made changes -
            Assignee Vladislav Vaintroub [ wlad ] Marko Mäkelä [ marko ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            issue.field.resolutiondate 2024-05-03 13:29:22.0 2024-05-03 13:29:21.584
            marko Marko Mäkelä made changes -
            Fix Version/s 10.5.25 [ 29626 ]
            Fix Version/s 10.6.18 [ 29627 ]
            Fix Version/s 10.11.8 [ 29630 ]
            Fix Version/s 11.0.6 [ 29628 ]
            Fix Version/s 11.1.5 [ 29629 ]
            Fix Version/s 11.2.4 [ 29631 ]
            Fix Version/s 11.4.2 [ 29633 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.6 [ 24028 ]
            Fix Version/s 10.11 [ 27614 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            danblack Daniel Black made changes -
            danblack Daniel Black made changes -

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.