Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-31801

Undefined behavior in ed25519

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Minor
    • Resolution: Duplicate
    • None
    • N/A
    • N/A
    • None

    Description

      The library used by the ed25519 authentication plugin exhibits undefined behavior as reported by UBSAN.

      This is edited example output from UBSAN that shows that negative values are used in left shift operations.

      ref10/fe_sq.c:121:76: runtime error: left shift of negative value -46510040
          #0 0x7f17794c866d in ref10_fe_sq ref10/fe_sq.c:121
          #1 0x7f17794a4700 in ref10_ge_frombytes_negate_vartime ref10/ge_frombytes.c:21
          #2 0x7f17794a18c2 in crypto_sign_open ref10/open.c:24
      

      Analyzing the code in the MariaDB server, it is seen that it also uses signed integers with both left and right shift operations. The latter is not as big of a problem (I assume) than the former as right shift of negative integers is only implementation defined whereas a left shift is undefined behavior.

      I believe marko can confirm that he has also seen this when running with UBSAN.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              markus makela markus makela
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.