Incorrect bitwise right-shift results for shift counts ≥ 2^32 due to 32-bit truncation

Description

MariaDB produces incorrect results for bitwise right-shift operations (`>>`) when the shift count exceeds the range of a 32-bit unsigned integer.

For example:

SELECT b'101010' >> 4294967297 AS v;

Actual result on MariaDB:

21

Expected result:

0

The binary literal `b'101010'` represents the integer `42`. MariaDB performs bitwise shifting on a 64-bit `BIGINT` representation.

Therefore, shifting this value right by `4294967297` bits should produce zero, because the shift count exceeds the 64-bit operand width.

However, MariaDB appears to truncate the shift count to 32 bits before performing the operation.

Additional experiments show:

These results are consistent with the following incorrect conversion:

effective_shift = shift_count mod 2^32

Consequently:

• `4294967296` becomes `0`, producing `42 >> 0 = 42`.
• `4294967297` becomes `1`, producing `42 >> 1 = 21`.
• `4294967298` becomes `2`, producing `42 >> 2 = 10`.

The issue is also supported by MariaDB source code.

In `sql/sql_type_int.h`, the implementation of `Longlong_null::operator>>()` contains:

ulonglong res;

uint shift;

if (is_null() || llshift.is_null())

    return Longlong_null();

shift = (uint) llshift.value();

res = 0;

if (shift < sizeof(longlong) * 8)

    res = ((ulonglong) value()) >> shift;

return Longlong_null(res);

The assignment:

shift = (uint) llshift.value();

converts the original 64-bit shift count into a 32-bit unsigned integer before checking whether the shift count is smaller than 64.

This conversion discards the upper 32 bits, allowing extremely large shift counts to be incorrectly interpreted as small values.

As a result, MariaDB can return nonzero values for right shifts that should produce zero.

Source code:

https://github.com/MariaDB/server/blob/mariadb-12.3.3/sql/sql_type_int.h

How to repeat

No database tables or additional setup are required.

1. Execute the original query

SELECT b'101010' >> 4294967297 AS v;

Actual result (MariaDB):

21

Expected result:

0

2. Verify the 32-bit truncation pattern

SELECT b'101010' >> 4294967296 AS v;

-- Actual: 42

-- Expected: 0

SELECT b'101010' >> 4294967297 AS v;

-- Actual: 21

-- Expected: 0

SELECT b'101010' >> 4294967298 AS v;

-- Actual: 10

-- Expected: 0

All three shift counts are greater than 64 and should therefore produce zero.

Instead, the results correspond to shifting by `0`, `1`, and `2` bits, respectively.

3. Metamorphic Oracle: Sequential Right-Shift Equivalence

For a nonnegative 64-bit integer `x` and nonnegative shift counts `a` and `b`, the following equivalence should hold:

(x >> a) >> b = x >> (a + b)

This property follows from the semantics of logical right shifts, assuming the sum does not overflow.

However, MariaDB violates this equivalence when the combined shift count reaches `2^32`.

Original query (sequential right shifts):

SELECT (42 >> 4294967295) >> 1 AS v;

Expected and predicted MariaDB result:

0

The first shift count is already greater than 64, so the intermediate result is zero. Shifting zero by one additional bit still produces zero.

Semantically equivalent query (combined right shift):

SELECT 42 >> (4294967295 + 1) AS v;

Predicted MariaDB result:

42

The combined shift count is `4294967296` (`2^32`). Due to the suspected 32-bit truncation, MariaDB interprets this shift count as zero and effectively computes `42 >> 0 = 42`.

Metamorphic Oracle:

SELECT

    (42 >> 4294967295) >> 1 AS original_result,

    42 >> (4294967295 + 1) AS equivalent_result,

    ((42 >> 4294967295) >> 1) =

    (42 >> (4294967295 + 1)) AS oracle_pass;

Predicted MariaDB result:

original_result   = 0

equivalent_result = 42

oracle_pass       = 0

Both expressions should return zero according to 64-bit right-shift semantics. However, the implementation is expected to produce different results because the combined shift count is truncated to 32 bits.

This metamorphic relation provides a single-DBMS correctness oracle without relying on another database system. It also demonstrates that the issue violates an algebraic equivalence of bitwise right-shift operations, rather than merely producing an unexpected value for an unusually large shift count.

4. Expected behavior

For a 64-bit bitwise right-shift operation, a shift count greater than or equal to 64 should produce zero.

The original shift count must be checked without losing its upper bits.

In particular:

SELECT 42 >> 4294967296;

SELECT 42 >> 4294967297;

SELECT 42 >> 4294967298;

All should return `0`.

5. Actual behavior

MariaDB truncates the shift count to 32 bits before checking its range.

Consequently, the queries return `42`, `21`, and `10`, respectively, instead of zero.

Suggested fix

The implementation of `Longlong_null::operator>>()` should validate the complete shift count before converting it to a narrower integer type.

In particular, the following statement should be reviewed:

shift = (uint) llshift.value();

This conversion can discard the upper 32 bits of the shift count.

A possible approach is to preserve the original 64-bit value when checking the shift boundary and perform any narrowing conversion only after confirming that the shift count is within the valid range.

The corresponding left-shift operator (`Longlong_null::operator<<()`) uses a similar conversion and should also be examined for the same issue.

Regression tests are recommended for shift counts around the following boundaries:

• `63`: largest shift count below the 64-bit width.
• `64`: first shift count that should produce zero.
• `4294967295`: maximum 32-bit unsigned value.
• `4294967296`: first value exceeding the 32-bit unsigned range.
• `4294967297` and `4294967298`: additional values demonstrating incorrect truncation.

The fix should ensure that the full shift count is preserved during boundary validation and that excessively large shift counts consistently produce zero.