range optimizer derives wrong boundary when constant bitwise expression involves IS NULL on NOT NULL column

Description:

A WHERE predicate containing a bitwise OR expression with (- TRUE) and a NOT NULL column's IS NULL check produces inconsistent results depending on whether the range optimizer derives an index scan boundary or the expression evaluator computes the predicate row-by-row.
The range optimizer appears to fold the constant expression (-1 | 0) as unsigned (18446744073709551615), producing an empty range. However, when the same predicate is evaluated in a derived table (forcing expression evaluation), the constant is folded as signed (-1), yielding TRUE for rows where -1 < c0.
This violates the invariant that a predicate must evaluate to the same truth value regardless of execution path.

Version:

mariadb:11.8.5

Steps to Reproduce:

CREATE TABLE t0(c0 REAL SIGNED  UNIQUE NOT NULL) engine=MyISAM;

INSERT INTO t0 VALUES (-1698810807);

INSERT INTO t0 VALUES (1300539435);

SELECT t0.c0 FROM t0 WHERE (+ (((- true) | (t0.c0 IS NULL)) < t0.c0));-- cardinality: 0

SELECT ref0 FROM (SELECT t0.c0 AS ref0, (+ (((- true) | (t0.c0 IS NULL)) < t0.c0)) AS ref1 FROM t0) AS s WHERE ref1;-- cardinality: 1

Expected Result:

Both queries should return the same result. Since c0 is NOT NULL, t0.c0 IS NULL is always FALSE (0). The expression (-1 | 0) evaluates to -1 in signed context, and -1 < -1698810807 is FALSE, -1 < 1300539435 is TRUE. Therefore Query A should return 1 row (1300539435), matching Query B.

Actual Result:

Query A returns 0 rows
Query B returns 1 row

mysql> SELECT t0.c0 FROM t0 WHERE (+ (((- true) | (t0.c0 IS NULL)) < t0.c0));-- cardinality: 0

Empty set (0.00 sec)

mysql> SELECT ref0 FROM (SELECT t0.c0 AS ref0, (+ (((- true) | (t0.c0 IS NULL)) < t0.c0)) AS ref1 FROM t0) AS s WHERE ref1;-- cardinality: 1

+------------+

| ref0       |

+------------+

| 1300539435 |

+------------+

1 row in set (0.00 sec)

Explain Evidence:

Query A (range access) — the optimizer folds the constant as an unsigned large integer, deriving an empty range:

{

  "table": {

    "table_name": "t0",

    "access_type": "range",

    "key": "c0",

    "used_key_parts": ["c0"],

    "rows": 1,

    "attached_condition": "<cache>(-1 | (/*always not null*/ 1 is null)) < t0.c0"

  }

}

Query B (index full scan) — the same expression is evaluated per-row, correctly matching one row:

{

  "table": {

    "table_name": "t0",

    "access_type": "index",

    "key": "c0",

    "rows": 2,

    "attached_condition": "<cache>(-1 | (/*always not null*/ 1 is null)) < t0.c0 is true"

  }

}

Analysis:

The range optimizer folds (- TRUE) | (t0.c0 IS NULL) using the unsigned result type of bitwise | (BIGINT UNSIGNED), producing 18446744073709551615. This huge boundary makes the range empty. However, the expression evaluator treats the folded constant as signed (-1), allowing the comparison -1 < 1300539435 to succeed.
The inconsistency lies in how the sign attribute of a bitwise constant expression is preserved (or lost) between the range optimizer's boundary derivation and the runtime expression evaluator.