How to replay the problem?
Please

• copy t_s_3A.test to the directory main
• run the test like
  ./mysql-test-run.pl --mem --suite=main t_s_3A | tee prt
• just check the output in "prt"
  and do not become confused by the fact that MTR reports a "pass".

Here is a more concise test case that demonstrates the issue on 10.2. I suspect that it could be present on 10.0 and 10.1 as well:

--source include/have_innodb.inc

CREATE TABLE t_innodb_INPLACE(c INT) ENGINE = InnoDB;

CREATE TABLE t_innodb_COPY(c INT) ENGINE = InnoDB;

INSERT INTO t_innodb_INPLACE VALUES (NULL);

INSERT INTO t_innodb_COPY VALUES (NULL);

ALTER IGNORE TABLE t_innodb_COPY    MODIFY COLUMN c INT NOT NULL, ALGORITHM = COPY;

--error ER_INVALID_USE_OF_NULL

ALTER IGNORE TABLE t_innodb_INPLACE MODIFY COLUMN c INT NOT NULL, ALGORITHM = INPLACE;

SELECT * FROM t_innodb_INPLACE;

SELECT * FROM t_innodb_COPY;

DROP TABLE t_innodb_INPLACE, t_innodb_COPY;

The problem is that ALGORITHM=INPLACE is throwing an error, while ALGORITHM=COPY allows the operation to go through (the NULL is being updated to 0).
Because the attached test case t_s_3A.test was ignoring all errors, the errors were propagated.

On MariaDB 10.3, the ALTER IGNORE TABLE will not fail, but it will actually produce the warning. So, we must fix different things in this area.

Starting with MariaDB 10.2, SQL_MODE includes STRICT_TRANS_TABLES by default. In older versions, you would have to set that in order to enable ALGORITHM=INPLACE for introducing a NOT NULL attribute to a column.

The sql_mode in the test effectively turns off STRICT_ALL_TABLES or STRICT_TRANS_TABLES. Starting with MDEV-16365 (and MDEV-14168) in MariaDB 10.3.8, this should be equivalent to executing ALTER IGNORE TABLE, which should update the NULL column values in a similar way as ALGORITHM=COPY would.

In MariaDB 10.0, 10.1 and 10.2, we should have refused ALTER IGNORE TABLE when introducing a NOT NULL attribute. (This is available when one of strict_all_tables or strict_trans_tables is set in sql_mode.) ALGORITHM=INPLACE allowed the ALTER IGNORE TABLE but returned an error for the NULL value. With the fix in place, 10.1 and 10.2 will refuse ALTER IGNORE TABLE for introducing NOT NULL attribute, and ALGORITHM=COPY will do the right thing.

While merging the fix from 10.1 to 10.2, I noticed that a regression had been introduced in MariaDB 10.2.2: ALTER IGNORE TABLE would fail to refuse ALGORITHM=INPLACE for ADD PRIMARY KEY or ADD UNIQUE KEY. I restored that code and added a test case.

For MariaDB Server 10.3, we need a bigger fix. Either we will have to refuse the ALTER IGNORE TABLE if indexed virtual columns depend on the column, or we will have to compute the dependent indexed virtual column values when updating the NULL column values.

The bug in MariaDB 10.3 only affects generated stored columns. For indexed virtual columns, we would use ALGORITHM=COPY because the ALTER_COLUMN_VCOL flag would be set in ha_alter_info->handler_flags.

The bug in MariaDB 10.3 is not limited to ALTER IGNORE TABLE. It also occurs when sql_mode does not include either of STRICT_ALL_TABLES or STRICT_TRANS_TABLES. The bug is that the generated stored columns are not being recomputed when a NULL value in a base column is being replaced with something else. I believe that it is easiest to fall back to ALGORITHM=COPY in such cases.

I merged the fixes up to 10.3 so far. Part of the problem in 10.3 and 10.4 remains to be fixed by MDEV-18819.