[MDEV-26776] mroonga_libgroonga_support_lz4 is set to ON even when lz4 is not available, CREATE behavior changes Created: 2021-10-06  Updated: 2021-10-09  Resolved: 2021-10-09

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - Mroonga
Affects Version/s: N/A
Fix Version/s: N/A

Type: Bug Priority: Critical
Reporter: Elena Stepanova Assignee: Sergei Golubchik
Resolution: Not a Bug Votes: 0
Labels: None

Issue Links:
Problem/Incident
is caused by MDEV-12933 sort out the compression library chaos Closed

 Description   

The variable mroonga_libgroonga_support_lz4 is documented as OFF by default. In fact, on older versions, e.g. on 10.6, it appears to be system/build-dependent: on builds where lz4 is available it is set to ON by default, otherwise OFF. For example on the latest 10.6 build from buildbot, on Debian-10 it is ON and on SLES-12 it is OFF. When a table with lz4 compression is attempted to be created, if lz4 support is OFF, a warning is issued ("Warning 16506 The column flag 'COMPRESS_LZ4' is unsupported. It is ignored") and the compression is ignored.

However on 10.7 with compression providers, mroonga_libgroonga_support_lz4 is always ON, even on the same SLES-12 which doesn't have lz4 at all. And on Debian-10 it is ON regardless whether the provider is loaded or not. And then, when a table with lz4 compression is attempted to be created, an error ER_PROVIDER_NOT_LOADED occurs instead of the warning described above.

The test case is just running something like

create table test.t1 (a int, b text COMMENT 'FLAGS "COLUMN_SCALAR|COMPRESS_LZ4"') engine = mroonga charset = utf8;

with a specific combination of the variable and the providers.



 Comments   
Comment by Sergei Golubchik [ 2021-10-08 ]

Isn't it the same in InnoDB and elsewhere? The engine always supports lz4 (and all other compressions can be possibly enabled at compile time). But you'll get an error when you're actually trying to use it if you don't have the provider loaded.

Comment by Elena Stepanova [ 2021-10-08 ]

Isn't it the same in InnoDB and elsewhere?

No, but the exact analogy doesn't work here. It is a variable for Mroonga, but it is a status value for InnoDB.

MariaDB [test]> show variables like '%lz4%';
+--------------------------------+-------+
| Variable_name                  | Value |
+--------------------------------+-------+
| mroonga_libgroonga_support_lz4 | ON    |
+--------------------------------+-------+
1 row in set (0.002 sec)
 
MariaDB [test]> show status like '%lz4%';
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| Innodb_have_lz4 | OFF   |
+-----------------+-------+
1 row in set (0.002 sec)

MariaDB [test]> install soname 'provider_lz4';
Query OK, 0 rows affected (0.026 sec)
 
MariaDB [test]> show status like '%lz4%';
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| Innodb_have_lz4 | ON    |
+-----------------+-------+
1 row in set (0.002 sec)

The engine always supports lz4 (and all other compressions can be possibly enabled at compile time)

Do you mean it as an intended change in 10.7?
It wasn't the case before, as the description tries to explain. In 10.6 the value of the variable depended on whether lz4 was available at compile time or not. On systems where it wasn't, it was set to OFF.
Now even on the same systems where lz4 is still not available it is set to ON.
If it's an intended change, then okay. I surely don't think it is critical (the priority value is a technicality), and documentation for the variable needs to be adjusted anyway.

Comment by Sergei Golubchik [ 2021-10-09 ]

It's a read-only variable. Basically, it serves a purpose of showing the status, whether mroonga knows about lz4, although, technically a user can turn an existing support OFF.

What was changed is its default value. Yes, it was an intended change. Mroonga now always supports lz4 internally. If one loads a provider plugin, mroonga will use it.

Comment by Elena Stepanova [ 2021-10-09 ]

For the record, the variable is not just informational, it also changes the behavior of CREATE. I tried to indicate it in the description, but it turned out inaccurate. Here is another attempt.

  default value behavior with ON behavior with OFF
10.6 with lz4 ON CREATE / INSERT succeeds CREATE succeeds with a warning, INSERT succeeds
10.6 without lz4 OFF CREATE / INSERT succeeds CREATE succeeds with a warning, INSERT succeeds
pp-10.7 without lz4 provider* ON CREATE succeeds, INSERT fails CREATE succeeds with a warning, INSERT succeeds
pp-10.7 with lz4 provider ON CREATE / INSERT succeeds CREATE succeeds with a warning, INSERT succeeds

* same regardless whether provider_lz4 is not installed or it had never been built for the system

The bold outcomes above indicate behavior with the default value of the variable, and red indicates a change. It is not a great change, as it goes like this:

MariaDB [test]> create table t1 (a int, b text COMMENT 'FLAGS "COLUMN_SCALAR|COMPRESS_LZ4"') engine=Mroonga;
Query OK, 0 rows affected (0.099 sec)
 
MariaDB [test]> insert into t1 values (1,'foo');
Query OK, 1 row affected (0.001 sec)
 
MariaDB [test]> insert into t1 values (2,REPEAT('foo',1000));
ERROR 4183 (HY000): MariaDB tried to use the LZ4 compression, but its provider plugin is not loaded

However I don't think it's critical enough, so if it was intended, then so be it.

In general, judging by the results, I think that the variable was meant to be changed to non-default only one way, ON => OFF, to make the compression option be ignored. It was not expected to be changed from OFF to ON, and the result in this case was not good even before.
It also appears that there is no check for the presence of the compression algorithm upon table creation, Mroonga merely checks the value of the variable. If it's OFF, table creation causes a warning and the compression option is ignored, otherwise it succeeds.
The presence of the compression algorithm only becomes important when a long enough value is inserted into the table.

Generated at Thu Feb 08 09:47:51 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.