Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
10.2.7
-
None
Description
So I noticed something a bit funny about the MariaDB 10.2 install process, when it comes to header files. It seems there are kinda two competing install processes.
The source code includes a libmariadb/ subdirectory, which is actually (in the git repo) the mariadb-connector-c project as a submodule. This project has its own include/ subdirectory, which contains several header files and some install rules for them in a CMakeLists.txt file.
But the mariadb source also has its own include/ subdirectory, which similarly contains several header files and some install rules for them. In several cases, the install rules for both include/ subdirectories try to install a file with the same name to the same place: this includes at least mysql.h and errmsg.h . You can spot cases like this in the console output by looking for the string 'Up-to-date', which basically seems to be CMake indicating that a file listed for installation already exists and won't be replaced.
AFAICS, at least during our (Fedora's) builds, the libmariadb/include/ subdirectory gets to go first, and then when the include/ subdirectory runs, it doesn't overwrite any file that the libmariadb/include/ subdirectory already installed. But as there are more headers in the include/ subdirectory, you ultimately wind up with a mish-mash of headers installed, some from the libmariadb/ subproject and some from mariadb itself.
This is particularly significant in the case of mysql.h; what actually winds up installed is libmariadb/include/mysql.h , not include/mysql.h . There are some significant differences between the two; for instance, the one from libmariadb/include/ includes mariadb_version.h from the same directory , whereas the mysql.h from include/ includes mysql_version.h from the same directory. libmariadb's mariadb_version.h defines rather fewer things than mariadb's mysql_version.h does. So ultimately, for instance, if you install MariaDB 10.2, then write some code that just does #include mysql.h , then you will have no MYSQL_SERVER_VERSION defined.
This is definitely confusing and ambiguous, and I think it's a straight up bug. I would think that when installing MariaDB itself then we really want to install include/mysql.h , not libmariadb/include/mysql.h .
Attachments
Issue Links
- causes
-
-
- Closed
-
Activity
Well, the header you want to use, is that header that gets installed, from libmariadb. Ultimately, this is the client library header.
I understand that it is very confusing that we need 2 mysql.h, the thing is that libmariadb's mysql.h cannot be used in server compilation, due to possibly lacking structures, and I do not remember what else. Ultimately, we need a one true mysql.h, and this needs to come from libmariadb project, and it needs to contain whatever you expect it to contain, possibly including MYSQL_SERVER_VERSION , and also there exists a bug https://jira.mariadb.org/browse/CONC-210 about mysql.h not usable in server's compilation.
But to set expectation, the order in which it installs for you is not the wrong order.
Hum...okay. Still, seems odd that both sets of headers get installed, in that case? What you wind up with is not really either the 'main project' headers or the 'libmariadb' headers, but some sort of mish-mash of both....
Note, this is not about compiling the server itself. We're compiling all the various packages in Fedora that are built against mariadb, several of which expect things that are available via include/mysql.h but are not available via libmariadb/include/mysql.h (things from mysql_version.h being the most obvious so far).
I have for now tweaked Fedora's mariadb package to give the include/ headers precedence over the libmariadb/include/ headers, as this seems to better match what projects are expecting and what MariaDB 10.1 did, till this mess is sorted out...
Here's an example of a project which has run into the "MYSQL_SERVER_VERSION is missing" problem and added an explict #include mysql_version.h to fix it:
https://github.com/brianmario/mysql2/issues/851
https://github.com/brianmario/mysql2/pull/857
I guess it would really help if someone could explain what (if any) parts of the old interface (symbols, definitions etc.) are actually intended to be lost in this changeover? So I at least know whether we should be sending patches to downstream projects to stop using things which are intended to have gone away, or sending patches for mariadb-connector-c to add things which aren't intended to have gone away...
I'm going to fix it in 10.2.8.
The logic is — if you want to compile a client, you should use libmariadb headers and link with libmariadb.so (or .a).
If you compile a server plugin (e.g. a storage engine), you should use server headers.
Sergei: if you could post a patch when you have it ready that'd be great, then we can apply it to our Rawhide package. thanks.
Of course. But — so that you know what to expect — most likely I'll have it within a week before the next 10.2 release, not earlier. Release schedule is at https://jira.mariadb.org/
AdamW, I'm trying this fix: https://github.com/MariaDB/server/commit/1c0bfcd3ea22654a9c92b3a4c86c967ffcf0e63c
all server includes go under /usr/include/mysql/server. So clients can be compiled as before, they'll use connector-c headers (installed under /usr/include/mysql as before). If someone needs server headers (e.g. to compile a server plugin), she will have to adjust include paths. And if one wants to link with libmysqld, embedded server (Akonadi does) — it also requires server includes at the moment.
Hi Sergei, thanks for working on it. Just FWIW if it were me I might have chosen to keep the server includes in the old location - since that's what all previous versions, and old MySQL, used to put there - and put the client includes in a new location. But I think both choices are fine and we can certainly work with either.
Can I suggest that it'd be good to also enhance `mariadb_config` so it can be used for both cases? I'm not sure if you can somehow expose the alternative choices in the pkgconfig file, but I think you can at least enhance `mariadb_config`, since you entirely control the behaviour of that tool.
Technically, both server and client includes were in /usr/include/mysql before. So, no matter what is moved, some projects will need to adjust include paths. And there are orders of magnitude more MariaDB/MySQL clients (that need client includes) than external MariaDB storage engines (I know only of, perhaps, two projects that need server includes).
I'll see what we can do about mariadb_config. This tool is part of connector-c codebase, it would be somewhat strange if it'd know about server includes (C/C can be built and installed independently too). If that wouldn't work, I'll fix mysql_config instead.
Haven't fixed mariadb_config or mysql_config. Not quite clear what needs to be fixed or how, so let's wait for use cases — for any project that needs server includes (external storage engine, anyone?).
Well, all I'm suggesting is there should be a way to get the server include path as well as the client include path. I guess if we can guarantee the server include path will always be 'client include path plus /server' then that might be enough, and there's no need to add an argument you can use to get the command to spit out the exact server include path...
> all server includes go under /usr/include/mysql/server. So clients can be compiled as before, they'll use connector-c headers (installed under /usr/include/mysql as before).
In homebrew (the mac os package mgr.) the mariadb and mariadb-connector-c packages are marked as conflicting. Meaning, they cannot both be installed. But, if I understand correctly, we should be installing both in order to compile clients, so should we open an issue to alert the homebrew team about this change? (The repository with the "formulae", as brew calls them, is https://github.com/Homebrew/homebrew-core)
PS: just FYI the issue that brought me here is that the ruby client no longer compiles (most ruby people [not all] use homebrew) (see https://github.com/brianmario/mysql2/issues/878 and https://github.com/brianmario/mysql2/pull/880)
Jared: I don't think that's what Sergei means, no. I believe that the full mariadb will contain both the server and the client headers, in different directories. mariadb-connector-c will only contain the client headers, in the same place mariadb puts them. Again, this is just as I understand it.
AdamW, yes, that's right. mariadb-connector-c package will contain only client headers and it will be sufficient for compiling client applications.
jaredbeck, I suspect that now mariadb and mariadb-connector-c will not conflict anymore, so they can (but not should) be safely installed side by side. But please double check that (and do complain if they still conflict).
> I suspect that now mariadb and mariadb-connector-c will not conflict anymore ...
Thanks Sergei, I have passed along your analysis to the homebrew team: https://github.com/Homebrew/homebrew-core/pull/17202
As a result of the patch for this issue [1], mysql_version.h is installed under /usr/include/mysql/server as of MariaDB 10.2.8.
Given that 'mysql_config --cflags' still reports '-I/usr/include/mysql' in MariaDB 10.2.8, what would be the best way to discover mysql_version.h location for software that wants to be portable across MariaDB versions and MySQL flavors?
In MDEV-13773 I'm providing compatibility headers, mysql_version.h is one of them. It's not pushed into 10.2 at the moment of writing of this comment, but will be in 10.2.9. Feel free to look at the patch (linked from MDEV-13773) and comment on it, so that I could fix it to work for you. So far I've only tested it for projects mentioned in MDEV-13773.
Setting CMAKE_ALWAYS_INSTALL is a kinda hacky way to work around this - it tells CMake to always install files even if the target already exists. So it does the right thing (that is, prefers the files from include/ over the ones from libmariadb/include/ ) so long as include/ gets installed after libmariadb/include/ . If the order ever flipped, it'd do the wrong thing.