[#MDEV-24071] embedded server crashes when mysql_library_init called after mysql_library

General thoughts - it might not be easily doable, and perhaps we should not be fixing it (for one, it probably never worked, and the effort might be too high for "embedded server" component which hade been practically unmaintained for decade(s)).

The reason for failing reinit is that there is quite a lot of static data inside the server, which is changed during server initialization, and this static data is not reset when the server is shut down. There is not reason usually to reset the static data if the server is running in normal mode , as a process. As a workaround, if someone manages to dynamically load and unload the embedded library, I guess the static data will be all reset, so it could be a workaround.

With that, I fixed a crash and an assertion in the server, so now it crashes/asserts in innodb, thus marko will be a new proud owner of this bug

to reproduce, clone 10.6-MDEV-24071, compile with embedded/debug, and execute embtest, located in libmysqld/examples (or libmysqld\examples\Debug on Windows) . For me , it asserts with this callstack

3      embtest.exe!dict_table_get_first_index(const dict_table_t * table)

 4      embtest.exe!dict_load_foreigns(const char * table_name, const char * * col_names, unsigned __int64 trx_id, bool check_charsets, dict_err_ignore_t ignore_err, std::deque<char const *,ut_allocator<char const *,1>> & fk_tables)

 5      embtest.exe!dict_load_table_one(const st_::span<char const> & name, dict_err_ignore_t ignore_err, std::deque<char const *,ut_allocator<char const *,1>> & fk_tables)

 6      embtest.exe!dict_sys_t::load_table(const st_::span<char const> & name, dict_err_ignore_t ignore)

 7      embtest.exe!dict_sys_t::load_sys_tables()

 8      embtest.exe!dict_boot()

 9      embtest.exe!srv_start(bool create_new_db)

 10     embtest.exe!innodb_init(void * p)

 11     embtest.exe!ha_initialize_handlerton(st_plugin_int * plugin)

 12     embtest.exe!plugin_initialize(st_mem_root * tmp_root, st_plugin_int * plugin, int * argc, char * * argv, bool options_only)

 13     embtest.exe!plugin_init(int * argc, char * * argv, int flags)

 14     embtest.exe!init_server_components()

 15     embtest.exe!init_embedded_server(int argc, char * * argv, char * * groups)

 16     embtest.exe!mysql_server_init(int argc, char * * argv, char * * groups)

 17     embtest.exe!main()

[MDEV-24071] embedded server crashes when mysql_library_init called after mysql_library_end Created: 2020-10-30 Updated: 2023-04-27
Status:	Open
Project:	MariaDB Server
Component/s:	Embedded Server
Affects Version/s:	10.4.15
Fix Version/s:	10.4, 10.5, 10.6

[MDEV-24071] embedded server crashes when mysql_library_init called after mysql_library_end Created: 2020-10-30 Updated: 2023-04-27