Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-11111

Redesign how embedded server is built and how it works

Details

    • Task
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      Currently, building embedded server requires special compilation. The EMBEDDED_SERVER preprocessor constant is quite intrusive, in that it affects the core structures of the server, includng but not limited to THD. Thus, almost the whole server (with very few exceptions) needs to be recompiled for embedded. Since developers prefer faster compilation and not likely to use embedded in the tests, the result is that embedded also does not have enough coverage.
      Also, it is unclear and problematic how embedded should be integrated with Connector C, which is since 10.2 the client we use (so far partially, not in embedded or server itself)

      What we can do instead is following: instead of special compilation for the server, make client start bona-fide server in a background thread (thread start function would be mysqld_main()).

      Even if client and server run in the same process, they still could still use IPC for communication such as "private" Unix domain socket with proper permission and unique name in the datadir, or Windows pipe .

      A server might want to know if it runs as embedded, and client has to know it as well,so both client and server could have boolean variables (is_client_embedded, is_server_embedded), set to true in the embedded library and false otherwise (this is a substitute for preprocessor constant EMBEDDED_SERVER).

      In addition, server or its plugins might want to communicate with other MySQL servers (federated, connect, replication, spider). Connector/C could export special flag MYSQL_OPT_NOT_EMBEDDED for mysql_real_connect() that would prevent internal connection even inside embedded library. Alternatively, it could export mysql_real_connect_notembedded() function for that purpose.

      Attachments

        Issue Links

          Activity

            The goal of implementing embedded the way it is, was to avoid copying data between the client and the server as much as possible. The server part doesn't "send" data to the client part, it simply hands the buffer over.

            Questions:

            • can embedded be simplified, while still preserving this property?
            • is this property something users care about? In "personal phone book" applications it certainly doesn't matter.
            serg Sergei Golubchik added a comment - The goal of implementing embedded the way it is, was to avoid copying data between the client and the server as much as possible. The server part doesn't "send" data to the client part, it simply hands the buffer over. Questions: can embedded be simplified, while still preserving this property? is this property something users care about? In "personal phone book" applications it certainly doesn't matter.
            wlad Vladislav Vaintroub added a comment - - edited
            • I do not know the initial thoughts behind the embedded library, but at this stage, I'm almost sure that in case it is used, it would be done because of simpler packaging. I think, users do not care about zero copy.
            • I feel like there a merit in having an API (server service) to execute queries from inside the server. I do not think this API needs to resemble MYSQL C API . just the ability to execute queries in plugins would be great. I'm not sure what it has to do with embedded, gut feeling is that once we have that , along with an API to start and shutdown server, this would enable a much better embedded (the one that would not need an almost 100% recompile).
            wlad Vladislav Vaintroub added a comment - - edited I do not know the initial thoughts behind the embedded library, but at this stage, I'm almost sure that in case it is used, it would be done because of simpler packaging. I think, users do not care about zero copy. I feel like there a merit in having an API (server service) to execute queries from inside the server. I do not think this API needs to resemble MYSQL C API . just the ability to execute queries in plugins would be great. I'm not sure what it has to do with embedded, gut feeling is that once we have that , along with an API to start and shutdown server, this would enable a much better embedded (the one that would not need an almost 100% recompile).
            • quite possibly. I cannot think of a really high-performance applications using libmysqld
            • it's kind of a holy grail. lots of talks for lots for years. in MariaDB it's MDEV-4641.
            serg Sergei Golubchik added a comment - quite possibly. I cannot think of a really high-performance applications using libmysqld it's kind of a holy grail. lots of talks for lots for years. in MariaDB it's MDEV-4641 .

            People

              wlad Vladislav Vaintroub
              wlad Vladislav Vaintroub
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.