Parallel Query: Study how to create worker threads

What is the way to create another worker thread from the SQL layer?

• Initially we should use a server variable that will control how many workers to create
• We should request workers from the thread pool (right?)
  • And return them to the thread pool when they are done.

Code a basic function that will

• request N workers
• Let these workers do something
• Then return control to the main thread.
• The main thread should wait for workers to finish (and start processing their result as soon as they are done).

Followups to this task:

• workers should be intialized to use MariaDB internal environment (mysys, etc)

TODO - first things

• Study the threadpool code to answer the above questions: (AI may also help with this)
  • How to get a worker thread
  • How to return a thread

• How does one wait for multiple worker threads at the same time and handle them as soon as they terminate, provided they can do it in an arbitrary order?

MS1: Thread creation demo

• Introduce a variable parallel_worker_threads.
• Add "parallel_init() and and parallel_cleanup()" which will check if (parallel_worker_threads>1 (or 0?) ) and create/wait for parallel threads to finish.
  • The threads will do nothing for now.
• Add parallel_test() which is invoked at query execution somewhere (first take: in JOIN::exec(select_lex->select_id==1)). It will call parallel_init() , parallel_cleanup().

At the moment, it looks like we could follow the approach of INSERT DELAYED in creating/initializing worker threads:

• delayed_get_table() calls mysql_thread_create() to create a thread
• handle_delayed_insert() is the thread function, it does needed in-thread initialization. (Q: does it also inform the thread pool that there is now one more thread running?)

TODO: ask wlad perhaps if we should still use the thread pool?

MS2: Shipping of something from workers to the main thread

Query execution may produce warnings (or errors).
Should we try shipping them as the first thing we pass between the threads.

The idea is to have a basic demo where,

• The worker threads produce warnings
• The main thread is waiting for their progress. In the future, it should be waiting for any kind of progress from any of the threads and it should start processing the input as soon as any thread provides. For now, it will only accept warnings and emit them to the client.

MS3: Visibility in I_S and P_S.

Are the created threads visible in INFORMATION_SCHEMA.PROCESSLIST (likely not) or performance_schema.threads (could be) ?

What should be done to make them visible in both?