[MDEV-6712] THD specifics for plugins Created: 2014-09-09  Updated: 2015-03-02  Resolved: 2014-12-04

Status: Closed
Project: MariaDB Server
Component/s: Plugins
Fix Version/s: 10.1.2

Type: Task Priority: Major
Reporter: Jonas Oreland Assignee: Sergei Golubchik
Resolution: Fixed Votes: 0
Labels: plugins

Attachments: File thd_specifics.patch     File thd_specifics.path     File thd_specifics.v2.patch    

 Description   

This patch adds generic infrastructure for plugins to store
things on a THD object (rather that polluting THD object with
new member variables)

The feature is modeled after pthread specifics hence
the plugin shall at startup create a key, that can later be used
by multiple thread in parallel wo/ locks (if access is made to different
THD objects). The creation/destruction of a key is though mutex protected.
The feature is similar to ha_data, but exists for any type of plugin.



 Comments   
Comment by Jonas Oreland [ 2014-09-09 ]

this is a contribution

Comment by Sergei Golubchik [ 2014-10-12 ]

An alternative implementation: https://code.launchpad.net/~atcurtis/maria/10.0-opaque-thdvar/+merge/214139

The implementation is very simple. And it's declarative, follows the spirit of the MariaDB/MySQL Plugin API. But if feels like a hack, it uses thdvars for something they weren't supposed to be.

Comment by Sergei Golubchik [ 2014-10-20 ]

Furthermore, it looks like neither implementation is really needed. One can simply declare, for example,

MYSQL_THDVAR_INT(my_tls, PLUGIN_VAR_NOCMDARG | PLUGIN_VAR_NOSYSVAR,
  0, 0, 0, 0, 0, 0, 0);

It will create a THD-local variable of the type int, that can later be used as, say

  if (THDVAR(thd, my_tls) > 10)
    THDVAR(thd, my_tls)=5;

For pointers one can use MYSQL_THDVAR_STR. With PLUGIN_VAR_MEMALLOC flag it will even free the memory when a connection is terminated, which is more than both contributed patches do.

What I would see useful, though — to be able to attach a custom destructor with the such a THD-local variable. Similar to

       int pthread_key_create(pthread_key_t *key, void (*destructor)(void*));

But as neither contributed patch offers this functionality, I suppose it is not practically needed.

Comment by Jonas Oreland [ 2014-10-21 ]

Hi serg,

1) questions regarding MYSQL_THDVAR_INT(my_tls, PLUGIN_VAR_NOCMDARG | PLUGIN_VAR_NOSYSVAR)

  • when will the "slot" for this be allocated (e.g can I allocate it only if some condition is met??)
  • when will the storage for the variable be allocated
  • how it the THDVAR(thd, my_tls) implemented ? (hash or array)
  • is THDVAR(thd, my_tls) mutex encumbered

2) I can add the destructor function if you like

Comment by Sergei Golubchik [ 2014-10-21 ]

2) yes, I know you can, that was not the point. I merely observed that both in your and Antony patches (and supposedly, you needed this feature for some practical plugin of yours) there are no destructors. Which means they are not as important as I believed they are.

1)

  • no, the slot is allocated when a plugin is loaded. this follows the spirit of the Plugin API, which is declarative where possible, and the work is done by the server. Less bolier-plate code.
  • when this variable or any variable with a larger slot number is accessed for the first time. This includes user-visible session plugin variables, like @@innodb_lock_wait_timeout, they also have slot numbers in the same namespace.
  • array. It's basically thd->variables.dynamic_variables_ptr + my_tls->offset and a cast to the correct type.
  • there's a mutex when a memory is allocated (on the first access, unless a variable with a larger slot number was already accessed), then no.

Thinking about it... It is possible to make mutex locks less likely, by allocating the memory when THD is created. But in case a new plugin is INSTALL-ed, all existing THD's might still need to allocate more memory, and that would need a mutex.

Comment by Jonas Oreland [ 2014-10-21 ]

Hi again,

I think the functionality is fine then.
a mutex when memory is allocated is good enough.

though, I think the thd_key_create,thd_

{get,set}

_specific,thd_key_delete
API is nice (in fact nicer than macro inflated THDVAR...which is a personal
preference)
shall I port it to be on top of already
existing thd->variables.dynamic_variables_ptr ?

if, yes, do you have a place where i can start looking ?

/Jonas

On Tue, Oct 21, 2014 at 9:29 AM, Sergei Golubchik (JIRA) <

Comment by Sergei Golubchik [ 2014-10-21 ]

adding thd_key_create() and thd_key_delete() might be tricky, because normally slot allocation happens when a plugin is loaded and the server sets my_tls->offset variable. But set/get are easy, and you can also create nicer macros if you'd like. Think of that functionality not as of THD version of pthread_key_* functions, but as of THD portable version of gcc's __thread keyword. Like in (see gcc manual)

__thread int i;

This is also declarative and also one can create variables of different types. And also one cannot do it conditionally. For all 0's in the MYSQL_THDVAR_xxx macro definition, they only make sense for user-visible variables, for hidden ones you can do

#define TLS_INT(x) MYSQL_THDVAR_INT(x, PLUGIN_VAR_NOCMDARG | PLUGIN_VAR_NOSYSVAR, 0, 0, 0, 0, 0, 0, 0)

and then declare variables with

TLS_INT(x);

if you'd like thd_key_get() and thd_key_set(), you can do

#define thd_key_get(T,V)    THDVAR(T,V)
#define thd_key_set(T,V,S)  THDVAR(T,V)=(S)

but if you prefer functions, not macros, there has to be one function per type, like:

typedef DECLARE_MYSQL_THDVAR_SIMPLE(int, int);
int inline thd_key_get_int(THD *thd, MYSQL_SYSVAR_NAME(int) *var)
{
  return *var->resolve(thd, var->offset);
}

where the first line creates a named type, so that it could be used in the function definition.

Comment by Jonas Oreland [ 2014-10-22 ]

made a new version on top of THDVAR functionality.

Comment by Jonas Oreland [ 2014-10-22 ]

btw, a reason for insisting on using a non THDVAR based interface.

  • some (most?) parts of mysqld are not easily pluggable
  • hence we write non-plugin code.
  • that code might want to share per-thd information to things that are implemented as plugins.

an easy way of doing this to use the thd_key functionality, sharing only the key between the different modules (and obviously the data-definition)
the most common way would be to add member variables to the THD...

Maybe the THDVAR macros might be used outside of plugins too...but i'm not sure,
and I'm not a big macro fan.

Comment by Sergei Golubchik [ 2014-11-28 ]

jonaso, I've just noticed that you didn't specify a license under which this contribution is available to us.
You can either say here that we can use it under new (also called "3-clause") BSD license or submit a signed MCA.

Comment by Jonas Oreland [ 2014-11-28 ]

you can use it under new (also called "3-clause") BSD license

/Jonas

On Fri, Nov 28, 2014 at 1:43 PM, Sergei Golubchik (JIRA) <

Comment by Sergei Golubchik [ 2014-11-29 ]

http://lists.askmonty.org/pipermail/commits/2014-November/007093.html

Comment by Jonas Oreland [ 2015-03-02 ]

hi serg,

what do you think of current patch ?

the problem is that we use the thd variables from our vio backend,
and if the thd variables is cleared before vio_close there is a memory leak.

/Jonas

Generated at Thu Feb 08 07:14:01 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.