[CONPY-157] Python GIL deadlock on server connecting Created: 2021-07-31  Updated: 2021-07-31  Resolved: 2021-07-31

Status: Closed
Project: MariaDB Connector/Python
Component/s: Other
Affects Version/s: 1.0.7
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Zhai Zhaoxuan Assignee: Georg Richter
Resolution: Not a Bug Votes: 0
Labels: None
Environment:

OS: Archlinux
Python: 3.9.6


 Description   

When the user code calls mariadb.connect to connect to a database, the mariadb package calls a C function MrdbConnection_connect.

Before MrdbConnection_connect returns, the thread hold Python GIL. It means the whole python process is locked, and unable to continue.

If the user run a ssh tunnel using python in same process and the user tries to proxy the mariadb connection using this tunnel, a deadlock will happen.

The MrdbConnection_connect requires the ssh tunnel to send its data to real server. But it does not give the GIL. So ssh tunnel can never run and send the data to real server, because it need take the GIL to eval code.

So is it possible to realease GIL before calling C function? This will resolve the deadlock.

 Comments   
Comment by Georg Richter [ 2021-07-31 ]

By default the GIL is locked when calling a c module from Python to prevent raise conditions and to guarantee thread safety.

That means a c extension can only release the GIL when not interacting with Python API, which happens when Connector/Python is executing api functions:

static int
 
MrdbConnection_Initialize(MrdbConnection *self,
 
        PyObject *args,
 
        PyObject *dsnargs)
 
{
 
    /* parse parameters via Python API */
 
    ....
 
    Py_BEGIN_ALLOW_THREADS;
 
    mysql_real_connect(self->mysql, host, user, password, schema, port,
 
            socket, client_flags);
 
    Py_END_ALLOW_THREADS;
 
  
}

Generated at Thu Feb 08 03:30:40 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.