[CONPY-252] No proper way to close result set from server side cursor select that doesn't hang the program. Created: 2023-02-23 Updated: 2023-11-06 |
|
| Status: | Needs Feedback |
| Project: | MariaDB Connector/Python |
| Component/s: | Other |
| Affects Version/s: | 1.1.5post1 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Minor |
| Reporter: | Michael Conrad | Assignee: | Georg Richter |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Environment: |
|
||
| Python Version: | 3.10.9 |
| Description |
|
I'm running a query on millions of records and need to use server side cursors. I'm able to get streaming working, but when I close the result set either via exiting the context manager block or via an explicit #close() everything hangs while it waits for the driver to pull in and discard the remaining data associated with the server side cursor. The mariadb client code indicates that a #close() on a cursor is supposed to: If the cursor has pending or unread results, .close() will cancel them The cursor will be unusable from this point forward; an Error
if not self.connection._closed: Shouldn't the #close() on a result object stop the steaming of data from the server instead of causing a potentially many hour or longer hang waiting on the result set to be streamed over the network then discarded? Is this incomplete or incorrect documentation? I did find in the pymysql lib where it explicitly exhausts the cursor on close with a comment that there isn't a way to stop the stream of data. |
| Comments |
| Comment by Georg Richter [ 2023-02-24 ] | ||||||
|
Hi, could you please provide a sample program which demonstate this behavior? When using a server side cursor it shouldn't block (see https://github.com/mariadb-corporation/mariadb-connector-python/blob/1.1/testing/test/integration/test_cursor.py#L395) | ||||||
| Comment by Michael Conrad [ 2023-02-24 ] | ||||||
|
What do you mean by "blocking"? And how would I demonstrate the #close() hanging in a test? Timing? | ||||||
| Comment by Georg Richter [ 2023-02-25 ] | ||||||
|
The sample program should contain some information: e.g. How was the cursor created, what was executed, were data fetched etc. About blocking/unbuffered cursors:
|