Details
-
Task
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
None
-
None
Description
Add a cache for S3 connections to speed up S3 discovery
Currently discovery, as part of CREATE TABLE or accessing information schema) creates a new connection to S3. This can be quite slow in some circumstances ( > 1 second).
Each open S3 table has it's own dedicated S3 connection so this improvement will not affect access to already opened S3 tables.
To speed this up we should add a cache for S3 connections in the S3 engine.
One thing to consider is when to clear/flush the cache. It's probably not a big problem keeping the connection open 'forever', but it would be nice if there would be a way to flush the cache to reclaim memory and file descriptors. (FLUSH CACHES which would also call all storage engines to free their 'extra' caches' ?)
Attachments
Issue Links
- is part of
-
MDEV-17841 S3 Storage engine
- Closed