[MDEV-19965] Add cache for S3 connections Created: 2019-07-05  Updated: 2021-06-09

Status: Open
Project: MariaDB Server
Component/s: Storage Engine - S3
Fix Version/s: None

Type: Task Priority: Major
Reporter: Michael Widenius Assignee: Michael Widenius
Resolution: Unresolved Votes: 1
Labels: None

Issue Links:
PartOf
is part of MDEV-17841 S3 Storage engine Closed

 Description   

Add a cache for S3 connections to speed up S3 discovery

Currently discovery, as part of CREATE TABLE or accessing information schema) creates a new connection to S3. This can be quite slow in some circumstances ( > 1 second).

Each open S3 table has it's own dedicated S3 connection so this improvement will not affect access to already opened S3 tables.

To speed this up we should add a cache for S3 connections in the S3 engine.

One thing to consider is when to clear/flush the cache. It's probably not a big problem keeping the connection open 'forever', but it would be nice if there would be a way to flush the cache to reclaim memory and file descriptors. (FLUSH CACHES which would also call all storage engines to free their 'extra' caches' ?)


Generated at Thu Feb 08 08:55:42 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.