|
Add a cache for S3 connections to speed up S3 discovery
Currently discovery, as part of CREATE TABLE or accessing information schema) creates a new connection to S3. This can be quite slow in some circumstances ( > 1 second).
Each open S3 table has it's own dedicated S3 connection so this improvement will not affect access to already opened S3 tables.
To speed this up we should add a cache for S3 connections in the S3 engine.
One thing to consider is when to clear/flush the cache. It's probably not a big problem keeping the connection open 'forever', but it would be nice if there would be a way to flush the cache to reclaim memory and file descriptors. (FLUSH CACHES which would also call all storage engines to free their 'extra' caches' ?)
|