Put more content into the spec (not finished still).

tbleveldb storage engine compiles succesfully, but crashes when one tries to create a table.

It gets a SIGSEGV in leveldb::DBImpl::NewDB, the line

new_db.SetComparatorName(user_comparator()->Name());

has user_comparator() == NULL.

More updates to https://kb.askmonty.org/en/leveldb-storage-engine/, it's shaping up

More updates to the spec. Will be ready for another call early next week.

Discussion with Sergei about possible needed changes in SE API. He has pointed out that TokuDB has similar properties (e.g. INSERT works faster if it does not have to check whether it is inserting a duplicate)

Got the basic do-nothing storage engine to work. The tree is on launchpad, at lp:~maria-captains/maria/mysql-5.6-leveldb

I've hit an interesting problem when writing key comparator function.

According to the design, the key is "dbname.table_name.$primary_key_value", and $primary_key_value is encoded in MySQL's KeyTupleFormat. It is easy to compare values in KeyTupleFormat when one has the TABLE object for the table in question. But we do not necessarily have it. It is possible that LevelDB invokes the comparator when the table has not been open yet (for example, compaction process may do so).

To be exact, we don't need TABLE object, we need Field_xxx objects that describe the indexed columns. Field objects have int Field::key_cmp(const uchar *a,const uchar *b) function that performs comparisons for all types.

Possible solutions are:
S1. Re-implement key comparison functions for all types. We will need to save descriptions of all primary/secondary indexes (charsets/etc). It will take a lot of code to handle all of MySQL types.
S2. Force the server to open each LevelDB table on startup, so that we have a TABLE_SHARE object for each table.
S3. Store just enough information so that we're able to create the necessary Field objects ourselves.

InnoDB implements S1, for the most part. It stores internally key column attributes like field length, null-ability, whether it is unsigned, etc. It still relies on MySQL to handle some of the types.

I got an idea about S3 when looking at make_field() in sql/field.cc, the one with signature like this:

Field *make_field(TABLE_SHARE *share, uchar *ptr, uint32 field_length,
uchar *null_pos, uchar null_bit,
uint pack_flag, ...

If LeveDB SE stored all of the attributes of the function, it would be able to create Field* objects of its own.

Discussed the problem with SergeiG. Results :

There is also S4 /* not listed above but implied */ Convert keys into something that can be compared with memcmp(). The problem: index-only scans will not be possible, there is strxfrm() is available for strings, but not for int/double/decimal/etc.

S3 can be improved: MyISAM/Aria use ha_key_cmp() to compare key values. That function either compares key tuples directly, or it uses the result of _mi_pack_key(). Using that is better than messing with TABLE_SHARE and Field objects.

Was implementing S3. Finally came to the following cset (just pushed to launchpad):

MDEV-3841: LevelDB storage engine

• Correct comparison of rowkeys (aka PRIMARY KEYS) depending on their types:
  = use MyISAM's ha_key_cmp() function for comparison
  = borrow MyISAM's code that constructs HA_KEYDEF descriptors
  = Use a separate LevelDB instance to store the data dictionary (it turned out
  that we can't store data dictionary in the same instance: we get chicken-and-egg
  problem when we need to run recovery on startup)
• Make INSERT work (for now, it will blindly overwrite data that's already there)
• Make full table scan work
• Make primary key lookups work for ref access.

Blob columns are not supported yet.

Investigated how storage engine should handle blob columns.

• InnoDB has some useful comments in row_mysql_store_blob_ref(), row_mysql_read_blob_ref().
• calc_pack_length() in field.cc shows how many bytes each blob type is using for length.

When blob data is passed down into handler::write_row(), the blob is alloced and owned by SQL layer. Callee copies the data, but doesn't free.
When the data is returned from handler::rnd_next() (or some other function that reads record), blob memory is alloced and owned by the storage engine. In InnoDB, it remains valid until another record read call is made (check out prebuil->blob_heap)

Got blob handling to work (at least, basic tests pass).

Got UPDATE and DELETE commands to work for simple examples. (They don't work for complex examples because right now all changes are applied immediately)

Hi Sergei, how's going so far?

I'm still working on implementing row-locking system and hooking it to ha_leveldb. (More details in the email)

Hi Sergei,

In which version of MariaDB, LevelDB storage engine will be released?
What is the current status?

Hi Aris,

Current status is that a tree (based on mysql-5.6) with the engine is available. The engine has a number of known limitations that make it difficult for it to compete against a more feature-complete engine like InnoDB. Right now, I cannot say which version of MariaDB the engine will be pushed to.

Hi Sergei
i am a Chineses database developer，i am looking for which version of mysql running storage engines Leveldb.Can you tell me the specific address？Thankyou Looking forward to your reply。

Hi Vessel, are you sure you need LevelDB storage engine? It was a prototype which didn't reach the production stage. It was a basis for RocksDB storage engine, which is a part of MariaDB Server starting from version 10.2.

Hi,Sergei
Yes,i am sure i need LevelDB storage engine.Because my lab developed a new database flashkv on the basis of leveldb ,now i want to load flashkv into mysql as a storage engine. Since the upper layer of leveldb and flashkv is the same ,flashkv can be directly introduced if there is the interface of leveldb.So i am sure i need Leveldb Storage engine.Hope you could help me.Thanks,looking forward to your reply.

Ok, I hope I'm not mixing anything up, The latest tree that I can find with LevelDB code is this one: https://code.launchpad.net/~maria-captains/maria/mysql-5.6-leveldb

Vessel,

Please keep in mind that MariaDB doesn't support this engine, branch or code to any level, there will be no bugfixes, development, consultation or guidance in regard to it.

Hi Sergei
Thank you,You've been a big help

Hi Elena
Okay,thanks for your reply.