https://github.com/MariaDB/server/pull/12

From November 3: I have now finally completed the second round of review. All other seems correct to me expect:

There is still requirement that system has either openSSL or YaSSL, in my understanding MariaDB source tree should compile without both of these based on
https://mariadb.com/kb/en/mariadb/documentation/getting-started/compiling-mariadb-from-source/Build_Environment_Setup_for_Linux/

If I do the merge, I will merge the actual page encryption implementation to also XtraDB.

Hi,

I have problems with unittest/eperi/pageenc-t test:

./pageenc-t
ok 1 - secret can be decrypted
ok 2 - secret can be decrypted
ok 3 - secret can be decrypted
ok 4 - secret can be decrypted
ok 5 - secret can be decrypted
ok 6 - secret can be decrypted
ok 7 - page type 8 or 9 will not be encrypted! file File row_format_compressedaa
ok 8 - File row_format_compressedab
ok 9 - File row_format_compressedac
ok 10 - File row_format_compressedad
ok 11 - page type 8 or 9 will not be encrypted! file File row_format_dynamicaa
ok 12 - File row_format_dynamicab
ok 13 - File row_format_dynamicac
ok 14 - File row_format_dynamicad
ok 15 - page type 8 or 9 will not be encrypted! file File row_format_redundantaa
ok 16 - File row_format_redundantab
ok 17 - File row_format_redundantac
ok 18 - File row_format_redundantad
ok 19 - page type 8 or 9 will not be encrypted! file File row_format_compactaa
ok 20 - File row_format_compactab
ok 21 - File row_format_compactac
ok 22 - File row_format_compactad
ok 23 - File compressed
ok 24 - File compressed_full
ok 25 - File compressed_6bytes_av
ok 26 - page type 8 or 9 will not be encrypted! file File xaa
ok 27 - File xab
ok 28 - File xac
ok 29 - File xad
ok 30 - File xae
ok 31 - File xaf
InnoDB: Corruption: Page is marked as encrypted
InnoDB: but decryption verification failed with error 1, encryption key 255.
ok 32 - Detect decryption error in xab encryption result 1
Warning: 512 bytes lost at 0x7f2f71443760, allocated by T@0 at addr2line: option requires an argument – 'e'
Usage: addr2line [option(s)] [addr(s)]
Convert addresses into line number/file name pairs.
If no addresses are specified on the command line, they will be read from stdin
The options are:
@<file> Read options from <file>
-a --addresses Show addresses
-b --target=<bfdname> Set the binary file format
-e --exe=<executable> Set the input file name (default is a.out)
-i --inlines Unwind inlined functions
-j --section=<name> Read section-relative offsets instead of addresses
-p --pretty-print Make the output easier to read for humans
-s --basenames Strip directory names
-f --functions Show function names
-C --demangle[=style] Demangle function names
-h --help Display this information
-v --version Display the program's version

addr2line: supported targets: elf64-x86-64 elf32-i386 elf32-x86-64 a.out-i386-linux pei-i386 pei-x86-64 elf64-l1om elf64-k1om elf64-little elf64-big elf32-little elf32-big pe-x86-64 pe-i386 plugin srec symbolsrec verilog tekhex binary ihex
0x7f2f6e4caf2a,

Is this something critical ? On the other hand, I might not even include these tests as we normally use mtr tests to test features.

Hi,

I would also need help on following valgrind warnings:

==15701== 192 bytes in 8 blocks are definitely lost in loss record 3 of 6
==15701== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==15701== by 0xDE8E2D: EncKeys::parseLine(char const*, unsigned long) (EncKeys.cc:316)
==15701== by 0xDE8934: EncKeys::parseFile(char const*, unsigned long, char const*) (EncKeys.cc:248)
==15701== by 0xDE8539: EncKeys::initKeysThroughFile(char const*, char const*, char const*) (EncKeys.cc:162)
==15701== by 0xDE82C8: EncKeys::initKeys(char const*, char const*, int, char const*) (EncKeys.cc:125)
==15701== by 0xD86BEF: KeySingleton::getInstance(char const*, char const*, int, char const*) (KeySingleton.cc:48)
==15701== by 0xB3B02F: innobase_init(void*) (ha_innodb.cc:3459)
==15701== by 0x80DD0B: ha_initialize_handlerton(st_plugin_int*) (handler.cc:523)
==15701== by 0x6054BF: plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool) (sql_plugin.cc:1359)
==15701== by 0x605D44: plugin_init(int*, char**, int) (sql_plugin.cc:1579)
==15701== by 0x52BC3E: init_server_components() (mysqld.cc:5025)
==15701== by 0x52CD0A: mysqld_main(int, char**) (mysqld.cc:5609)
==15701== by 0x522B74: main (main.cc:25)
==15701== 196 bytes in 1 blocks are still reachable in loss record 4 of 6
==15701== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==15701== by 0xE4F00A: sf_malloc (safemalloc.c:115)
==15701== by 0xE3CDEB: my_malloc (my_malloc.c:100)
==15701== by 0x5292BF: my_str_malloc_mysqld (mysqld.cc:3589)
==15701== by 0xEA3C83: pcre_compile2 (pcre_compile.c:9113)
==15701== by 0xEA2F63: pcre_compile (pcre_compile.c:8737)
==15701== by 0xDE9399: EncKeys::isComment(char const*) (EncKeys.cc:404)
==15701== by 0xDE8C08: EncKeys::parseLine(char const*, unsigned long) (EncKeys.cc:287)
==15701== by 0xDE8934: EncKeys::parseFile(char const*, unsigned long, char const*) (EncKeys.cc:248)
==15701== by 0xDE8539: EncKeys::initKeysThroughFile(char const*, char const*, char const*) (EncKeys.cc:162)
==15701== by 0xDE82C8: EncKeys::initKeys(char const*, char const*, int, char const*) (EncKeys.cc:125)
==15701== by 0xD86BEF: KeySingleton::getInstance(char const*, char const*, int, char const*) (KeySingleton.cc:48)
==15701== by 0xB3B02F: innobase_init(void*) (ha_innodb.cc:3459)
==15701== by 0x80DD0B: ha_initialize_handlerton(st_plugin_int*) (handler.cc:523)
==15701== by 0x6054BF: plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool) (sql_plugin.cc:1359)
==15701== by 0x605D44: plugin_init(int*, char**, int) (sql_plugin.cc:1579)
==15701== 412 bytes in 1 blocks are still reachable in loss record 6 of 6
==15701== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==15701== by 0xE4F00A: sf_malloc (safemalloc.c:115)
==15701== by 0xE3CDEB: my_malloc (my_malloc.c:100)
==15701== by 0x5292BF: my_str_malloc_mysqld (mysqld.cc:3589)
==15701== by 0xEA3C83: pcre_compile2 (pcre_compile.c:9113)
==15701== by 0xEA2F63: pcre_compile (pcre_compile.c:8737)
==15701== by 0xDE8C74: EncKeys::parseLine(char const*, unsigned long) (EncKeys.cc:294)
==15701== by 0xDE8934: EncKeys::parseFile(char const*, unsigned long, char const*) (EncKeys.cc:248)
==15701== by 0xDE8539: EncKeys::initKeysThroughFile(char const*, char const*, char const*) (EncKeys.cc:162)
==15701== by 0xDE82C8: EncKeys::initKeys(char const*, char const*, int, char const*) (EncKeys.cc:125)
==15701== by 0xD86BEF: KeySingleton::getInstance(char const*, char const*, int, char const*) (KeySingleton.cc:48)
==15701== by 0xB3B02F: innobase_init(void*) (ha_innodb.cc:3459)
==15701== by 0x80DD0B: ha_initialize_handlerton(st_plugin_int*) (handler.cc:523)
==15701== by 0x6054BF: plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool) (sql_plugin.cc:1359)
==15701== by 0x605D44: plugin_init(int*, char**, int) (sql_plugin.cc:1579)
==15701== by 0x52BC3E: init_server_components() (mysqld.cc:5025)

Hi,

Unfortunately, page encryption does not work correctly with page compression. After shutdown, all tables refuse to decrypt. I add the current diff (containing my test cases) to this bug.

Patch to MariaDB 10.1

We have fixed memory leaks in EncKeys.cc. It's now pushed to https://github.com/MariaDB/server/pull/12.

Concerning page compression,
We tested with page compression some weeks ago and it worked.
That was with Page_compressed=1 as table creation parameter.
In the later maria db trunk, it seemed, that page compression could not be enabled anymore that way.
According to debugger the innodb_compression_algorithm parameter was always 0, which means no compression. That means, it could be enabled, but it had no effect.
We changed the source, so that innodb_compression_algorithm defaulted to 1.
With this change, it worked like that
create table t(..) page_compressed=1 page_encryption=1
This worked also after restarting the maria db server.

But since we could not really enable page compression in another way than manipulating the source code we did not test it in combination with page encryption for some weeks apart from unit test.

Can you explain how to enable page compression?

SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
— zlib, this is needed, default = 0
set global innodb_compression_algorithm = 1;
create table innodb_compact(c1 bigint not null, b char(200)) engine=innodb row_format=compact page_encryption=1 page_encryption_key=1 page_compressed=1;

thank you.
We inspect it...

There is one bug on shutdown that I fixed:

innobase_end(

/*=========*/

	handlerton*		hton,	/*!< in/out: InnoDB handlerton */

	ha_panic_function	type __attribute__((unused)))

					/*!< in: ha_panic() parameter */

{

	int	err= 0;

	DBUG_ENTER("innobase_end");

	DBUG_ASSERT(hton == innodb_hton_ptr);

	if (innodb_inited) {

		THD *thd= current_thd;

		if (thd) { // may be UNINSTALL PLUGIN statement

		 	trx_t* trx = thd_to_trx(thd);

		 	if (trx) {

		 		trx_free_for_mysql(trx);

		 	}

		}

		srv_fast_shutdown = (ulint) innobase_fast_shutdown;

		innodb_inited = 0;

		hash_table_free(innobase_open_tables);

		innobase_open_tables = NULL;

		if (innobase_shutdown_for_mysql() != DB_SUCCESS) {

			err = 1;

		}

		srv_free_paths_and_sizes();

		my_free(internal_innobase_data_file_path);

		mysql_mutex_destroy(&innobase_share_mutex);

		mysql_mutex_destroy(&commit_cond_m);

		mysql_cond_destroy(&commit_cond);

		mysql_mutex_destroy(&pending_checkpoint_mutex);

	}

        /* This should be after innobase_shutdown_for_mysql() */

	KeySingleton::getInstance().~KeySingleton();

	DBUG_RETURN(err);

}

Coud it be the fact that on page compressed tables only FIL_PAGE_DATA + FIL_PAGE_COMPRESSED_SIZE + actual_size = mach_read_from_2(buf+FIL_PAGE_DATA) is defined, rest of page is "random".

Memory leaks are now fixed.

page compressed table's page is encrypted completely apart from the header.
This looks like random data after the actual size position.
But decrypting the page should restore the original bytes.
We can reproduce the bug and try to find the reason..

Idea with page compression is that we can decrease the actual write and rest of the page is trimmed (posix_fallocate(punch_hole|keep_size)). Write is always done on sector boundary (>=512,...,64K). Empty space on that page is not initialized. I just wonder about the order of operations:

(a) compress, encrypt , in this case you may not encrypt whole page, because we do not write all of it to storage

(b) encrypt, compress, in this case you can encrypt whole page

Case (a) could be better for compression efficiency, thus I would prefer that.

(a) compress, encrypt is the order

The bug is really that the page size is not detected correctly.
we receive 4 k to encrypt and we receive 16k to decrypt.
actual size is for example 125, but there are some bytes set after the actual size position, e.g. in page 0.
000006b0 00 00 00 00 00 00 00 00 7c 59 39 ab 1d 4d 7a 68 |........|Y9..Mzh|
at position 6b0 + 8. What is that?

I will change it - for page compressed case, so that I store the real size (512...64k) of the page to encrypt and use this size in decryption.
This should work.

Are there other case, where the page size changes between write and read operations?

Let me give a example. Lets assume that page (16k) is compressed to 10K (including FIL_HEADER), now rest of the page 6K is not initialized it is "random data", we modify actual write size to 12K (to abey 512 sector size), again data 10K-12K is not zeroed. Encryption can encrypt only that 10K not more. For decrypt, you need to somehow find out from the page, how much actual data it has to decrypt.

Page size does not change between write and read, the amount of data we physically write does at the moment only on page compressed tables. It is possible to configure you database to use different page size (!16K) but that is static.

I will leave the 1st two bytes from page compressed page payload (after 38 byte header) unencrypted and derive an adequate size for encryption/decryption from these.
As I understand this is the real size of the data and everything behind is not (security) relevant.

I agree with this aproach, there is no security relevant data on FIL_HEADER+2 and after the actual payload.

I pushed it to
https://github.com/MariaDB/server/pull/12

I changed it the following way:
the incoming data (for write operation) is encrypted and the size of the data is stored. Since it is a power of 2 only the log base 2 number is stored.
Decryption can determine the size of the encrypted data.
I think it's Ok, because the page compression code is executed before encryption and determines the size of the written data already.
Advantage: the page encryption code can be used nearly unchanged for page compressed pages.
Disadvantage: a bit more data is encrypted than necessary. But only the bytes that are actually written and AES encryption is usually fast.
I've tested it with Linux and seems to be working.

Is page compression usable with Windows? mariadb crashes if a table is created with page_compressed enabled.
Since I cannot enable page compression in Windows I made no changes for that.
Page encryption works with windows. Some data types were changed in the last commits to support Windows 64 build.

Hi,

Page compression should work also on Windows (at least it does on our test environment), where it crashes ?

R: Jan

it crashes immediately after creating a table. This is the output after create table ab(pos int)page_compressed=1 row_format=compact;
InnoDB: Operation Windows aio to file ..\..\..\storage\xtradb\os\os0file.cc and at line 5459
141120 9:33:22 [ERROR] InnoDB: File .\test\ab.ibd: 'Windows aio' returned OS error 0. Cannot continue operation

In my source version it's the line
} else if (os_file_handle_error(slot->name, "Windows aio", _FILE, __LINE_)) {

—
Isn't it, that in os_aio_func in xtradb/../os0file.cc
after calling os_aio_array_reserve_slot and processing the #ifdef WIN_ASYNC_IO path (~ line 5317)
the number of bytes which are written with the WriteFile Function (~ line 5329) is the size of the page and not the actual size?
If I change this for the page_compressed + page_encryption case, so that only slot->len bytes are written, I can create a table and do some operation on the table, but it is not stable with larger amount of data.
Apart from that, I think the uncompressed buffer would be used for WriteFile

Received a question about generating keys.
„How you generate initialization vector ? I.e. lets assume my secred key is MySECredKey1024#!!@New, how that can be encoded any tools "

Either hex-encode a 16/24/32 byte key and a 16 byte iv of your own choice or maybe simpler,
generate a key/iv derived from the secret.

Openssl command line tool can be used for example.
Generate AES 256 key with initialization vector.
openssl enc -P -aes-256-cbc -md sha1 -k MySECredKey1024#!!@New
Note, that the ‚!‘ is escaped.
The output is
salt=F410C43FF14E24C1
key=6FF5262ED4EB548BC2EB6E6442013836EEA64E6E9148636020174209B5B963BD
iv =C0F16C0D61257502762D03899276E690

Note, that for deriving this key/iv from the password, one needs to know the password and the salt. The salt can be passed to openssl as command line argument.

I do not follow the statement "Note, that for deriving this key/iv from the password, one needs to know the password and the salt. The salt can be passed to openssl as command line argument." ? Sorry, I'm not a security expert :-<

R: Jan

"isn't it, that in os_aio_func in xtradb/../os0file.cc
after calling os_aio_array_reserve_slot and processing the #ifdef WIN_ASYNC_IO path (~ line 5317)
the number of bytes which are written with the WriteFile Function (~ line 5329) is the size of the page and not the actual size?
If I change this for the page_compressed + page_encryption case, so that only slot->len bytes are written, I can create a table and do some operation on the table, but it is not stable with larger amount of data.
Apart from that, I think the uncompressed buffer would be used for WriteFile"

I do not fully understant the statement. Idea is as follows, assume that 16K page is compressed to 10K and then we allign that to sector boundary (using 512 sector size) = 12K. We should write 12K and then call os_file_trim function and punch hole to 12-16K.

R: Jan

The "salt" is usuallly used to achieve, that same plain text results in different cipher texts.
In this case, the salt is used by openssl tool so that the same secret would result in different key/iv pairs it the command would be executed multiple times..
To generate the same keys, the salt must be known and passed to openssl or the salt feature may be turned of. I've not investigated whether or how it can be turned off. It is indeed turned off, if the salt is always the same. It can be set like that:
openssl enc -S F410C43FF14E24C1 -P -aes-256-cbc -md sha1 -k MySECredKey1024#!!@New
Then it will result in the key/iv pair from above.

Sorry my ignorance but if I understant correctly, salt is not stored on key file, so how it then can decrypt the page ?

Salt is not needed to decrypt any page, it is just used by the openssl command line tool to produce a key/iv pair derived from something human readable like a password.
For decrypting only the key/iv pair is relevant.
Salt is only a security feature to avoid, that two persons with the same password generate the same key/iv pair and is part of the usage of the openssl tool, which can be used to generate a valid aes key/iv pair.
This openssl tool is not needed, it just gives a possibilty to generate the key/iv pair derived of something human readable.
The MySECredKey1024#!!@New is not a valid aes key because these must be of a defined size.
You can alternatively edit the bytes directly (16,24, 32 bytes) and hex-encode them to generate a key and a iv (16 bytes), this means generate a key/iv pair "manually".
What if the key/iv is lost for some reason?
if you used openssl as describe above, you could reproduce the key/iv pair if you know the password AND the salt.
And for this case, the salt is needed. For nothing else. You can surely disable the salt feature in the openssl tool, may be by setting it to zero length.

Concerning windows.
If I step through the code after generating a page_compressed table, and arrive at the WriteFile function,
ret = WriteFile(file, buffer, (DWORD) n, &len,
&(slot->control));

I wonder why n is 16k here and the buffer is not compressed although the page itself was compressed earlier (it has e.g page type 5 and not 0x8632 = FIL_PAGE_PAGE_COMPRESSED). This may be correct, I'm not aware. I assume that the buffer that is used by WriteFile is finally written to disk so I'm confused. It is indeed written to disk immediately before a crash occurrs and has the wrong page type which I can see in the ibd file (in contrast to linux ibd filed, where a page has type 0x8632).
I'm using the same configuration in Windows and Linux. I'm not able to test this combination page_encryption/page_compressed in Windows.

I can confirm that current implementation does work on Linux. Now that that is achieved there is few things I do not like:

• Encrypt temporal buffers : tmp_buf = static_cast<byte *>(ut_malloc(64));
• Decrypt temporal buffers: tmp_page_buf = static_cast<byte *>(ut_malloc(len)); tmp_buf= static_cast<byte *>(ut_malloc(64)); memset(tmp_page_buf,0, len); and memset(in_buf, 0, len);

I would either like to avoid these or allocate additional memory in slot structure and pass it here, is those memsets necessary ?

There is still potential memory leak on pageenc-t test:

 jan@jan-GE70-0NC-0ND ~/mysql/10.1-bugs/unittest/eperi $ ./pageenc-t

ok 1 - secret can be decrypted

ok 2 - secret can be decrypted

ok 3 - secret can be decrypted

ok 4 - secret can be decrypted

ok 5 - secret can be decrypted

ok 6 - secret can be decrypted

ok 7 - page type 8 or 9 will not be encrypted! file File row_format_compressedaa

ok 8 - File row_format_compressedab

ok 9 - File row_format_compressedac

ok 10 - File row_format_compressedad

ok 11 - page type 8 or 9 will not be encrypted! file File row_format_dynamicaa

ok 12 - File row_format_dynamicab

ok 13 - File row_format_dynamicac

ok 14 - File row_format_dynamicad

ok 15 - page type 8 or 9 will not be encrypted! file File row_format_redundantaa

ok 16 - File row_format_redundantab

ok 17 - File row_format_redundantac

ok 18 - File row_format_redundantad

ok 19 - page type 8 or 9 will not be encrypted! file File row_format_compactaa

ok 20 - File row_format_compactab

ok 21 - File row_format_compactac

ok 22 - File row_format_compactad

ok 23 - File compressed page_compressed write size: 16384

ok 24 - File compressed

ok 25 - File compressed_full page_compressed write size: 16384

ok 26 - File compressed_full

ok 27 - File compressed_6bytes_av page_compressed write size: 16384

ok 28 - File compressed_6bytes_av

ok 29 - File compressed page_compressed write size: 4096

ok 30 - File compressed

ok 31 - page type 8 or 9 will not be encrypted! file File xaa

ok 32 - File xab

ok 33 - File xac

ok 34 - File xad

ok 35 - File xae

ok 36 - File xaf

InnoDB: Corruption: Page is marked as encrypted

InnoDB: but decryption verification failed with error 1, encryption key 255.

ok 37 - Detect decryption error in xab encryption result 1

Warning:  512 bytes lost at 0x7f5559093760, allocated by T@0 at addr2line: option requires an argument -- 'e'

Just one confirmation on encryption/decryption, it should not change the write_size value !

R: Jan

Encryption does not change the write_size.

Decryption sets slot->write_size in the os_aio_slot_t structure at one point. It is set to the page size, or, together with page compressed, to the size of the compressed page (the actual size which was calculated by the page compression function). Since decrypt code is executed before fil_decompress and fil_decompress set this value itself, I think it has no effect. If in doubt, this can be changed.

Other points will be addressed later -> next week

I've now pushed changes...

https://github.com/MariaDB/server/pull/12

What changed:
Temporary encryption buffers are now passed to slot structure.
Memset was removed if unnecessary
Memory leaks in unit tests are fixed. There's still a memory leak in my source version, but not cause by our parts.
A litte change was made for windows i/o if page encryption is active.

May we ask when the whole merge process will approximately be done?

Hi,

fil_encrypt_page() is fine. But fil_decrypt_page is a mess, there is too many "buffer"'s. Is all this really necessary ?

buf

out_buf

in_buf

in_buffer

tmp_page_buf

tmp_buf

Do you really need all of these?

Hi.

Sorry, there's always space for improvement..
buf is the encrypted and after processing the decrypted data, this is needed.
out_buf is the encrypted buffer after encryption and not part of fil_decrypt_page. It's needed.
in_buf is if no destination buffer for decryption is reserved in the slot structure than it is created locally. it's needed.
in_buffer was introduced because you requested the "in_buf" to be aligned in the 1st review. I think this is wrong, it does not not be aligned, because it is only used locally.
tmp_page_buf is probably not really needed.
tmp_buf is because we encrypt/decrypt twice. it needs 64 bytes. It holds an interim result. It could be omitted, but since it is needed in fil_encrypt_page i want to keep it for readability.

I want to change fil_decrypt_page so that in_buffer and tmp_page_buf disappear. If further actions are wanted than let me know.

Hi,

Only the buffers that are used on IO needs to be alligned, temporal buffers it is unnecessary. Please remvoe those that are not needed.

R: Jan

I changed. As mentioned i kept one temporay buffer.
pushed to https://github.com/MariaDB/server/pull/12

To repeat the question from above since there is some interest for this here:
When the merge process is approximately completed?

I hope end of this week to MariaDB 10.1.2.

Hi,

One additional question: If page encryption fails do we have original page available, instead of crashing I rather would write that.

R: Jan

I would not write that because the plain text would be on disk and nobody knows it.
But since AES encryption is finally a permutation of bytes, I can not imagine a situation where it can really fail in reality apart from having no more memory or hardware failure. The encryption key is availabe because it is read at start-up.
This means, for every input encryption should be possible.

mfg
Ludger

Hi,

There is two invocation of KeySingleton

	KeySingleton& keys = KeySingleton::getInstance();

		if (!keys.isAvailable()) {

			err = AES_KEY_CREATION_FAILED;

		} else if (keys.getKeys(encryption_key) == NULL) {

			err = PAGE_ENCRYPTION_KEY_MISSING;

		} else {

			char* keyString = keys.getKeys(encryption_key)->key;

			key_len = strlen(keyString)/2;

			my_aes_hexToUint(keyString, (unsigned char*)&rkey, key_len);

		}

and

	KeySingleton& keys = KeySingleton::getInstance();

		if (!keys.isAvailable()) {

			err = AES_KEY_CREATION_FAILED;

		} else if (keys.getKeys(encryption_key) == NULL) {

			err = PAGE_ENCRYPTION_KEY_MISSING;

		} else {

			char* ivString = keys.getKeys(encryption_key)->iv;

			if (ivString == NULL) return buf;

			my_aes_hexToUint(ivString, (unsigned char*)&iv, 16);

		}

May I ask why, and why if I try to combine these like

KeySingleton& keys = KeySingleton::getInstance();

	if (!keys.isAvailable()) {

		err = AES_KEY_CREATION_FAILED;

	} else if (keys.getKeys(encryption_key) == NULL) {

		err = PAGE_ENCRYPTION_KEY_MISSING;

	} else {

		char* keyString = keys.getKeys(encryption_key)->key;

		char* ivString = keys.getKeys(encryption_key)->iv;

		key_len = strlen(keyString)/2;

		my_aes_hexToUint(keyString, (unsigned char*)&rkey, key_len);

		my_aes_hexToUint(ivString, (unsigned char*)&iv, 16);

	}

Page is corrupted.

Hi,

You may forget above, I can combine them, there was bug on checksum writing. I have now merged and tested the XtraDB changes and done some cleanups. Also fixed some Windows compiler errors (math functions). I will not include unittest to MariaDB, instead I will do testing using mtr. My next step is to do same merge to InnoDB storage engine, this should be now easy.

I run to some problems with Windows compiler, with InnoDB I get following kind of errors:

171> Creating library C:/buildbot/winx64-packages/build/mariadb-10.1.2/storage/innobase/RelWithDebInfo/ha_innodb.lib and object C:/buildbot/winx64-packages/build/mariadb-10.1.2/storage/innobase/RelWithDebInfo/ha_innodb.exp

171>EncKeys.obj : error LNK2001: unresolved external symbol pcre_free

171>EncKeys.obj : error LNK2019: unresolved external symbol pcre_exec referenced in function "private: bool __cdecl EncKeys::isComment(char const *)" (?isComment@EncKeys@@AEAA_NPEBD@Z)

171>EncKeys.obj : error LNK2019: unresolved external symbol pcre_compile referenced in function "private: bool __cdecl EncKeys::isComment(char const *)" (?isComment@EncKeys@@AEAA_NPEBD@Z)

171>C:\buildbot\winx64-packages\build\mariadb-10.1.2\storage\innobase\RelWithDebInfo\ha_innodb.dll : fatal error LNK1120: 3 unresolved externals

You should be able to get this branch with:

(1) git clone git@github.com:MariaDB/server.git 10.1-eperi
(2) cd 10.1-eperi
(c) git checkout bb-10.1-eperi

R: Jan

Ok solved by:

diff --git a/storage/innobase/CMakeLists.txt b/storage/innobase/CMakeLists.txt

index 2a4d161..6898144 100644

--- a/storage/innobase/CMakeLists.txt

+++ b/storage/innobase/CMakeLists.txt

@@ -473,5 +473,5 @@ ENDIF()

 MYSQL_ADD_PLUGIN(innobase ${INNOBASE_SOURCES} STORAGE_ENGINE

   MODULE_ONLY

   MODULE_OUTPUT_NAME ha_innodb

-  LINK_LIBRARIES ${ZLIB_LIBRARY} ${LINKER_SCRIPT})

+  LINK_LIBRARIES ${ZLIB_LIBRARY} ${LINKER_SCRIPT} pcre pcreposix)

Ok.. We need pcre.
I can not remember having changed anything for xtradb apart from inserting

#ifdef _WIN_
#define PCRE_STATIC 1
#endif

before including pcre.h
to make it usable with windows.
Looks like it is statically linked for xtradb.

About the encryption method is CTR and GCM more secure than CBC ? Can we use CTR/GCM ?

R: Jan

I'm not aware about theoretical security of block cipher modes ctr, cgm, cbc...
Assuming that encryption algorithm AES is secure and that should be the case according to current knowledge, it should be secure to use any of these.
CBC is more commonly used.
With the yaSSL library provided with mariadb I think there's no support for other mode than CBC (or ECB, which is regarded as less safe).
Since this library is used, It can not be changed easily.