[MDEV-11101] storage/tokudb uses much more space than other engines Created: 2016-10-21  Updated: 2021-10-28

Status: Confirmed
Project: MariaDB Server
Component/s: Storage Engine - TokuDB
Affects Version/s: 5.5, 10.0, 10.1, 10.2
Fix Version/s: 10.2

Type: Bug Priority: Minor
Reporter: Mark Callaghan Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None

Issue Links:
Relates
relates to MDEV-19780 Remove the TokuDB storage engine Closed

 Description   

This is from the 10.2 MariaDB branch. TokuDB uses 161M, no other engine uses more than 20M. Can we apply compression to its impact on the github repo and my time to download it?

--- du -hs * | grep M
 
1.1M    client
 
1.2M    debian
 
3.0M    extra
 
1.5M    include
 
3.3M    libmariadb
 
171M    mysql-test
 
1.7M    mysys
 
6.9M    pcre
 
2.7M    plugin
 
1.2M    scripts
 
18M     sql
 
2.9M    sql-bench
 
248M    storage
 
16M     strings


--- du -hs storage/* | grep M
 
 
 
12M     connect
 
13M     innobase
 
3.5M    maria
 
18M     mroonga
 
1.6M    myisam
 
1.7M    perfschema
 
19M     rocksdb
 
5.9M    spider
 
162M    tokudb
 
11M     xtradb


--- du -hs storage/tokudb/* | grep M
 
 
 
143M    mysql-test
 
18M     PerconaFT


--- du -hs storage/tokudb/PerconaFT/* | grep M
 
 
 
3.9M    PerconaFT/ft
 
5.1M    PerconaFT/src
 
6.2M    PerconaFT/third_party


--- du -hs storage/tokudb/mysql-test/* | grep M
 
 
 
98M     mysql-test/tokudb
 
18M     mysql-test/tokudb_alter_table
 
13M     mysql-test/tokudb_bugs
 
14M     mysql-test/tokudb_parts



 Comments   
Comment by Mark Callaghan [ 2016-10-21 ]

Worst 50 offenders from mysql-test. Number in first column is 1k blocks. Largest result file is 30M, many others are multi-MB. This wastes time, network and disk space for everyone using the MariaDB branch.

find mysql-test/ -type f -exec ls -s {} \; | sort -rnk 1,1 | head -50
30280 mysql-test/tokudb/r/savepoint-5.result
6248 mysql-test/tokudb/t/change_column_all_1000_10.test
6220 mysql-test/tokudb/r/change_column_all_1000_10.result
4772 mysql-test/tokudb/t/change_column_all_1000_1.test
4744 mysql-test/tokudb/r/change_column_all_1000_1.result
3692 mysql-test/tokudb_bugs/r/db739_upsert.result
3692 mysql-test/tokudb_bugs/r/db739_replace.result
3692 mysql-test/tokudb_bugs/r/db739_insert.result
3440 mysql-test/tokudb/r/bulk-fetch.result
3440 mysql-test/tokudb/r/bulk-fetch2.result
3196 mysql-test/tokudb_parts/r/partition_alter4_tokudb.result
3056 mysql-test/tokudb/r/hotindex-del-slow.result
3056 mysql-test/tokudb/r/hotindex-del-fast.result
1620 mysql-test/tokudb/t/fast_update_blobs_fixed_varchar.test
1612 mysql-test/tokudb/t/fast_update_blobs_with_varchar.test
1608 mysql-test/tokudb/r/fast_update_blobs_fixed_varchar.result
1600 mysql-test/tokudb/r/fast_update_blobs_with_varchar.result
1524 mysql-test/tokudb_alter_table/r/hcad_all_add.result
1524 mysql-test/tokudb_alter_table/r/hcad_all_add3.result
1524 mysql-test/tokudb_alter_table/r/hcad_all_add2.result
1520 mysql-test/tokudb_alter_table/t/hcad_all_add.test
1520 mysql-test/tokudb_alter_table/t/hcad_all_add3.test
1520 mysql-test/tokudb_alter_table/t/hcad_all_add2.test
1476 mysql-test/tokudb/r/hotindex-insert-1.result
1476 mysql-test/tokudb/r/hotindex-insert-0.result
1472 mysql-test/tokudb/r/hotindex-update-1.result
1472 mysql-test/tokudb/r/hotindex-update-0.result
1368 mysql-test/tokudb/t/bulk-fetch.test
1368 mysql-test/tokudb/t/bulk-fetch2.test
1360 mysql-test/tokudb/r/fast_update_varchar.result
1356 mysql-test/tokudb/r/hotindex-del-1.result
1356 mysql-test/tokudb/r/hotindex-del-0.result
1296 mysql-test/tokudb_parts/r/partition_alter1_2_tokudb.result
1168 mysql-test/tokudb/t/fast_update_varchar.test
1132 mysql-test/tokudb_parts/r/partition_basic_tokudb.result
1064 mysql-test/tokudb_alter_table/r/hcad_all_drop.result
1060 mysql-test/tokudb_alter_table/t/hcad_all_drop.test
1024 mysql-test/tokudb_parts/r/partition_alter1_1_2_tokudb.result
960 mysql-test/tokudb/t/change_column_bin.test
944 mysql-test/tokudb/t/change_column_char.test
888 mysql-test/tokudb/r/change_column_bin.result
872 mysql-test/tokudb/r/change_column_char.result
856 mysql-test/tokudb/t/fast_update_blobs.test
848 mysql-test/tokudb/r/fast_update_blobs.result
752 mysql-test/tokudb_parts/r/partition_max_parts_inv_tokudb.result
736 mysql-test/tokudb/r/type_temporal_fractional.result
728 mysql-test/tokudb_parts/r/partition_alter2_2_2_tokudb.result
724 mysql-test/tokudb_parts/r/partition_alter2_2_1_tokudb.result
724 mysql-test/tokudb_parts/r/partition_alter2_1_2_tokudb.result
724 mysql-test/tokudb_parts/r/partition_alter2_1_1_tokudb.result

Comment by Marko Mäkelä [ 2021-10-28 ]

For what it is worth, TokuDB was disabled (the source code is dead weight) in MariaDB Server 10.5 and removed in 10.6 (MDEV-19780). But, I do not think that it helped reduce the size of the repository.

Generated at Thu Feb 08 07:47:19 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.