Details

    Description

      The TokuDB storage engine has been deprecated by upstream:

      TokuDB is deprecated in the 8.0 series and will be supported through the 8.0 series until further notice. This storage engine will not be included in the next major release of Percona Server for MySQL. We recommend MyRocks as a long-term migration path.

      We have neither knowledge nor resources to take over TokuDB maintenance.
      TokuDB users can consider migrating to InnoDB or RocksDB, imperfect as it is. Or ask TokuDB owner to resume TokuDB maintenance.

      Attachments

        Issue Links

          Activity

            bb-10.5-MDEV-19780 looks OK with respect to the TokuDB removal

            marko Marko Mäkelä added a comment - bb-10.5- MDEV-19780 looks OK with respect to the TokuDB removal

            TokuDB is now disabled in 10.5.0.

            serg Sergei Golubchik added a comment - TokuDB is now disabled in 10.5.0.
            Alex/AT Alex/AT added a comment -

            Actually it's a very bad idea.
            I have converted few huge tables to RocksDB and well... it's barely usable.
            First of all, it takes a whole lot of gigabytes of RAM to sustain the tables, MariaDB server memory usage went from 10G on TokuDB to 20G on RocksDB.
            Second, TokuDB has online DDL, not only index creation, but column operations. And it's pretty much replication safe, compared to RocksDB.
            Imagine extending a 100-200GB worth RocksDB table with a new index or column, locking table for the while of operation, stalling the system, and you'll quickly understand what's terribly wrong about it.

            To sum it, please re-enable TokuDB in current versions. Even if it won't get any new fixes, it's production ready and was the only lifesaver many people with big data tables had.

            Alex/AT Alex/AT added a comment - Actually it's a very bad idea. I have converted few huge tables to RocksDB and well... it's barely usable. First of all, it takes a whole lot of gigabytes of RAM to sustain the tables, MariaDB server memory usage went from 10G on TokuDB to 20G on RocksDB. Second, TokuDB has online DDL, not only index creation, but column operations. And it's pretty much replication safe, compared to RocksDB. Imagine extending a 100-200GB worth RocksDB table with a new index or column, locking table for the while of operation, stalling the system, and you'll quickly understand what's terribly wrong about it. To sum it, please re-enable TokuDB in current versions. Even if it won't get any new fixes, it's production ready and was the only lifesaver many people with big data tables had.
            Alex/AT Alex/AT added a comment -

            Sorry for commenting again on a closed issue, but I think it would be good for some users to know.

            A tolerable replacement for TokuDB is InnoDB + page compression (we use zlib, but it would be good to have something like zstd & lzma compiled in for next versions, maybe this can be considered).
            While it's terrible on HDDs, on SSDs in our case of few hundred gigabytes tables it gives around only 50% performance drop compared to TokuDB on large queries, inserts are around 15% to 30% slower. Which is precisely tolerable for our case - of course everyone mileage may vary. Page compression actually gives around 2x to 4x worse compression compared to TokuDB (depending on table structure and data), but this is still better than 10x to 20x growth we have with uncompressed InnoDB. Disk throughput corresponds, and while it's still 1.5x to 4x load on queries, it's tolerable (while uncompressed InnoDB easily saturates 10G iSCSI link on the same query set of ours).

            I saw another issue (don't remember the number) suggesting page compression removal for InnoDB, please don't! If this would be removed, this would be a total disaster. I want to stress Rocks is unusable for our case in terms of replication support, transaction isolation behavior, lack of online DDL, memory consumption and much more.

            Alex/AT Alex/AT added a comment - Sorry for commenting again on a closed issue, but I think it would be good for some users to know. A tolerable replacement for TokuDB is InnoDB + page compression (we use zlib, but it would be good to have something like zstd & lzma compiled in for next versions, maybe this can be considered). While it's terrible on HDDs, on SSDs in our case of few hundred gigabytes tables it gives around only 50% performance drop compared to TokuDB on large queries, inserts are around 15% to 30% slower. Which is precisely tolerable for our case - of course everyone mileage may vary. Page compression actually gives around 2x to 4x worse compression compared to TokuDB (depending on table structure and data), but this is still better than 10x to 20x growth we have with uncompressed InnoDB. Disk throughput corresponds, and while it's still 1.5x to 4x load on queries, it's tolerable (while uncompressed InnoDB easily saturates 10G iSCSI link on the same query set of ours). I saw another issue (don't remember the number) suggesting page compression removal for InnoDB, please don't! If this would be removed, this would be a total disaster. I want to stress Rocks is unusable for our case in terms of replication support, transaction isolation behavior, lack of online DDL, memory consumption and much more.

            Alex/AT, it sounds like you are using page_compressed with InnoDB, and not ROW_FORMAT=COMPRESSED (which I would like to remove; see MDEV-23497 for a first ‘deprecation warning’ step in 10.6)? Did you find a filesystem where fragmentation is not an issue? Note: in 10.5, MDEV-15528 and MDEV-8139 improved the handling of page_compressed tables, by punching holes over completely freed pages.

            Please reply in MDEV-22839.

            marko Marko Mäkelä added a comment - Alex/AT , it sounds like you are using page_compressed with InnoDB, and not ROW_FORMAT=COMPRESSED (which I would like to remove; see MDEV-23497 for a first ‘deprecation warning’ step in 10.6)? Did you find a filesystem where fragmentation is not an issue? Note: in 10.5, MDEV-15528 and MDEV-8139 improved the handling of page_compressed tables, by punching holes over completely freed pages. Please reply in MDEV-22839 .

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.