If before or upon an upgrade the server was not shut down properly, the upgraded TokuDB cannot start:
No race conditions or special workflow are required to render the engine broken this way, just usual forced termination.
There is no proper workaround for it:
<prohaska7> tokudb will not load if its recovery log is an old version that did not shutdown cleanly.
<prohaska7> bring up the old version (probably with clients locked out) and shut it down. this will write a nice shutdown entry into the tokudb recovery log. then bring up the new version (which should see the old log with the nice shutdown and proceed)
<prohaska7> one could create a new instance of the old version, put the tokudb files in its data directory, bring it up and shut it down. then copy the tokudb log files to the new version and bring it up.
<prohaska7> clean shutdown is required.
There was an actual user's bug report about it already:
MDEV-6165 , so the issue is not hypothetical.