Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-4783

New .frm file format

    XMLWordPrintable

Details

    • Task
    • Status: Open (View Workflow)
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      Instead of work around old unireg frm file, we could create a new human readable, with jornaling (maybe transactional) and with big limit (for enum fields for example) that could remove limits of today unireg frm format (enum limit, partition informations, etc)

      an idea could be using JSON or YAML, or like drizzle use protobuff that allow a binary fast parser, or maybe something like VIEW frm file

      from email at maria-developers:

      --------
      2013/7/10 Jean Weisbuch <jean@phpnet.org>
      >Hi,
      >Maybe a transitional setting that would create/update both the "legacy" frm file and a file with the new format when required could be a solution for backward compatibility.
      >In that case, it would read the new format definition file if present.
      >Another possibility could be that we could choose the default format and only newly created tables would use that format but it wouldnt modify the existing ones (or maybe via an ALTER).

      i like the ALTER idea, something like ALTER TABLE xxxx FRM_FORMAT='OLD/JSON/YAML/TEXT/xyz/protobuff/.... etc'

      >It also could be of use to have a tool that could offline convert the files between the two format in case MariaDB would have to be downgraded or replaced by MySQL or another fork that does not support the new definition file format.

      it's nice too, i put a MDEV when my frm file crashed, maybe we could put this idea there: https://mariadb.atlassian.net/browse/MDEV-4637


      what format we should use? what format is easy to develop, fast, compact, human compatible, etc...? json,yaml,protobuf,text file like view format? any idea?

      i think that at least file should have a hash (md5 for example) to easly find crashed files, should be easly readable by humans (json/yaml/view like) and fast to read/write (protobuf for example or some binary format)

      i was thinking about something like (json example):
      "

      {some json definition}

      \n
      BINARY FORMAT\n
      MD5SUM"

      the MD5SUM is a MD5 value from "{" to end of BINARY FORMAT (before "\n")
      if we remove the BINARY FORMAT, MD5 will change, if we don't have the BINARY FORMAT mariadb read the json, and save the binary format
      if BINARY FORMAT exists we should use it, if some user changed the json it should remove binary format and md5 or mysql should report a error "No MD5SUM on frm file %s", "Wrong MD5SUM on frm file %s", when file don't have a BINARY_FORMAT we could log it as a warning "No binary format at frm file %s, creating..."
      I think it's a easy format to change (add coments) and repair file via "vi","nano","notepad" etc, i don't know how easy it's to implement at source code (json + protobuf/binary format + logs)

      what you think guys?

      Attachments

        Issue Links

          Activity

            People

              sanja Oleksandr Byelkin
              rspadim roberto spadim
              Votes:
              4 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.