Uploaded image for project: 'MariaDB Connector/C'
  1. MariaDB Connector/C
  2. CONC-125

Import and export popular data formats from and to dynamic columns

Details

    • Task
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • 2.1
    • None
    • None

    Description

      Provide import and export functions for popular data formats like JSON, XML (limited), PHP, ...
      for Connector/C and MariaDB Server (which use same code base for dynamic columns)

      Tasks:

      Dynamic column library (includes MariaDB Server and Connector/C).
      1) Add support for arrays with ordinal indexes (currently dynamic columns are limited to use string indexes only)
      2) Provide a library for importing/exporting data to and from Dynamic columns.
      The following data formats should be supported:

      • JSON
      • XML (limited to unique keys only)
      • PHP,
      • ...

      MariaDB Server:
      3) Provide user defined function (UDF) based on library (task 2) for importing and exporting data

      MariaDB Connector/C:
      3) Provide a plugin interface based on library (task 2) for importing and exporting data

      Examples of usage:

      1) MariaDB Server

      SET @a:='{"menu": {
      		 
        "id": "file",
      		 
        "value": "File",
      		 
        "popup": {
      		 
          "menuitem": [
      		 
            {"value": "New", "onclick": "CreateNewDoc()"},
      		 
            {"value": "Open", "onclick": "OpenDoc()"},
      		 
            {"value": "Close", "onclick": "CloseDoc()"}
      		 
          ]
      		 
        }
      		 
      }}';
      		 
      INSERT INTO my_table (dyncol) VALUES (COLUMN_IMPORT("JSON", @a))

      2) Connector/C

       char *json_str='{'menu': {
      		 
        'id': 'file',
      		 
        'value': 'File',
      		 
        'popup': {
      		 
          'menuitem': [
      		 
            {'value': 'New', 'onclick': 'CreateNewDoc()'},
      		 
            {'value': 'Open', 'onclick': 'OpenDoc()'},
      		 
            {'value': 'Close', 'onclick': 'CloseDoc()'}
      		 
          ]
      		 
        }
      		 
      }}";
      		 
      rc= mariadb_dyncol_import(dyncol, "json", json_str);

      Programming language: C

      Attachments

        Activity

          Ascending order - Click to sort in descending order
          albertoesmp Alberto M Esmorís Pena added a comment -

          Iam interested in this proposal. I have some experience in C. Also I have done similar works before using Java for data management in JSON and XML and some PHP wrappers for REST services using JSON as data format too.

          albertoesmp Alberto M Esmorís Pena added a comment - Iam interested in this proposal. I have some experience in C. Also I have done similar works before using Java for data management in JSON and XML and some PHP wrappers for REST services using JSON as data format too.
          sanja Oleksandr Byelkin added a comment -

          I have to say that JSON/XML & Co is not the biggest part of the project.

          As I can see it (the task) one should take old dynamic column format (numeric keys) and expand type to modern types supported by dynamic columns everything else is simple. So task is stand alone and do not require deep knowledge of the server and client.

          sanja Oleksandr Byelkin added a comment - I have to say that JSON/XML & Co is not the biggest part of the project. As I can see it (the task) one should take old dynamic column format (numeric keys) and expand type to modern types supported by dynamic columns everything else is simple. So task is stand alone and do not require deep knowledge of the server and client.
          tushar.bihany@gmail.com Tushar Bihany added a comment -

          This proposal caught my attention.I am really interested in this idea.From my understanding,we can use libxml2 and Jansson for the import/export purpose and Lucy for indexing purpose.Kindly enlighten me more on this area.

          tushar.bihany@gmail.com Tushar Bihany added a comment - This proposal caught my attention.I am really interested in this idea.From my understanding,we can use libxml2 and Jansson for the import/export purpose and Lucy for indexing purpose.Kindly enlighten me more on this area.
          sanja Oleksandr Byelkin added a comment -

          I think we better not use third party libraries because:
          1) maintaining compatibility and dependencies
          2) licence sometimes problem
          3) formats are simple to parse and print data in them is not a task at all (and we do not need something else)

          sanja Oleksandr Byelkin added a comment - I think we better not use third party libraries because: 1) maintaining compatibility and dependencies 2) licence sometimes problem 3) formats are simple to parse and print data in them is not a task at all (and we do not need something else)
          tushar.bihany@gmail.com Tushar Bihany added a comment -

          If third party libs are not to be used,we can parse without them and serialize/deserialize accordingly.

          tushar.bihany@gmail.com Tushar Bihany added a comment - If third party libs are not to be used,we can parse without them and serialize/deserialize accordingly.
          arleneflavie Arlene Flavie added a comment - - edited

          Hi am Arlene Flavie and i am interested in this task.
          concerning 1: adding support for arrays . Does this mean the dynamic column should hold an array instead of a string such that the array indices become the indexed values when manipulating the column. something like

          INSERT INTO tb_dycol (friends) VALUES( ARRAY['serg', 'Byelkin', 'elenst', 'holyfoot', 'monty', 'knielson', 'danblack'] );

          If thats the case i get it as it will greatly increase performance as queries will be of O(1) if array indices are used.

          arleneflavie Arlene Flavie added a comment - - edited Hi am Arlene Flavie and i am interested in this task. concerning 1: adding support for arrays . Does this mean the dynamic column should hold an array instead of a string such that the array indices become the indexed values when manipulating the column. something like INSERT INTO tb_dycol (friends) VALUES( ARRAY ['serg', 'Byelkin', 'elenst', 'holyfoot', 'monty', 'knielson', 'danblack'] ); If thats the case i get it as it will greatly increase performance as queries will be of O(1) if array indices are used.
          sanja Oleksandr Byelkin added a comment -

          1) Now dynamic columns support other dynamic column as a value (new format support it).
          2) old format of dynamic columns uses numbers as a keys so they can be interpreted as an array, but old format do not support all types (including embedded dynamic columns) because have only 3 bits to represent type.

          Taking into account above I think that simplest way to support arrays is add new 'old' format in addition to that 2 we have. But we are open to any other better solutions.

          sanja Oleksandr Byelkin added a comment - 1) Now dynamic columns support other dynamic column as a value (new format support it). 2) old format of dynamic columns uses numbers as a keys so they can be interpreted as an array, but old format do not support all types (including embedded dynamic columns) because have only 3 bits to represent type. Taking into account above I think that simplest way to support arrays is add new 'old' format in addition to that 2 we have. But we are open to any other better solutions.

          People

            georg Georg Richter
            georg Georg Richter
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.