[CONC-125] Import and export popular data formats from and to dynamic columns Created: 2015-03-02  Updated: 2016-02-18

Status: Open
Project: MariaDB Connector/C
Component/s: None
Affects Version/s: 2.1
Fix Version/s: None

Type: Task Priority: Major
Reporter: Georg Richter Assignee: Georg Richter
Resolution: Unresolved Votes: 1
Labels: gsoc15, gsoc16


 Description   

Provide import and export functions for popular data formats like JSON, XML (limited), PHP, ...
for Connector/C and MariaDB Server (which use same code base for dynamic columns)

Tasks:

Dynamic column library (includes MariaDB Server and Connector/C).
1) Add support for arrays with ordinal indexes (currently dynamic columns are limited to use string indexes only)
2) Provide a library for importing/exporting data to and from Dynamic columns.
The following data formats should be supported:

  • JSON
  • XML (limited to unique keys only)
  • PHP,
  • ...

MariaDB Server:
3) Provide user defined function (UDF) based on library (task 2) for importing and exporting data

MariaDB Connector/C:
3) Provide a plugin interface based on library (task 2) for importing and exporting data

Examples of usage:

1) MariaDB Server

SET @a:='{"menu": {
 
  "id": "file",
 
  "value": "File",
 
  "popup": {
 
    "menuitem": [
 
      {"value": "New", "onclick": "CreateNewDoc()"},
 
      {"value": "Open", "onclick": "OpenDoc()"},
 
      {"value": "Close", "onclick": "CloseDoc()"}
 
    ]
 
  }
 
}}';
 
INSERT INTO my_table (dyncol) VALUES (COLUMN_IMPORT("JSON", @a))

2) Connector/C

 char *json_str='{'menu': {
 
  'id': 'file',
 
  'value': 'File',
 
  'popup': {
 
    'menuitem': [
 
      {'value': 'New', 'onclick': 'CreateNewDoc()'},
 
      {'value': 'Open', 'onclick': 'OpenDoc()'},
 
      {'value': 'Close', 'onclick': 'CloseDoc()'}
 
    ]
 
  }
 
}}";
 
rc= mariadb_dyncol_import(dyncol, "json", json_str);

Programming language: C

 Comments   
Comment by Alberto M Esmorís Pena [ 2015-03-04 ]

Iam interested in this proposal. I have some experience in C. Also I have done similar works before using Java for data management in JSON and XML and some PHP wrappers for REST services using JSON as data format too.

Comment by Oleksandr Byelkin [ 2015-03-04 ]

I have to say that JSON/XML & Co is not the biggest part of the project.

As I can see it (the task) one should take old dynamic column format (numeric keys) and expand type to modern types supported by dynamic columns everything else is simple. So task is stand alone and do not require deep knowledge of the server and client.

Comment by Tushar Bihany [ 2015-03-09 ]

This proposal caught my attention.I am really interested in this idea.From my understanding,we can use libxml2 and Jansson for the import/export purpose and Lucy for indexing purpose.Kindly enlighten me more on this area.

Comment by Oleksandr Byelkin [ 2015-03-09 ]

I think we better not use third party libraries because:
1) maintaining compatibility and dependencies
2) licence sometimes problem
3) formats are simple to parse and print data in them is not a task at all (and we do not need something else)

Comment by Tushar Bihany [ 2015-03-09 ]

If third party libs are not to be used,we can parse without them and serialize/deserialize accordingly.

Comment by Arlene Flavie [ 2015-03-17 ]

Hi am Arlene Flavie and i am interested in this task.
concerning 1: adding support for arrays . Does this mean the dynamic column should hold an array instead of a string such that the array indices become the indexed values when manipulating the column. something like

INSERT INTO tb_dycol (friends) VALUES( ARRAY['serg', 'Byelkin', 'elenst', 'holyfoot', 'monty', 'knielson', 'danblack'] );

If thats the case i get it as it will greatly increase performance as queries will be of O(1) if array indices are used.

Comment by Oleksandr Byelkin [ 2015-03-17 ]

1) Now dynamic columns support other dynamic column as a value (new format support it).
2) old format of dynamic columns uses numbers as a keys so they can be interpreted as an array, but old format do not support all types (including embedded dynamic columns) because have only 3 bits to represent type.

Taking into account above I think that simplest way to support arrays is add new 'old' format in addition to that 2 we have. But we are open to any other better solutions.

Generated at Thu Feb 08 03:03:04 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.