[CONPY-171] Return a Python dict/list when querying a JSON column Created: 2021-10-15  Updated: 2023-01-23

Status: Open
Project: MariaDB Connector/Python
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Minor
Reporter: Jonny Assignee: Georg Richter
Resolution: Unresolved Votes: 0
Labels: None


 Description   

As in psycopg2, it would be wonderful if you get a dict / list in the result when you query a JSON column, so that you don't have to run a json.loads yourself.



 Comments   
Comment by Georg Richter [ 2021-10-15 ]

Hi,

not sure if this will break existing applications. However you can use a simple converter:

>>> import mariadb
>>> from mariadb.constants import *
>>> import json
>>> 
>>> def json_convert(s):
...     return json.loads(s)
... 
>>> conversions= {
...     **{FIELD_TYPE.JSON: json_convert},
... }
>>> 
>>> connection= mariadb.connect(db="test", converter=conversions)
>>> 
>>> cursor= connection.cursor()
>>> 
>>> cursor.execute("create temporary table t2 (a json DEFAULT NULL CHECK (json_valid(`a`)))")
>>> json_str= '{"id": 1, "name": "Monty"}'
>>> cursor.execute("insert into t2 values (?)", (json_str,))
>>> 
>>> cursor.execute("select * from t2")
>>> row= cursor.fetchone()
>>> print(row[0])
{'id': 1, 'name': 'Monty'}
>>> print(type(row[0]))
<class 'dict'>

Comment by Jonny [ 2021-10-15 ]

It could be implemented with a flag.

cur = conn.cursor(convertJSON=True)

Comment by Jonny [ 2021-10-15 ]

It does not work for me:

MariaDB [test]> desc test;
+-------+-------------+------+-----+---------+-------+
| Field | Type        | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| a     | varchar(10) | YES  |     | NULL    |       |
| data  | longtext    | YES  |     | NULL    |       |
+-------+-------------+------+-----+---------+-------+
2 rows in set (0.001 sec)
 
MariaDB [test]> select * from test;
+------+------+
| a    | data |
+------+------+
| foo  | {}   |
+------+------+

#!/usr/bin/python3
 
import mariadb
from mariadb.constants import FIELD_TYPE
import json
 
def json_convert(s):
    return json.loads(s)
 
def varchar_convert(s):
    return "bar"
 
conversions = {
    **{FIELD_TYPE.JSON: json_convert},
    **{FIELD_TYPE.VARCHAR: varchar_convert}
}
 
conn = mariadb.connect(
    user="test",
    password="test",
    host="127.0.0.1",
    port=3306,
    database="test",
    converter=conversions
)
 
cursor = conn.cursor()
cursor.execute("select * from test")
for row in cursor:
    print(row)

('foo', '{}')

  1. pip3 list | grep mariadb
    mariadb 1.0.7

mariadb-server-10.3

Do I need a special version?

Comment by Georg Richter [ 2021-10-15 ]

Sorry, you will need Mariadb 10.5.2 Server or newer (see https://jira.mariadb.org/browse/MDEV-17832).

Comment by Jonny [ 2021-10-15 ]

Thank you. I updated to Debian Bullseye with MariaDB 10.5.12 and it works fine.

However, I noticed one thing and that the field type varchar doesn't seem to be correct.

create table test (`varchar` varchar(10));
insert into test (`varchar`) values ('foo');
desc test;
+---------+-------------+------+-----+---------+-------+
| Field   | Type        | Null | Key | Default | Extra |
+---------+-------------+------+-----+---------+-------+
| varchar | varchar(10) | YES  |     | NULL    |       |
+---------+-------------+------+-----+---------+-------+

def varchar_convert(s):
    return "bar"
 
conversions = {
    **{FIELD_TYPE.VARCHAR: varchar_convert}
}
cursor.execute("select * from test")
for row in cursor:
    print(row)
 
print("VARCHAR", FIELD_TYPE.VARCHAR)
 
for desc in cursor.description:
    print(desc)

('foo',)
VARCHAR 15
('varchar', 253, 10, 40, 0, 0, True, 0)

FIELD_TYPE.VARCHAR is 15. cursor.description shows 253.

Generated at Thu Feb 08 03:30:47 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.