[CONPY-68] jsonfield (longtext) returning as bytes Created: 2020-05-27  Updated: 2020-08-10  Resolved: 2020-08-10

Status: Closed
Project: MariaDB Connector/Python
Component/s: Generic
Affects Version/s: 0.9.59
Fix Version/s: 1.0.1

Type: Bug Priority: Major
Reporter: Luciano Barcaro Assignee: Georg Richter
Resolution: Fixed Votes: 0
Labels: None


 Description   

Internally, mariadb converts json columns to longtext, but all queries involving json columns are returning as bytes. Should return as string.

import mariadb
import json
 
db = mariadb.connect(host='xxx', user='yyy', password='zzz', database='temp')
qr = db.cursor(buffered=True)
 
qr.execute('''create table tmp_json(
    id int unsigned not null primary key,
    myJson json)''')
 
qr.execute('show columns from tmp_json')
for col in qr:
    print(col)
# Returns:
# ('id', 'int(10) unsigned', 'NO', 'PRI', None, '')
# ('myJson', 'longtext', 'YES', '', None, '')
 
content = {'a': 'aaa', 'b': 'bbb', 'c': 123}
 
qr.execute('insert into tmp_json (id, myJson) values (?, ?)', (1, json.dumps(content)))
qr.execute('select myJson from tmp_json where id=1')
 
myJson, = qr.fetchone()
print(myJson, type(myJson))
# Returns:
# b'{"a": "aaa", "b": "bbb", "c": 123}' <class 'bytes'>
 
qr.execute('drop table tmp_json')



 Comments   
Comment by Georg Richter [ 2020-06-15 ]

In MariaDB column type Json is an alias, it creates a column of type longtext with a binary collation. There the metadata contains a binary flag and will be converted to a byte array:

$ mysql -ugeorg test --column_type_info -e"create table t1(a json);select a from t1"

Field   1:  `a`
Catalog:    `def`
Database:   `test`
Table:      `t1`
Org_table:  `t1`
Type:       BLOB
Collation:  utf8_general_ci (33)
Length:     4294967295
Max_length: 0
Decimals:   0
Flags:      BLOB BINARY

To get a python string object, you need to modify your statement:

qr.execute('select convert(myJson, char) from tmp_json where id=1')

Comment by Georg Richter [ 2020-08-10 ]

Reopening this issue:

If a BLOB has a non binary collation, it should be converted to unicode and the BINARY flag should be ignored:

georg@mozart:~/work$ mysql test --column-type-info -e"create temporary table t1(a json);select * from t1;"
Field   1:  `a`
Catalog:    `def`
Database:   `test`
Table:      `t1`
Org_table:  `t1`
Type:       BLOB
Collation:  latin1_swedish_ci (8)
Length:     4294967295
Max_length: 0
Decimals:   0
Flags:      BLOB BINARY

Comment by Georg Richter [ 2020-08-10 ]

Fixed. Rev. 067a78dd13adae905cef6f6c18941d4cbcbd0b2e

Generated at Thu Feb 08 03:30:00 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.