[MDEV-8407] Numeric errors, server crash with COLUMN_JSON() on DECIMAL with precision > 40 - Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Fixed
Affects Version/s: 10.0(EOL), 10.1(EOL)
Fix Version/s: 5.5.48, 10.0.23, 10.1.10
Component/s: Dynamic Columns
Labels:
None
Environment:
Darwin yossy.local 14.3.0 Darwin Kernel Version 14.3.0: Mon Mar 23 11:59:05 PDT 2015; root:xnu-2782.20.48~5/RELEASE_X86_64 x86_64

Sprint:
10.0.23

Description

Lines 4 through 11 of the following input have numeric errors in the JSON output. Line 12 crashes the server.

$ mysql -fN <<SQL

SELECT COLUMN_JSON(COLUMN_CREATE('x', 0.12345678901234567890123456789012345));

SELECT COLUMN_JSON(COLUMN_CREATE('x', 0.123456789012345678901234567890123456));

SELECT COLUMN_JSON(COLUMN_CREATE('x', 0.1234567890123456789012345678901234567));

SELECT COLUMN_JSON(COLUMN_CREATE('x', 0.12345678901234567890123456789012345678));

SELECT COLUMN_JSON(COLUMN_CREATE('x', 0.123456789012345678901234567890123456789));

SELECT COLUMN_JSON(COLUMN_CREATE('x', 0.1234567890123456789012345678901234567890));

SELECT COLUMN_JSON(COLUMN_CREATE('x', 0.12345678901234567890123456789012345678901));

SELECT COLUMN_JSON(COLUMN_CREATE('x', 0.123456789012345678901234567890123456789012));

SELECT COLUMN_JSON(COLUMN_CREATE('x', 0.1234567890123456789012345678901234567890123));

SELECT COLUMN_JSON(COLUMN_CREATE('x', 0.12345678901234567890123456789012345678901234));

SELECT COLUMN_JSON(COLUMN_CREATE('x', 0.123456789012345678901234567890123456789012345));

SELECT COLUMN_JSON(COLUMN_CREATE('x', 0.1234567890123456789012345678901234567890123467));

SQL

{"x":0.12345678901234567890123456789012345}

{"x":0.123456789012345678901234567890123456}

{"x":0.1234567890123456789012345678901234567}

{"x":0.12345678901234567890123456789012345670}

{"x":0.123456789012345678901234567890123456700}

{"x":0.1234567890123456789012345678901234567000}

{"x":0.12345678901234567890123456789012345670000}

{"x":0.123456789012345678901234567890123456700000}

{"x":0.1234567890123456789012345678901234567000000}

{"x":0.12345678901234567890123456789012345670000000}

{"x":0.123456789012345678901234567890123456700000000}

ERROR 2013 (HY000) at line 12: Lost connection to MySQL server during query

/usr/local/bin/mysqld_safe: line 183:  2570 Abort trap: 6

…

150701 09:15:49 mysqld_safe mysqld restarted

Lines 6 through 13 of the following input have numeric errors in the JSON output. Line 14 crashes the server.

fsb@yossy [~] ��

$ mysql -fN <<SQL

> SELECT COLUMN_JSON(COLUMN_CREATE('x', 12345678901234567890123456789012345));

> SELECT COLUMN_JSON(COLUMN_CREATE('x', 123456789012345678901234567890123456));

> SELECT COLUMN_JSON(COLUMN_CREATE('x', 1234567890123456789012345678901234567));

> SELECT COLUMN_JSON(COLUMN_CREATE('x', 12345678901234567890123456789012345678));

> SELECT COLUMN_JSON(COLUMN_CREATE('x', 123456789012345678901234567890123456789));

> SELECT COLUMN_JSON(COLUMN_CREATE('x', 1234567890123456789012345678901234567890));

> SELECT COLUMN_JSON(COLUMN_CREATE('x', 12345678901234567890123456789012345678901));

> SELECT COLUMN_JSON(COLUMN_CREATE('x', 123456789012345678901234567890123456789012));

> SELECT COLUMN_JSON(COLUMN_CREATE('x', 1234567890123456789012345678901234567890123));

> SELECT COLUMN_JSON(COLUMN_CREATE('x', 12345678901234567890123456789012345678901234));

> SELECT COLUMN_JSON(COLUMN_CREATE('x', 123456789012345678901234567890123456789012345));

> SELECT COLUMN_JSON(COLUMN_CREATE('x', 1234567890123456789012345678901234567890123467));

> SELECT COLUMN_JSON(COLUMN_CREATE('x', 12345678901234567890123456789012345678901234678));

> SELECT COLUMN_JSON(COLUMN_CREATE('x', 123456789012345678901234567890123456789012346789));

> SQL

{"x":12345678901234567890123456789012345}

{"x":123456789012345678901234567890123456}

{"x":1234567890123456789012345678901234567}

{"x":12345678901234567890123456789012345678}

{"x":123456789012345678901234567890123456789}

{"x":0234567890123456789012345678901234567890}

{"x":00345678901234567890123456789012345678901}

{"x":000456789012345678901234567890123456789012}

{"x":0000567890123456789012345678901234567890123}

{"x":00000678901234567890123456789012345678901234}

{"x":000000789012345678901234567890123456789012345}

{"x":0000000001234567890123456789012345678901234567}

{"x":00000000012345678901234567890123456789012345678}

ERROR 2013 (HY000) at line 14: Lost connection to MySQL server during query

/usr/local/bin/mysqld_safe: line 183:  2584 Abort trap: 6

…

150701 09:16:00 mysqld_safe mysqld restarted

Note 1: JSON supports decimal number with arbitrary precision and scale.

Note 2: Apart from being numerically wrong, lines 6 through 13 of the output in the second test
are invalid JSON, which allows only numbers with magnitude less than one to begin with the digit 0.

Attachments

Activity

Ascending order - Click to sort in descending order

Elena Stepanova added a comment - 2015-07-01 18:49

Thanks for the report.

The truncated values might be due to the limitation on the scale (30 digits for DECIMAL); although, I'd expect there should be a warning somewhere – either on adding the value or on retrieving it. I'll leave it to sanja to decide what would be the right behavior.

The crash is definitely a bug.

ATTN sanja:
On some reason, for me on 10.0 it only happens on a release build made by buildbot, but not on my builds from source (even if I also build non-debug).
On 10.1 I get a crash on a debug build also.

Here is the stack trace from this build http://buildbot.askmonty.org/buildbot/builders/kvm-bintar-centos5-amd64/builds/1545 (the latest 10.0 as of now).

#2  <signal handler called>

#3  __memcpy_ssse3 () at ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:119

#4  0x0000000000bcb9e4 in dynstr_append_mem (str=0x7f479301e3e0, append=0x7f479301e200 "0.123456789012345678901234567890123456700000000", length=3158064) at /home/buildbot/buildbot/build/mysys/string.c:112

#5  0x0000000000bd6184 in mariadb_dyncol_val_str (str=0x7f479301e3e0, val=0x7f479301e300, cs=0x1410aa0, quote=<optimized out>) at /home/buildbot/buildbot/build/mysys/ma_dyncol.c:3901

#6  0x0000000000bd88e4 in mariadb_dyncol_json_internal (str=<optimized out>, json=0x7f479301e3e0, lvl=1) at /home/buildbot/buildbot/build/mysys/ma_dyncol.c:4184

#7  0x00000000007ac6b7 in Item_func_dyncol_json::val_str (this=0x7f478a01b578, str=0x7f479301e7e0) at /home/buildbot/buildbot/build/sql/item_strfunc.cc:4616

#8  0x0000000000734190 in Item::send (this=0x7f478a01b578, protocol=0x7f478bad2508, buffer=0x7f479301e7e0) at /home/buildbot/buildbot/build/sql/item.cc:6485

#9  0x0000000000515472 in Protocol::send_result_set_row (this=0x7f478bad2508, row_items=<optimized out>) at /home/buildbot/buildbot/build/sql/protocol.cc:904

#10 0x000000000056508c in select_send::send_data (this=0x7f478a01b7d0, items=...) at /home/buildbot/buildbot/build/sql/sql_class.cc:2552

#11 0x00000000005f0130 in JOIN::exec_inner (this=0x7f478a01b7f0) at /home/buildbot/buildbot/build/sql/sql_select.cc:2452

#12 0x00000000005ee19a in exec (this=<optimized out>) at /home/buildbot/buildbot/build/sql/sql_select.cc:2368

#13 mysql_select (thd=0x7f478bad2008, rref_pointer_array=0x7f478bad6190, tables=0x0, wild_num=0, fields=..., conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7f478a01b7d0, unit=0x7f478bad5828, select_lex=0x7f478bad5f18) at /home/buildbot/buildbot/build/sql/sql_select.cc:3308

#14 0x00000000005f1dcd in handle_select (thd=0x7f478bad2008, lex=0x7f478bad5760, result=0x7f478a01b7d0, setup_tables_done_option=0) at /home/buildbot/buildbot/build/sql/sql_select.cc:373

#15 0x0000000000595bf0 in execute_sqlcom_select (thd=0x7f478bad2008, all_tables=0x0) at /home/buildbot/buildbot/build/sql/sql_parse.cc:5274

#16 0x0000000000598847 in mysql_execute_command (thd=0x7f478bad2008) at /home/buildbot/buildbot/build/sql/sql_parse.cc:2562

#17 0x000000000059fc86 in mysql_parse (thd=0x7f478bad2008, rawbuf=0x7f478a01b020 "SELECT COLUMN_JSON(COLUMN_CREATE('x', 0.123456789012345678901234567890123456789012345))", length=<optimized out>, parser_state=0x7f4793020bd0) at /home/buildbot/buildbot/build/sql/sql_parse.cc:6529

#18 0x00000000005a1bb7 in dispatch_command (command=COM_QUERY, thd=0x7f478bad2008, packet=<optimized out>, packet_length=87) at /home/buildbot/buildbot/build/sql/sql_parse.cc:1308

#19 0x00000000005a22f9 in do_command (thd=0x7f478bad2008) at /home/buildbot/buildbot/build/sql/sql_parse.cc:999

#20 0x000000000066c7d4 in do_handle_one_connection (thd_arg=0x7f478bad2008) at /home/buildbot/buildbot/build/sql/sql_connect.cc:1378

#21 0x000000000066c912 in handle_one_connection (arg=0x7f478bad2008) at /home/buildbot/buildbot/build/sql/sql_connect.cc:1293

#22 0x0000000000a84d39 in pfs_spawn_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/storage/perfschema/pfs.cc:1860

#23 0x00007f4792c0fb50 in start_thread (arg=<optimized out>) at pthread_create.c:304

#24 0x00007f4791b8b95d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112

Elena Stepanova added a comment - 2015-07-01 18:49 Thanks for the report. The truncated values might be due to the limitation on the scale (30 digits for DECIMAL); although, I'd expect there should be a warning somewhere – either on adding the value or on retrieving it. I'll leave it to sanja to decide what would be the right behavior. The crash is definitely a bug. ATTN sanja : On some reason, for me on 10.0 it only happens on a release build made by buildbot, but not on my builds from source (even if I also build non-debug). On 10.1 I get a crash on a debug build also. Here is the stack trace from this build http://buildbot.askmonty.org/buildbot/builders/kvm-bintar-centos5-amd64/builds/1545 (the latest 10.0 as of now). #2 <signal handler called> #3 __memcpy_ssse3 () at ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:119 #4 0x0000000000bcb9e4 in dynstr_append_mem (str=0x7f479301e3e0, append=0x7f479301e200 "0.123456789012345678901234567890123456700000000", length=3158064) at /home/buildbot/buildbot/build/mysys/string.c:112 #5 0x0000000000bd6184 in mariadb_dyncol_val_str (str=0x7f479301e3e0, val=0x7f479301e300, cs=0x1410aa0, quote=<optimized out>) at /home/buildbot/buildbot/build/mysys/ma_dyncol.c:3901 #6 0x0000000000bd88e4 in mariadb_dyncol_json_internal (str=<optimized out>, json=0x7f479301e3e0, lvl=1) at /home/buildbot/buildbot/build/mysys/ma_dyncol.c:4184 #7 0x00000000007ac6b7 in Item_func_dyncol_json::val_str (this=0x7f478a01b578, str=0x7f479301e7e0) at /home/buildbot/buildbot/build/sql/item_strfunc.cc:4616 #8 0x0000000000734190 in Item::send (this=0x7f478a01b578, protocol=0x7f478bad2508, buffer=0x7f479301e7e0) at /home/buildbot/buildbot/build/sql/item.cc:6485 #9 0x0000000000515472 in Protocol::send_result_set_row (this=0x7f478bad2508, row_items=<optimized out>) at /home/buildbot/buildbot/build/sql/protocol.cc:904 #10 0x000000000056508c in select_send::send_data (this=0x7f478a01b7d0, items=...) at /home/buildbot/buildbot/build/sql/sql_class.cc:2552 #11 0x00000000005f0130 in JOIN::exec_inner (this=0x7f478a01b7f0) at /home/buildbot/buildbot/build/sql/sql_select.cc:2452 #12 0x00000000005ee19a in exec (this=<optimized out>) at /home/buildbot/buildbot/build/sql/sql_select.cc:2368 #13 mysql_select (thd=0x7f478bad2008, rref_pointer_array=0x7f478bad6190, tables=0x0, wild_num=0, fields=..., conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7f478a01b7d0, unit=0x7f478bad5828, select_lex=0x7f478bad5f18) at /home/buildbot/buildbot/build/sql/sql_select.cc:3308 #14 0x00000000005f1dcd in handle_select (thd=0x7f478bad2008, lex=0x7f478bad5760, result=0x7f478a01b7d0, setup_tables_done_option=0) at /home/buildbot/buildbot/build/sql/sql_select.cc:373 #15 0x0000000000595bf0 in execute_sqlcom_select (thd=0x7f478bad2008, all_tables=0x0) at /home/buildbot/buildbot/build/sql/sql_parse.cc:5274 #16 0x0000000000598847 in mysql_execute_command (thd=0x7f478bad2008) at /home/buildbot/buildbot/build/sql/sql_parse.cc:2562 #17 0x000000000059fc86 in mysql_parse (thd=0x7f478bad2008, rawbuf=0x7f478a01b020 "SELECT COLUMN_JSON(COLUMN_CREATE('x', 0.123456789012345678901234567890123456789012345))", length=<optimized out>, parser_state=0x7f4793020bd0) at /home/buildbot/buildbot/build/sql/sql_parse.cc:6529 #18 0x00000000005a1bb7 in dispatch_command (command=COM_QUERY, thd=0x7f478bad2008, packet=<optimized out>, packet_length=87) at /home/buildbot/buildbot/build/sql/sql_parse.cc:1308 #19 0x00000000005a22f9 in do_command (thd=0x7f478bad2008) at /home/buildbot/buildbot/build/sql/sql_parse.cc:999 #20 0x000000000066c7d4 in do_handle_one_connection (thd_arg=0x7f478bad2008) at /home/buildbot/buildbot/build/sql/sql_connect.cc:1378 #21 0x000000000066c912 in handle_one_connection (arg=0x7f478bad2008) at /home/buildbot/buildbot/build/sql/sql_connect.cc:1293 #22 0x0000000000a84d39 in pfs_spawn_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/storage/perfschema/pfs.cc:1860 #23 0x00007f4792c0fb50 in start_thread (arg=<optimized out>) at pthread_create.c:304 #24 0x00007f4791b8b95d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112

Tom added a comment - 2015-07-01 19:26

COLUMN_CREATE and COLUMN_GET cope with 65 digits of precision, as default MySQL DECIMALS do. That's why I suspect this lies closer to the JSON part.

$ mysql -Ne "SELECT COLUMN_GET(COLUMN_CREATE('x', 12345678901234567890123456789012345678901234567890123456789012345), 'x' as DECIMAL(65))"

+-------------------------------------------------------------------+

| 12345678901234567890123456789012345678901234567890123456789012345 |

+-------------------------------------------------------------------+

Tom added a comment - 2015-07-01 19:26 COLUMN_CREATE and COLUMN_GET cope with 65 digits of precision, as default MySQL DECIMALS do. That's why I suspect this lies closer to the JSON part. $ mysql -Ne "SELECT COLUMN_GET(COLUMN_CREATE('x', 12345678901234567890123456789012345678901234567890123456789012345), 'x' as DECIMAL(65))" +-------------------------------------------------------------------+ | 12345678901234567890123456789012345678901234567890123456789012345 | +-------------------------------------------------------------------+

Elena Stepanova added a comment - 2015-07-01 20:11

I was referring to the first set of lines in the description.
Yes, the total DECIMAL's precision is 65 digits, but the maximum scale (the number of digits to the right of the decimal point) is only 30. https://dev.mysql.com/doc/refman/5.6/en/precision-math-decimal-characteristics.html
Try this:

MariaDB [test]> select cast(0.123456789012345678901234567890123456789 as decimal(65,64));

ERROR 1425 (42000): Too big scale 64 specified for '0.123456789012345678901234567890123456789'. Maximum is 30.

MariaDB [test]> select cast(0.123456789012345678901234567890123456789 as decimal(35,30));

+-------------------------------------------------------------------+

| cast(0.123456789012345678901234567890123456789 as decimal(35,30)) |

+-------------------------------------------------------------------+

|                                  0.123456789012345678901234567890 |

+-------------------------------------------------------------------+

1 row in set (0.01 sec)

Sorry I missed the second set of lines before. Yes, I agree, it looks erroneous to me as well, and explicit AS in COLUMN_CREATE does not solve it either.

MariaDB [test]> SELECT COLUMN_JSON(COLUMN_CREATE('x', 123456789012345678901234567890123456789012346789 as decimal(50,0)));

+----------------------------------------------------------------------------------------------------+

| COLUMN_JSON(COLUMN_CREATE('x', 123456789012345678901234567890123456789012346789 as decimal(50,0))) |

+----------------------------------------------------------------------------------------------------+

| {"x":000000000123456789012345678901234567890123456789}                                             |

+----------------------------------------------------------------------------------------------------+

1 row in set (0.00 sec)

Elena Stepanova added a comment - 2015-07-01 20:11 I was referring to the first set of lines in the description. Yes, the total DECIMAL's precision is 65 digits, but the maximum scale (the number of digits to the right of the decimal point) is only 30. https://dev.mysql.com/doc/refman/5.6/en/precision-math-decimal-characteristics.html Try this: MariaDB [test]> select cast (0.123456789012345678901234567890123456789 as decimal (65,64)); ERROR 1425 (42000): Too big scale 64 specified for '0.123456789012345678901234567890123456789' . Maximum is 30. MariaDB [test]> select cast (0.123456789012345678901234567890123456789 as decimal (35,30)); + -------------------------------------------------------------------+ | cast (0.123456789012345678901234567890123456789 as decimal (35,30)) | + -------------------------------------------------------------------+ | 0.123456789012345678901234567890 | + -------------------------------------------------------------------+ 1 row in set (0.01 sec) Sorry I missed the second set of lines before. Yes, I agree, it looks erroneous to me as well, and explicit AS in COLUMN_CREATE does not solve it either. MariaDB [test]> SELECT COLUMN_JSON(COLUMN_CREATE( 'x' , 123456789012345678901234567890123456789012346789 as decimal (50,0))); + ----------------------------------------------------------------------------------------------------+ | COLUMN_JSON(COLUMN_CREATE( 'x' , 123456789012345678901234567890123456789012346789 as decimal (50,0))) | + ----------------------------------------------------------------------------------------------------+ | { "x" :000000000123456789012345678901234567890123456789} | + ----------------------------------------------------------------------------------------------------+ 1 row in set (0.00 sec)

Sergei Golubchik added a comment - 2015-12-09 18:00

ok to push

Sergei Golubchik added a comment - 2015-12-09 18:00 ok to push

Oleksandr Byelkin added a comment - 2015-12-10 12:26

fixed in 5.5:
revision-id: fa25921b59aacdc6be050653f6cce17df12c6937 (mariadb-5.5.46-39-gfa25921)
parent(s): d67aacb4fbb449ffa8db4e3d70fe8756d62b5222
committer: Oleksandr Byelkin
timestamp: 2015-12-10 11:22:53 +0100
message:

~~MDEV-8407~~ Numeric errors, server crash with COLUMN_JSON() on DECIMAL with precision > 40

In fact it was error in decimal library (incorrect processing of buffer overflow) invisible from other server parts because of buffer allocation and precision tests.

—

Oleksandr Byelkin added a comment - 2015-12-10 12:26 fixed in 5.5: revision-id: fa25921b59aacdc6be050653f6cce17df12c6937 (mariadb-5.5.46-39-gfa25921) parent(s): d67aacb4fbb449ffa8db4e3d70fe8756d62b5222 committer: Oleksandr Byelkin timestamp: 2015-12-10 11:22:53 +0100 message: MDEV-8407 Numeric errors, server crash with COLUMN_JSON() on DECIMAL with precision > 40 In fact it was error in decimal library (incorrect processing of buffer overflow) invisible from other server parts because of buffer allocation and precision tests. —

People

Assignee:: Oleksandr Byelkin

Reporter:: Tom

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 2015-07-01 16:27

Updated:: 2015-12-10 17:29

Resolved:: 2015-12-10 12:29

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server