[MDEV-29932] Invalid expr in cleanup_session_expr() upon INSERT DELAYED Created: 2022-11-01  Updated: 2023-11-27  Resolved: 2023-11-10

Status: Closed
Project: MariaDB Server
Component/s: Data Manipulation - Insert, Virtual Columns
Affects Version/s: 10.3, 10.4, 10.5, 10.6, 10.7, 10.8, 10.9, 10.10, 10.11, 11.0
Fix Version/s: 10.4.33, 10.5.24, 10.6.17, 10.11.7, 11.0.5, 11.1.4

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Aleksey Midenkov
Resolution: Fixed Votes: 0
Labels: ASAN, memory_corruption, pushing

Issue Links:
Duplicate
is duplicated by MDEV-28941 SIGABRT in __libc_message + malloc_pr... Stalled
Relates
relates to MDEV-28566 Assertion `!expr->is_fixed()' failed ... Closed

 Description   

CREATE TABLE t (f TIMESTAMP DEFAULT FROM_UNIXTIME(1)) ENGINE=MyISAM;
INSERT DELAYED INTO t VALUES ();
 
# Cleanup
DROP TABLE t;

10.3 7d96cb47

==2035754==ERROR: AddressSanitizer: use-after-poison on address 0x62b000002240 at pc 0x558e88dff5e7 bp 0x7f4c8e8505a0 sp 0x7f4c8e850598
READ of size 4 at 0x62b000002240 thread T6
    #0 0x558e88dff5e6 in Virtual_column_info::need_refix() const /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/field.h:621
    #1 0x558e88dca0a6 in Virtual_column_info::cleanup_session_expr() /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/table.cc:2998
    #2 0x558e88dcace2 in TABLE::vcol_cleanup_expr(THD*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/table.cc:3095
    #3 0x558e8896b570 in close_thread_table(THD*, TABLE**) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_base.cc:897
    #4 0x558e8896aed9 in close_thread_tables(THD*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_base.cc:871
    #5 0x558e88a5265e in handle_delayed_insert /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_insert.cc:3266
    #6 0x558e8a4441ae in pfs_spawn_thread /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/perfschema/pfs.cc:1869
    #7 0x7f4c997dfea6 in start_thread nptl/pthread_create.c:477
    #8 0x7f4c996ffaee in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xfcaee)
 
0x62b000002240 is located 8256 bytes inside of 24608-byte region [0x62b000000200,0x62b000006220)
allocated by thread T5 here:
    #0 0x7f4c9a0e5e8f in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x558e8a5584ca in my_malloc /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/mysys/my_malloc.c:101
    #2 0x558e8a534a22 in reset_root_defaults /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/mysys/my_alloc.c:152
    #3 0x558e889d8713 in THD::init_for_queries() /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_class.cc:1338
    #4 0x558e88e963da in prepare_new_connection_state(THD*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_connect.cc:1239
    #5 0x558e88e96a9b in thd_prepare_connection(THD*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_connect.cc:1323
    #6 0x558e88e97087 in do_handle_one_connection(CONNECT*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_connect.cc:1393
    #7 0x558e88e96a55 in handle_one_connection /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_connect.cc:1308
    #8 0x558e8a4441ae in pfs_spawn_thread /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/perfschema/pfs.cc:1869
    #9 0x7f4c997dfea6 in start_thread nptl/pthread_create.c:477
 
Thread T6 created by T5 here:
    #0 0x7f4c9a0912a2 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:214
    #1 0x558e8a44459b in spawn_thread_v1 /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/perfschema/pfs.cc:1919
    #2 0x558e88a3bf77 in inline_mysql_thread_create /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/include/mysql/psi/mysql_thread.h:1275
    #3 0x558e88a4b6d6 in delayed_get_table /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_insert.cc:2423
    #4 0x558e88a3f2c1 in open_and_lock_for_insert_delayed /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_insert.cc:569
    #5 0x558e88a4034d in mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_insert.cc:755
    #6 0x558e88ae79c2 in mysql_execute_command(THD*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_parse.cc:4505
    #7 0x558e88aff665 in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_parse.cc:7855
    #8 0x558e88ad6cb3 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_parse.cc:1852
    #9 0x558e88ad386b in do_command(THD*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_parse.cc:1398
    #10 0x558e88e97159 in do_handle_one_connection(CONNECT*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_connect.cc:1403
    #11 0x558e88e96a55 in handle_one_connection /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/sql_connect.cc:1308
    #12 0x558e8a4441ae in pfs_spawn_thread /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/perfschema/pfs.cc:1869
    #13 0x7f4c997dfea6 in start_thread nptl/pthread_create.c:477
 
Thread T5 created by T0 here:
    #0 0x7f4c9a0912a2 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:214
    #1 0x558e8a44459b in spawn_thread_v1 /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/storage/perfschema/pfs.cc:1919
    #2 0x558e8880bd86 in inline_mysql_thread_create /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/include/mysql/psi/mysql_thread.h:1275
    #3 0x558e88823f63 in create_thread_to_handle_connection(CONNECT*) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/mysqld.cc:6677
    #4 0x558e888246c3 in create_new_thread /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/mysqld.cc:6747
    #5 0x558e8882582d in handle_connections_sockets() /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/mysqld.cc:7005
    #6 0x558e888232e3 in mysqld_main(int, char**) /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/mysqld.cc:6299
    #7 0x558e8880a614 in main /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/main.cc:25
    #8 0x7f4c99626d09 in __libc_start_main ../csu/libc-start.c:308
 
SUMMARY: AddressSanitizer: use-after-poison /home/jenkins/workspace/sandbox-elenst/Nightly-Build-CS/src/sql/field.h:621 in Virtual_column_info::need_refix() const
Shadow bytes around the buggy address:
  0x0c567fff83f0: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
  0x0c567fff8400: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
  0x0c567fff8410: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
  0x0c567fff8420: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
  0x0c567fff8430: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
=>0x0c567fff8440: f7 f7 f7 f7 f7 f7 f7 f7[f7]f7 f7 f7 f7 f7 f7 f7
  0x0c567fff8450: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
  0x0c567fff8460: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
  0x0c567fff8470: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
  0x0c567fff8480: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
  0x0c567fff8490: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==2035754==ABORTING

Reproducible on all current versions.



 Comments   
Comment by Alice Sherepa [ 2022-11-11 ]

on non-debug build:

MariaDB [test]> CREATE TABLE t (f TIMESTAMP DEFAULT FROM_UNIXTIME(1)) ENGINE=MyISAM;
Query OK, 0 rows affected (0,014 sec)
 
MariaDB [test]> INSERT DELAYED INTO t VALUES ();
Query OK, 1 row affected (0,001 sec)
 
MariaDB [test]> select 1 from t;
+---+
| 1 |
+---+
| 1 |
+---+
1 row in set (0,000 sec)
 
MariaDB [test]> drop table t;
ERROR 2013 (HY000): Lost connection to server during query

Version: '10.7.7-MariaDB'
221111 16:58:54 [ERROR] mysqld got signal 11 ;
 
Server version: 10.7.7-MariaDB
 
mysys/stacktrace.c:213(my_print_stacktrace)[0x558f8e0630ae]
sql/signal_handler.cc:236(handle_fatal_signal)[0x558f8da44b67]
sigaction.c:0(__restore_rt)[0x7f7e95462420]
sql/table.cc:3645(Virtual_column_info::cleanup_session_expr())[0x558f8d8de0b4]
sql/sql_list.h:429(base_list_iterator::next())[0x558f8d8de33d]
sql/sql_base.cc:979(close_thread_table(THD*, TABLE**))[0x558f8d79bcef]
sql/sql_base.cc:949(close_thread_tables(THD*))[0x558f8d79bf9b]
sql/sql_insert.cc:3442(handle_delayed_insert)[0x558f8d7d629d]
perfschema/pfs.cc:2204(pfs_spawn_thread)[0x558f8dc94e8c]
nptl/pthread_create.c:478(start_thread)[0x7f7e95456609]
 
Query (0x7f7e30011288): t

bb-10.10-release 695f20f1b55949ae5e4870805

Version: '10.10.2-MariaDB-debug-log' 
mariadbd: /10.10/sql/table.cc:3661: bool Virtual_column_info::cleanup_session_expr(): Assertion `need_refix()' failed.
221115 14:15:15 [ERROR] mysqld got signal 6 ;
 
Server version: 10.10.2-MariaDB-debug-log
 
sql/signal_handler.cc:236(handle_fatal_signal)[0x56196a63d4ba]
??:0(__assert_fail)[0x7f2efade2fd6]
sql/table.cc:3662(Virtual_column_info::cleanup_session_expr())[0x56196a113f54]
sql/table.cc:3756(TABLE::vcol_cleanup_expr(THD*))[0x56196a114b3d]
sql/sql_base.cc:988(close_thread_table(THD*, TABLE**))[0x561969b91e1f]
sql/sql_base.cc:960(close_thread_tables(THD*))[0x561969b9176a]
sql/sql_insert.cc:3444(handle_delayed_insert)[0x561969c9c6b0]
perfschema/pfs.cc:2203(pfs_spawn_thread)[0x56196ae323bc]
nptl/pthread_create.c:478(start_thread)[0x7f2efb2fd609]
??:0(clone)[0x7f2efaece133]
 
Query (0x60e000031448): t

Comment by Roel Van de Paar [ 2022-12-03 ]

Ran into this also. With MTR we get SIGSEGV in Virtual_column_info::cleanup_session_expr:

--source include/have_innodb.inc
CREATE TABLE t (c CHAR(1) DEFAULT @@version) ENGINE=MEMORY;
INSERT DELAYED INTO t SET a=1;
CREATE TABLE t (c1 INT) ENGINE=InnoDB COMMENT='abcdefghijklmnopqrstuvwxyz' UNION=(t,t2) INSERT_METHOD=FIRST;
SHUTDOWN;

Leads to:

10.11.2 936436ef437c73911c18854a8ce8dad1216331b8

Core was generated by `/test/MD291122-mariadb-10.11.2-linux-x86_64-dbg/bin/mariadbd --defaults-group-s'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000559b22a32f38 in Virtual_column_info::cleanup_session_expr (this=0x1514a4015c90) at /test/10.11_dbg/sql/table.cc:3662
3662	  return expr->walk(&Item::cleanup_excluding_fields_processor, 0, 0);
[Current thread is 1 (Thread 0x1514d94a4700 (LWP 1445983))]
#0  0x0000559b22a32f38 in Virtual_column_info::cleanup_session_expr (this=0x1514a4015c90) at /test/10.11_dbg/sql/table.cc:3662
#1  0x0000559b22a332d5 in TABLE::vcol_cleanup_expr (this=this@entry=0x1514a80036e8, thd=thd@entry=0x1514a417f838) at /test/10.11_dbg/sql/table.cc:3756
#2  0x0000559b2289ba48 in close_thread_table (thd=thd@entry=0x1514a417f838, table_ptr=table_ptr@entry=0x1514a417f930) at /test/10.11_dbg/sql/sql_base.cc:987
#3  0x0000559b2289beb0 in close_thread_tables (thd=0x1514a417f838) at /test/10.11_dbg/sql/sql_base.cc:961
#4  0x0000559b228dfc70 in handle_delayed_insert (arg=0x1514a417f818) at /test/10.11_dbg/sql/sql_insert.cc:3453
#5  0x00001514e0f96609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#6  0x00001514e0b82133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

And CLI:

SET sql_mode='';
CREATE TABLE t (c CHAR(1) DEFAULT @@version) ENGINE=MEMORY;
INSERT DELAYED INTO t SET a=1;
CREATE TABLE t (c1 INT) ENGINE=InnoDB COMMENT='abcdefghijklmnopqrstuvwxyz' UNION=(t,t2) INSERT_METHOD=FIRST;
SHUTDOWN;

Leads to:

10.11.2 936436ef437c73911c18854a8ce8dad1216331b8 (Debug)

Core was generated by `/test/MD291122-mariadb-10.11.2-linux-x86_64-dbg/bin/mysqld --no-defaults --core'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00005562f7e7bf38 in Virtual_column_info::cleanup_session_expr (
    this=0x154e70014a90) at /test/10.11_dbg/sql/table.cc:3662
3662	  return expr->walk(&Item::cleanup_excluding_fields_processor, 0, 0);
[Current thread is 1 (LWP 1582284)]
(gdb) bt
#0  0x00005562f7e7bf38 in Virtual_column_info::cleanup_session_expr (this=0x154e70014a90) at /test/10.11_dbg/sql/table.cc:3662
#1  0x00005562f7e7c2d5 in TABLE::vcol_cleanup_expr (this=this@entry=0x154e740038e8, thd=thd@entry=0x154e70021418) at /test/10.11_dbg/sql/table.cc:3756
#2  0x00005562f7ce4a48 in close_thread_table (thd=thd@entry=0x154e70021418, table_ptr=table_ptr@entry=0x154e70021510) at /test/10.11_dbg/sql/sql_base.cc:987
#3  0x00005562f7ce4eb0 in close_thread_tables (thd=0x154e70021418) at /test/10.11_dbg/sql/sql_base.cc:961
#4  0x00005562f7d28c70 in handle_delayed_insert (arg=0x154e700213f8) at /test/10.11_dbg/sql/sql_insert.cc:3453
#5  0x0000154ecb052609 in ?? ()
#6  0x0000000000000000 in ?? ()

10.11.2 936436ef437c73911c18854a8ce8dad1216331b8 (Optimized)

Core was generated by `/test/MD291122-mariadb-10.11.2-linux-x86_64-opt/bin/mysqld --no-defaults --core'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000559a9b468e04 in Virtual_column_info::cleanup_session_expr (
    this=<optimized out>) at /test/10.11_opt/sql/table.cc:3662
3662	  return expr->walk(&Item::cleanup_excluding_fields_processor, 0, 0);
[Current thread is 1 (Thread 0x15424404b700 (LWP 1582283))]
(gdb) bt
#0  0x0000559a9b468e04 in Virtual_column_info::cleanup_session_expr (this=<optimized out>) at /test/10.11_opt/sql/table.cc:3662
#1  0x0000559a9b469065 in TABLE::vcol_cleanup_expr (this=this@entry=0x154200002278, thd=thd@entry=0x1541fc01acc8) at /test/10.11_opt/sql/table.cc:3756
#2  0x0000559a9b315d7d in close_thread_table (thd=thd@entry=0x1541fc01acc8, table_ptr=table_ptr@entry=0x1541fc01adc0) at /test/10.11_opt/sql/sql_base.cc:987
#3  0x0000559a9b315fdb in close_thread_tables (thd=0x1541fc01acc8) at /test/10.11_opt/sql/sql_base.cc:961
#4  0x0000559a9b3540d9 in handle_delayed_insert (arg=0x1541fc01aca8) at /test/10.11_opt/sql/sql_insert.cc:3453
#5  0x000015425b90d609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#6  0x000015425b4f9133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

And ASAN:

10.11.0 fe1f8f2c6b6f3b8e3383168225f9ae7853028947 (Optimized, UBASAN)

2022-12-03 14:16:37 0 [Note] /test/UBASAN_MD010922-mariadb-10.11.0-linux-x86_64-opt/bin/mysqld: ready for connections.
Version: '10.11.0-MariaDB'  socket: '/test/UBASAN_MD010922-mariadb-10.11.0-linux-x86_64-opt/socket.sock'  port: 12046  MariaDB Server
2022-12-03 14:16:37 0 [Note] /test/UBASAN_MD010922-mariadb-10.11.0-linux-x86_64-opt/bin/mysqld (initiated by: root[root] @ localhost []): Normal shutdown
=================================================================
==1633076==ERROR: AddressSanitizer: use-after-poison on address 0x629000088c18 at pc 0x5562572445e1 bp 0x14a970803540 sp 0x14a970803530
READ of size 8 at 0x629000088c18 thread T15

Bug confirmed present in:
MariaDB: 10.3.38 (dbg), 10.3.38 (opt), 10.4.28 (dbg), 10.4.28 (opt), 10.5.19 (dbg), 10.5.19 (opt), 10.6.12 (dbg), 10.6.12 (opt), 10.7.8 (dbg), 10.7.8 (opt), 10.8.7 (dbg), 10.8.7 (opt), 10.9.5 (dbg), 10.9.5 (opt), 10.10.3 (dbg), 10.10.3 (opt), 10.11.2 (dbg), 10.11.2 (opt)

Bug (or feature/syntax) confirmed not present in:
MySQL: 5.5.62 (dbg), 5.5.62 (opt), 5.6.51 (dbg), 5.6.51 (opt), 5.7.38 (dbg), 5.7.38 (opt), 8.0.29 (dbg), 8.0.29 (opt)

Comment by Roel Van de Paar [ 2022-12-03 ]

UniqueID's seen (note the failing backtrace, likely due to memory corruption):

SIGSEGV|Virtual_column_info::cleanup_session_expr|Backtrace stopped: Cannot access memory at address
SIGSEGV|Virtual_column_info::cleanup_session_expr|TABLE::vcol_cleanup_expr|close_thread_table|close_thread_tables
SIGSEGV|TABLE::vcol_cleanup_expr|close_thread_table|close_thread_tables|handle_delayed_insert

Comment by Roel Van de Paar [ 2023-01-24 ]

A whole set of new stacks seen with

SET sql_mode='';
CREATE TABLE t (c TEXT GENERATED ALWAYS AS (1) VIRTUAL,INDEX i (c (1))) ENGINE=MyISAM;
INSERT t SELECT 1 ON DUPLICATE KEY UPDATE c=1;
INSERT DELAYED INTO t VALUES();
SELECT SLEEP(3);  # As the INSERT is DELAYED

Leads to all these stacks/UniqueID's:

SIGABRT|__libc_message|malloc_printerr|_int_free|__GI___libc_free
SIGSEGV|malloc_size_and_flag|my_free|Binary_string::free|Binary_string::real_alloc
SIGSEGV|malloc_size_and_flag|my_free|String::free|String::real_alloc
SIGSEGV|my_copy_8bit|charset_info_st::copy_fix|String_copier::well_formed_copy|Field_longstr::well_formed_copy_with_check
SIGSEGV|my_free|Binary_string::free|Binary_string::real_alloc|Binary_string::alloc

Bug confirmed present in:
MariaDB: 10.3.38 (dbg), 10.3.38 (opt), 10.4.28 (dbg), 10.4.28 (opt), 10.5.19 (dbg), 10.5.19 (opt), 10.6.12 (dbg), 10.6.12 (opt), 10.7.8 (dbg), 10.7.8 (opt), 10.8.7 (dbg), 10.8.7 (opt), 10.9.5 (dbg), 10.9.5 (opt), 10.10.3 (dbg), 10.10.3 (opt), 10.11.2 (dbg), 10.11.2 (opt), 11.0.1 (dbg), 11.0.1 (opt)

With this testcase, no ASAN issues were observed, but the ASAN builds crash with a different stack:

SIGSEGV|Field_blob::get_ptr|Field_blob::get_ptr|Field_blob::clear_temporary|unlink_blobs

Comment by Aleksey Midenkov [ 2023-09-18 ]

Please review bb-10.4-midenok

Comment by Nikita Malyavin [ 2023-10-19 ]

Great job, thanks! Ok to push

Generated at Thu Feb 08 10:12:22 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.