[MDEV-21285] RAND()/ENCODE() lack of concurrency Created: 2019-12-11  Updated: 2021-03-08

Status: Open
Project: MariaDB Server
Component/s: Server
Affects Version/s: 10.4.11
Fix Version/s: 10.4

Type: Bug Priority: Major
Reporter: Daniel Black Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None

Attachments: File rand-test.lua    
Issue Links:
Relates
relates to MDEV-21256 Improve InnoDB random number generato... Closed

 Description   

./rand-test.lua --threads=140 --db-driver=mysql --mysql-user=root --mysql-db=information_schema  --mysql-host=10.61.210.91  --histogram  run
 
{
    "errors":    0,
    "events": 2423705,
    "latency_avg": 0.0005767600,
    "latency_max": 0.0049197530,
    "latency_min": 0.0001138200,
    "latency_pct": 0.0005972396,
    "latency_sum": 1397.8983984280,
    "other":    0,
    "reads": 2423705,
    "reconnects":    0,
    "threads_running":    0,
    "time_interval": 10.0254642940,
    "time_total": 10.0225462340,
    "writes":    0

Samples: 715K of event 'cycles', Event count (approx.): 109823327400
  Overhead  Command  Shared Object       Symbol
+    9.85%  mysqld   mysqld              [.] SQL_CRYPT::init                                                                  ◆
+    9.42%  mysqld   mysqld              [.] my_rnd                                                                           ▒
+    8.05%  mysqld   [kernel.vmlinux]    [k] ctx_sched_in        

-    9.85%  mysqld   mysqld              [.] SQL_CRYPT::init                                                                  ◆
     __clone                                                                                                                  ▒
     start_thread                                                                                                             ▒
     handle_one_connection                                                                                                    ▒
     do_handle_one_connection                                                                                                 ▒
     do_command                                                                                                               ▒
     dispatch_command                                                                                                         ▒
     mysql_stmt_execute_common                                                                                                ▒
     Prepared_statement::execute_loop                                                                                         ▒
     Prepared_statement::execute                                                                                              ▒
     mysql_execute_command                                                                                                    ▒
     execute_sqlcom_select                                                                                                    ▒
     handle_select                                                                                                            ▒
     mysql_select                                                                                                             ▒
     JOIN::exec                                                                                                               ▒
     JOIN::exec_inner                                                                                                         ▒
     select_send::send_data                                                                                                   ▒
     Protocol::send_result_set_row                                                                                            ▒
     Item::send                                                                                                               ▒
     Type_handler_string_result::Item_send                                                                                    ▒
     Type_handler::Item_send_str                                                                                              ▒
   - Item_func_encode::val_str                                                                                                ▒
      - 9.82% Item_func_encode::seed                                                                                          ▒
           SQL_CRYPT::init     
 
       │         int idx= (uint) (my_rnd(&rand)*255.0);
  2.71 │ 70:   mr      r3,r30
  0.02 │     → bl      my_rnd
  2.45 │       nop
       │         char a= decode_buff[idx];
       │         decode_buff[idx]= decode_buff[i];
  4.57 │       lbzu    r8,1(r31)
       │         int idx= (uint) (my_rnd(&rand)*255.0);
  0.00 │       lfd     f0,0(r28)
       │       for (i=0 ; i<= 255 ; i++)
 19.45 │       cmpld   cr7,r31,r27
       │         int idx= (uint) (my_rnd(&rand)*255.0);
  0.05 │       fmul    f1,f1,f0
  0.00 │       fctiwuz f1,f1
 18.98 │       mffprwz r9,f1
  0.03 │       extsw   r9,r9
       │         char a= decode_buff[idx];
       │       add     r9,r30,r9
 23.24 │       lbz     r10,64(r9)
       │         decode_buff[idx]= decode_buff[i];
  2.10 │       stb     r8,64(r9)
       │         decode_buff[+i]=a;
  0.03 │       stb     r10,0(r31)
       │       for (i=0 ; i<= 255 ; i++)
       │     ↑ bne     cr7,70
       │       }
       │       for (i=0 ; i <= 255 ; i++)
  0.02 │       li      r9,256
       │       li      r10,0

-    9.42%  mysqld   mysqld              [.] my_rnd                                                                           ▒
     __clone                                                                                                                  ▒
     start_thread                                                                                                             ▒
     handle_one_connection                                                                                                    ▒
   - do_handle_one_connection                                                                                                 ▒
      - 9.42% do_command                                                                                                      ▒
           dispatch_command                                                                                                   ▒
           mysql_stmt_execute_common                                                                                          ▒
           Prepared_statement::execute_loop                                                                                   ▒
           Prepared_statement::execute                                                                                        ▒
           mysql_execute_command                                                                                              ▒
           execute_sqlcom_select                                                                                              ▒
           handle_select                                                                                                      ▒
           mysql_select                                                                                                       ▒
           JOIN::exec                                                                                                         ▒
           JOIN::exec_inner                                                                                                   ▒
           select_send::send_data                                                                                             ▒
           Protocol::send_result_set_row                                                                                      ▒
           Item::send                                                                                                         ▒
           Type_handler_string_result::Item_send                                                                              ▒
           Type_handler::Item_send_str                                                                                        ▒
         - Item_func_encode::val_str                                                                                          ▒
            - 9.26% Item_func_encode::seed                                                                                    ▒
               - 9.16% SQL_CRYPT::init                                                                                        ▒
                    my_rnd            
 
  0.04 │      rldicr r9,r6,1,62
       │      rand_st->seed2=(seed1+rand_st->seed2+33) % rand_st->max_value;
       │      addi   r8,r10,33
       │      seed1= (rand_st->seed1*3+rand_st->seed2) % rand_st->max_value;
  5.77 │      add    r9,r9,r6
  0.19 │      add    r10,r9,r10
  0.66 │      divdu  r9,r10,r7
  4.79 │      mulld  r9,r9,r7
  0.07 │      subf   r9,r9,r10
       │      rand_st->seed2=(seed1+rand_st->seed2+33) % rand_st->max_value;
  1.23 │      add    r8,r8,r9
       │      return (((double) seed1)/rand_st->max_value_dbl);
 19.94 │      mtvsrd vs12,r9
       │      rand_st->seed2=(seed1+rand_st->seed2+33) % rand_st->max_value;
  0.08 │      divdu  r10,r8,r7
       │      rand_st->seed1= seed1;
  3.22 │      std    r9,0(r3)
       │      return (((double) seed1)/rand_st->max_value_dbl);
 50.64 │      fcfidu f1,f12
       │    }
  3.38 │      fdiv   f1,f1,f0
       │      rand_st->seed2=(seed1+rand_st->seed2+33) % rand_st->max_value;
  0.19 │      mulld  r9,r10,r7
  2.58 │      subf   r9,r9,r8
  0.32 │      std    r9,8(r3)
       │    }

Inital report from otto on zulip



 Comments   
Comment by Marko Mäkelä [ 2019-12-12 ]

It looks like a significant amount of time is being spent in a floating-point instruction. But, can we avoid it in this particular case, without breaking compatibility? The test case rand-test.lua is using

SELECT ENCODE('hello',RAND());

As far as I understand the code of the ENCODE function in Item_func_encode::val_str(), both arguments are internally converted to strings. So, apparently, for compatibility, the RAND() value must be a floating-point value between 0 and slightly less than 1 that is then converted to a string.

I do not see an easy way to replace the floating-point arithmetics with integer arithmetics. As far as I can tell, we could theoretically ‘push down’ things like integer_constant*RAND() to a new predefined function INT_RAND(integer_constant) and then override Item::val_int() for it. Perhaps we could similarly translate ENCODE(something, RAND()) into ENCODE(something, INT_RAND(1<<32)), but I would say that it is approaching a gray area when it comes to the resulting distribution of the data.

Comment by Daniel Black [ 2019-12-16 ]

Yeh, was coming to the same conclusion. That `RAND()`, `UUID()` all to some extent need the same questionable RND algorithms as the randoms seeds end up in the binary log and that `binlog_format=statement` is still a thing. sql/password.c might be fixable. Moving existing implementation to `my_not_rnd` and adding a proper implementation like the `my_rnd_ssl`, except actually used (and integer based).

Generated at Thu Feb 08 09:05:59 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.