[MDEV-20980] multi thread for bulk insert Created: 2019-11-05  Updated: 2022-06-17  Resolved: 2022-06-17

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - Spider
Fix Version/s: N/A

Type: Task Priority: Major
Reporter: willhan Assignee: Nayuta Yanagisawa (Inactive)
Resolution: Won't Do Votes: 0
Labels: None


 Description   

multi thread for bulk insert when load data is very useful, so we implement it.

1. handler->ha_get_bg_result: means wait for the background thread in handler API, maybe we can use it in commit/rollback interface.
2. add parameter of "sql_type" in spider_bg_conn_search , to distinguish select and the other type.
3. spider_set_conn_bg_param_for_dml: check whether it can insert parallel mode. Maybe it should decided by a new spider parameter
4. spider_db_bulk_insert: main logic for parallel insert_lock_tables_list
5. spider_bg_conn_action: reuse the most logic. Take careful of dbton_handler->set_sql_for_exec
6. add spider_bgs_mode=1 spider_bg_dml=1 to control whether start multi thread



 Comments   
Comment by willhan [ 2019-11-05 ]

Hi kentoku
the commit is 74c5fd4c874

Comment by Nayuta Yanagisawa (Inactive) [ 2021-08-13 ]

Hi, willhan! Sorry for our silence. Could you give me the link to your commit, again? The above link seems to be dead. cc: Yongxin Xu

Comment by Yongxin Xu [ 2021-08-16 ]

Hi, nayuta-yanagisawa, I'm returning my office computer today, and I can no longer receive email via my "@tencent.com" email.
You can ask other team members for more information later.

Comment by Nayuta Yanagisawa (Inactive) [ 2021-08-16 ]

Yongxin Xu OK, thank you very much for everything, and good luck

Comment by Yongxin Xu [ 2021-08-19 ]

nayuta-yanagisawa Thank you so much, and I wish you the best too!

Comment by Nayuta Yanagisawa (Inactive) [ 2022-06-17 ]

No feedback from the reporter.

Generated at Thu Feb 08 09:03:41 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.