[MDEV-23648] s3.partition_move 'innodb' test failure - object deleted before copy attempted - Jira

XML

Word

Printable

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Critical
Resolution: Fixed
Affects Version/s: 10.5.5
Fix Version/s: 10.5.7
Component/s: Storage Engine - S3, Tests
Labels:
None

Description

worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019

s3.partition_move 'innodb'               [ fail ]

        Test ended at 2020-09-02 14:11:20

CURRENT_TEST: s3.partition_move

mysqltest: At line 61: query 'alter table archive exchange partition p1 with table p1' failed: 29: Expected object 's3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive#P#p1/data/000001' didn't exist

fake s3 server
04:11:19.772 [200 OK] s3.ListObjectsV1 127.0.0.1/justdoit/?delimiter=%2F&prefix=test%2F 127.0.0.1 271µs ↑ 73 B ↓ 513 B
04:11:19.775 [200 OK] s3.ListObjectsV1 127.0.0.1/justdoit/?delimiter=%2F&prefix=performance_schema%2F 127.0.0.1 158µs ↑ 73 B ↓ 527 B
04:11:19.777 [200 OK] s3.ListObjectsV1 127.0.0.1/justdoit/?delimiter=%2F&prefix=test%2F 127.0.0.1 209µs ↑ 73 B ↓ 513 B
04:11:19.778 [200 OK] s3.ListObjectsV1 127.0.0.1/justdoit/?delimiter=%2F&prefix=mtr%2F 127.0.0.1 165µs ↑ 73 B ↓ 512 B
04:11:19.810 [404 Not Found] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p0/frm 127.0.0.1 315µs ↑ 73 B ↓ 225 B
04:11:19.812 [404 Not Found] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p0/frm 127.0.0.1 213µs ↑ 73 B ↓ 225 B
04:11:19.820 [404 Not Found] s3.GetObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/seq_1_to_99/frm 127.0.0.1 374µs ↑ 73 B ↓ 664 B
04:11:19.823 [404 Not Found] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive/frm 127.0.0.1 267µs ↑ 73 B ↓ 225 B
04:11:19.827 [404 Not Found] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive#P#p0/aria 127.0.0.1 325µs ↑ 73 B ↓ 225 B
04:11:19.839 [200 OK] s3.PutObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive#P#p0/aria 127.0.0.1 11.79ms ↑ 8.1 KiB ↓ 259 B
04:11:19.845 [200 OK] s3.PutObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive#P#p0/index/000001 127.0.0.1 4.269ms ↑ 8.1 KiB ↓ 259 B
04:11:19.850 [200 OK] s3.PutObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive#P#p0/data/000001 127.0.0.1 4.199ms ↑ 16 KiB ↓ 259 B
04:11:19.852 [404 Not Found] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive/frm 127.0.0.1 262µs ↑ 73 B ↓ 225 B
04:11:19.857 [200 OK] s3.PutObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive/frm 127.0.0.1 3.79ms ↑ 1.1 KiB ↓ 259 B
04:11:19.858 [404 Not Found] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive/par 127.0.0.1 299µs ↑ 73 B ↓ 225 B
04:11:19.862 [200 OK] s3.PutObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive/par 127.0.0.1 3.565ms ↑ 114 B ↓ 259 B
04:11:19.866 [200 OK] s3.GetObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive/frm 127.0.0.1 825µs ↑ 73 B ↓ 1.4 KiB
04:11:19.868 [200 OK] s3.GetObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive#P#p0/aria 127.0.0.1 946µs ↑ 73 B ↓ 8.3 KiB
04:11:19.871 [404 Not Found] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/t1/frm 127.0.0.1 369µs ↑ 73 B ↓ 225 B
04:11:19.873 [404 Not Found] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/t1/frm 127.0.0.1 277µs ↑ 73 B ↓ 225 B
04:11:19.877 [404 Not Found] s3.GetObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/seq_100_to_399/frm 127.0.0.1 413µs ↑ 73 B ↓ 670 B
04:11:19.881 [404 Not Found] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/frm 127.0.0.1 300µs ↑ 73 B ↓ 225 B
04:11:19.883 [404 Not Found] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/frm 127.0.0.1 297µs ↑ 73 B ↓ 225 B
04:11:19.893 [404 Not Found] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/aria 127.0.0.1 290µs ↑ 73 B ↓ 225 B
04:11:19.898 [200 OK] s3.PutObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/frm 127.0.0.1 4.21ms ↑ 1.0 KiB ↓ 259 B
04:11:19.903 [200 OK] s3.PutObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/aria 127.0.0.1 4.384ms ↑ 8.1 KiB ↓ 259 B
04:11:19.908 [200 OK] s3.PutObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/index/000001 127.0.0.1 4.031ms ↑ 8.1 KiB ↓ 259 B
04:11:19.913 [200 OK] s3.PutObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/data/000001 127.0.0.1 3.358ms ↑ 16 KiB ↓ 259 B
04:11:19.915 [200 OK] s3.GetObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive/frm 127.0.0.1 748µs ↑ 73 B ↓ 1.4 KiB
04:11:19.918 [404 Not Found] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive#P#p1/aria 127.0.0.1 345µs ↑ 73 B ↓ 225 B
04:11:19.922 [200 OK] s3.PutObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive#P#p1/aria 127.0.0.1 3.906ms ↑ 8.1 KiB ↓ 259 B
04:11:19.927 [200 OK] s3.PutObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive#P#p1/data/000001 127.0.0.1 4.077ms ↑ 8.1 KiB ↓ 259 B
04:11:19.931 [200 OK] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive/frm 127.0.0.1 950µs ↑ 73 B ↓ 355 B
04:11:19.932 [204 No Content] s3.DeleteObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive/frm 127.0.0.1 603µs ↑ 73 B ↓ 226 B
04:11:19.937 [200 OK] s3.PutObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive/frm 127.0.0.1 4.089ms ↑ 1.1 KiB ↓ 259 B
04:11:19.939 [200 OK] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive/par 127.0.0.1 735µs ↑ 73 B ↓ 353 B
04:11:19.940 [204 No Content] s3.DeleteObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive/par 127.0.0.1 545µs ↑ 73 B ↓ 226 B
04:11:19.944 [200 OK] s3.PutObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive/par 127.0.0.1 3.34ms ↑ 122 B ↓ 259 B
04:11:19.947 [200 OK] s3.GetObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive/frm 127.0.0.1 912µs ↑ 73 B ↓ 1.4 KiB
04:11:19.949 [200 OK] s3.GetObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive#P#p0/aria 127.0.0.1 826µs ↑ 73 B ↓ 8.3 KiB
04:11:19.951 [200 OK] s3.GetObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive/frm 127.0.0.1 809µs ↑ 73 B ↓ 1.4 KiB
04:11:19.953 [200 OK] s3.GetObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive#P#p1/aria 127.0.0.1 783µs ↑ 73 B ↓ 8.3 KiB
04:11:19.955 [200 OK] s3.GetObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/frm 127.0.0.1 764µs ↑ 73 B ↓ 1.3 KiB
04:11:19.957 [200 OK] s3.GetObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/aria 127.0.0.1 978µs ↑ 73 B ↓ 8.3 KiB
04:11:19.958 [200 OK] s3.GetObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/data/000001 127.0.0.1 746µs ↑ 73 B ↓ 16 KiB
04:11:19.961 [200 OK] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/aria 127.0.0.1 692µs ↑ 73 B ↓ 355 B
04:11:19.962 [200 OK] s3.ListObjectsV1 127.0.0.1/justdoit/?prefix=s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c%2Fp1%2Findex 127.0.0.1 663µs ↑ 73 B ↓ 921 B
04:11:19.969 [200 OK] s3.CopyObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/#sql-exchange-1bbf43-4/index/000001 127.0.0.1 6.156ms ↑ 104 B ↓ 523 B
04:11:19.971 [204 No Content] s3.DeleteObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/index/000001 127.0.0.1 665µs ↑ 73 B ↓ 226 B
04:11:19.972 [200 OK] s3.ListObjectsV1 127.0.0.1/justdoit/?prefix=s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c%2Fp1%2Fdata 127.0.0.1 626µs ↑ 73 B ↓ 920 B
04:11:19.977 [200 OK] s3.CopyObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/#sql-exchange-1bbf43-4/data/000001 127.0.0.1 4.723ms ↑ 104 B ↓ 523 B
04:11:19.979 [204 No Content] s3.DeleteObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/data/000001 127.0.0.1 566µs ↑ 73 B ↓ 226 B
04:11:19.984 [200 OK] s3.CopyObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/#sql-exchange-1bbf43-4/aria 127.0.0.1 4.198ms ↑ 104 B ↓ 523 B
04:11:19.985 [204 No Content] s3.DeleteObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/aria 127.0.0.1 438µs ↑ 73 B ↓ 226 B
04:11:19.987 [200 OK] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/archive#P#p1/aria 127.0.0.1 692µs ↑ 73 B ↓ 355 B
04:11:19.989 [200 OK] s3.ListObjectsV1 127.0.0.1/justdoit/?prefix=s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c%2Farchive%23P%23p1%2Findex 127.0.0.1 440µs ↑ 73 B ↓ 566 B
04:11:19.990 [200 OK] s3.ListObjectsV1 127.0.0.1/justdoit/?prefix=s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c%2Farchive%23P%23p1%2Fdata 127.0.0.1 613µs ↑ 73 B ↓ 939 B
04:11:19.991 [404 Not Found] s3.CopyObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/data/000001 127.0.0.1 495µs ↑ 104 B ↓ 662 B
04:11:19.992 [404 Not Found] s3.CopyObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/aria 127.0.0.1 451µs ↑ 104 B ↓ 648 B
04:11:19.994 [404 Not Found] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/aria 127.0.0.1 223µs ↑ 73 B ↓ 225 B
04:11:19.997 [200 OK] s3.HeadObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/#sql-exchange-1bbf43-4/aria 127.0.0.1 965µs ↑ 73 B ↓ 355 B
04:11:19.999 [200 OK] s3.ListObjectsV1 127.0.0.1/justdoit/?prefix=s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c%2F%23sql-exchange-1bbf43-4%2Findex 127.0.0.1 852µs ↑ 73 B ↓ 961 B
04:11:20.000 [404 Not Found] s3.CopyObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/index/000001 127.0.0.1 735µs ↑ 104 B ↓ 664 B
04:11:20.002 [200 OK] s3.ListObjectsV1 127.0.0.1/justdoit/?prefix=s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c%2F%23sql-exchange-1bbf43-4%2Fdata 127.0.0.1 582µs ↑ 73 B ↓ 960 B
04:11:20.003 [404 Not Found] s3.CopyObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/data/000001 127.0.0.1 496µs ↑ 104 B ↓ 662 B
04:11:20.004 [404 Not Found] s3.CopyObject 127.0.0.1/justdoit/s3_test_5ac0aae8ecd211ea8032cabbbeb0fc5c/p1/aria 127.0.0.1 508µs ↑ 104 B ↓ 648 B

The server logs show that the missing object was deleted by s3 (04:11:19.979) before it tried to access it later.

Attachments

Issue Links

relates to

MDEV-23650 test S3 in buildbot

Closed

Activity

People

Assignee:: Michael Widenius

Reporter:: Daniel Black

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 2020-09-02 04:15

Updated:: 2020-10-21 00:12

Resolved:: 2020-10-21 00:12

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.