HDDS-13921. Conditional Copy (CopyObject)

[YutaLin]

What changes were proposed in this pull request?

support CopyObject conditions include

• x-amz-copy-source-if-match: - Copy only if source ETag matches
• x-amz-copy-source-if-none-match: - Copy only if source ETag doesn't match
• x-amz-copy-source-if-modified-since: - Copy only if source modified after
• x-amz-copy-source-if-unmodified-since: - Copy only if source not modified since
• If-None-Match, If-Match

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-13921

How was this patch tested?

add conditional copy object test and run action ci (https://github.com/YutaLin/ozone/actions/runs/25467526822)

[YutaLin]

Hi @ivandika3, @peterxcli
Could you help me review it? appreciate!

[peterxcli]

LGTM. Could you also add s3 sdk v1/v2 test?

[YutaLin]

Hi @peterxcli, thanks for the review!
I've added S3SDKV1/V2 tests.

[peterxcli]

@YutaLin I think this patch only handle the read side condition check, but we also need dest side:

Destination Object Conditions (AWS Release 2.37.0):

If-None-Match: "*" - Copy only if destination does NOT exist Prevents unintended overwrites of destination object
Error Codes: 412 Precondition Failed - When any condition is not met

[YutaLin]

@YutaLin I think this patch only handle the read side condition check, but we also need dest side:

Destination Object Conditions (AWS Release 2.37.0):
If-None-Match: "*" - Copy only if destination does NOT exist Prevents unintended overwrites of destination object
Error Codes: 412 Precondition Failed - When any condition is not met

Hi @peterxcli, the conditional read side check is
ObjectEndPoint #copyObject -> writeConditions is passed to the #copy-> #openKeyForPut is called with writeConditions -> #openKeyForPut applies If-None-Match, If-Match

Let me know if I miss anything, thanks!

[peterxcli]

how about also add write in here?

[YutaLin]

We don't need it now because IF-Match/IF-None-Match have same behavior on GET/HEAD/CopyObject.

[peterxcli]

Could you please elaborate? which of their behaviour is the same? Thanks!

[YutaLin]

All write operations use the same headers and have the same behavior:

• If-None-Match: * → create only if key doesn't exist
• If-Match: → overwrite only if ETag matches
• Both throw 412 on failure

[peterxcli]

oh, I got your point, write header are always the same, this class is only for distinguishing the copy/read request header(they have same suffix)
However the PreconditionContext name is too generic, let's rename it to sth like ReadCondition.

[peterxcli]

@YutaLin I think this patch only handle the read side condition check, but we also need dest side:

Destination Object Conditions (AWS Release 2.37.0):
If-None-Match: "*" - Copy only if destination does NOT exist Prevents unintended overwrites of destination object
Error Codes: 412 Precondition Failed - When any condition is not met

Hi @peterxcli, the conditional read side check is ObjectEndPoint #copyObject -> writeConditions is passed to the #copy-> #openKeyForPut is called with writeConditions -> #openKeyForPut applies If-None-Match, If-Match

Let me know if I miss anything, thanks!

thanks for explanation, then I think only thing remain is the test coverage for them.

I saw the copy request builder has following we should use the cover the write part in sdk v2:

public final String ifMatch() {
  return this.ifMatch;
}

public final String ifNoneMatch() {
  return this.ifNoneMatch;
}

v1 also need

[YutaLin]

thanks for explanation, then I think only thing remain is the test coverage for them.

I saw the copy request builder has following we should use the cover the write part in sdk v2:

public final String ifMatch() {
  return this.ifMatch;
}

public final String ifNoneMatch() {
  return this.ifNoneMatch;
}

v1 also need

Hi @peterxcli
I've add tests in sdkv2. I don't add it in v1 because it doesn't support destination conditions for CopyObject. Only v2 has ifMatch/IfNoneMatch method.
sources:
https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/CopyObjectRequest.html

CI: https://github.com/YutaLin/ozone/actions/runs/25591732427

[peterxcli]

I don't add it in v1 because it doesn't support destination conditions for CopyObject. Only v2 has ifMatch/IfNoneMatch method.

• https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/CopyObjectRequest.html#setMatchingETagConstraints-java.util.List-
• https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/CopyObjectRequest.html#setNonmatchingETagConstraints-java.util.List-

[YutaLin]

• https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/CopyObjectRequest.html#setMatchingETagConstraints-java.util.List-
• https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/CopyObjectRequest.html#setNonmatchingETagConstraints-java.util.List-

Hi @peterxcli, thanks for the feedback
getMatchingETagConstraints, setMatchingETagConstraints are already used for source object's ETag test.
see #setMatchingETagConstraints, #testCopyObjectWithSourceIfMatchFail, #testCopyObjectWithSourceIfNoneMatch and #testCopyObjectWithSourceIfNoneMatchFail