namespace: ensure that access latency and retention policy are always…

[kofemann]

… defined

Motivation:
Many parts in dcache expected that files access latency and retention policy are always defined. The namespace fallbacks to default values, if there are no explicit tags are defined. However, there is shortcat tha skips default values if WriteToken tag is specified. The lately introduces Archivemetadata is a functionality that directry affected by this behaviour:

java.lang.IllegalStateException: Attribute is not defined: RETENTION_POLICY
	at org.dcache.vehicles.FileAttributes.guard(FileAttributes.java:335)
	at org.dcache.vehicles.FileAttributes.getRetentionPolicy(FileAttributes.java:504)
	at org.dcache.webdav.DcacheResourceFactory$WriteTransfer.createNameSpaceEntry(DcacheResourceFactory.java:1940)
	at org.dcache.webdav.DcacheResourceFactory.createFile(DcacheResourceFactory.java:752)
	at org.dcache.webdav.DcacheDirectoryResource.createNew(DcacheDirectoryResource.java:148)
	at io.milton.http.http11.PutHandler.processCreate(PutHandler.java:229)
	at io.milton.http.http11.PutHandler.process(PutHandler.java:207)
	at org.dcache.webdav.DcacheStandardFilter.process(DcacheStandardFilter.java:50)
	at io.milton.http.FilterChain.process(FilterChain.java:46)
	at org.dcache.webdav.transfer.CopyFilter.process(CopyFilter.java:278)
	at io.milton.http.FilterChain.process(FilterChain.java:46)
	at io.milton.http.HttpManager.process(HttpManager.java:158)
	at org.dcache.webdav.MiltonHandler.handle(MiltonHandler.java:77)
	at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59)

Modification:
Remove dependency on WriteToken. Update Spacemanager (which expected no access latency and retention policy) to override the value, if space token has a different value. The mismatch is logged.

Result:
The access latency and retention policies are always defined.

Fixes: #8033
Acked-by: Dmitry Litvintsev
Target: master, 11.2
Require-book: no
Require-notes: yes
(cherry picked from commit 5acd486)

[vokac]

After applying this patch transfers still fails

[vokac@ui1 ~]$ export SRC=https://dcache.farm.particle.cz/atlas/atlasscratchdisk/SAM/1M
[vokac@ui1 ~]$ export DST=https://dcache.farm.particle.cz/atlas/atlasscratchdisk/SAM/x
[vokac@ui1 ~]$ export TSRC=$(curl --silent --cert /tmp/x509up_u$(id -u) --key /tmp/x509up_u$(id -u) --cacert /tmp/x509up_u$(id -u) --capath /etc/grid-security/certificates -X POST -H 'Content-Type: application/macaroon-request' -d '{"caveats": ["activity:DOWNLOAD"], "validity": "PT30M"}' "$SRC" | jq -r '.macaroon')
[vokac@ui1 ~]$ export TDST=$(curl --silent --cert /tmp/x509up_u$(id -u) --key /tmp/x509up_u$(id -u) --cacert /tmp/x509up_u$(id -u) --capath /etc/grid-security/certificates -X POST -H 'Content-Type: application/macaroon-request' -d '{"caveats": ["activity:UPLOAD,DELETE,LIST,MANAGE"], "validity": "PT30M"}' "$DST" | jq -r '.macaroon')


# HTTP PUT
[vokac@ui1 ~]$ curl -s --capath /etc/grid-security/certificates -L -X PUT -H "Authorization: Bearer $TDST" --upload-file /etc/services $DST
692252 bytes uploaded

[root@dcache ~]# chimera stat /atlas/atlasscratchdisk/SAM/x
  File: /atlas/atlasscratchdisk/SAM/x
  Size: 692252  Blocks: -  IO Blocks: -  regular file
Device: 17:13  Inode: 446  Links: 1
Access: (644/-rw-r--r--)  Uid: (2011)   Gid: (2011)
Access: Mar 6, 2026, 6:46:06 PM
Modify: Mar 6, 2026, 6:46:06 PM
Change: Mar 6, 2026, 6:46:06 PM
 Birth: Mar 6, 2026, 6:46:06 PM
Access Latency: online  Retention Policy: replica
ID: 00005F57F433EF004A998584376567A28389
QoS policy: unknown  state: unknown
State: data upload completed

[root@dcache ~]# chimera get RetentionPolicy /atlas/atlasscratchdisk/SAM/x
REPLICA


# HTTP TPC pull
[vokac@ui1 ~]$ curl -v --capath /etc/grid-security/certificates -L -X COPY -H 'RequireChecksumVerification: false' -H 'Credential: none' -H "Authorization: Bearer $TDST" -H "TransferHeaderAuthorization: Bearer $TSRC" -H "Source: $SRC" "$DST"
...
< HTTP/1.1 500 Server Error
< Server: dCache/11.2.1
< Connection: close
...

[root@dcache ~]# chimera stat /atlas/atlasscratchdisk/SAM/x
  File: /atlas/atlasscratchdisk/SAM/x
  Size: 0  Blocks: -  IO Blocks: -  regular file
Device: 17:13  Inode: 448  Links: 1
Access: (644/-rw-r--r--)  Uid: (2011)   Gid: (2011)
Access: Mar 6, 2026, 6:46:27 PM
Modify: Mar 6, 2026, 6:46:27 PM
Change: Mar 6, 2026, 6:46:27 PM
 Birth: Mar 6, 2026, 6:46:27 PM
Access Latency: not specified  Retention Policy: not specified
ID: 0000A1F5D7310C15469CB647D4C63F3D5564
QoS policy: unknown  state: unknown
State: data upload not completed

[root@dcache ~]# chimera get RetentionPolicy /atlas/atlasscratchdisk/SAM/x
null

[root@dcache ~]# journalctl 
Mar 06 18:46:27 dcache.farm.particle.cz dcache@doorsDomain[309618]: 06 Mar 2026 18:46:27 (WebDAV-dcache) [] /atlas/atlasscratchdisk/SAM/x
Mar 06 18:46:27 dcache.farm.particle.cz dcache@doorsDomain[309618]: java.lang.IllegalStateException: Attribute is not defined: RETENTION_POLICY
Mar 06 18:46:27 dcache.farm.particle.cz dcache@doorsDomain[309618]:         at org.dcache.vehicles.FileAttributes.guard(FileAttributes.java:335)
Mar 06 18:46:27 dcache.farm.particle.cz dcache@doorsDomain[309618]:         at org.dcache.vehicles.FileAttributes.getRetentionPolicy(FileAttributes.java:504)
Mar 06 18:46:27 dcache.farm.particle.cz dcache@doorsDomain[309618]:         at org.dcache.webdav.transfer.RemoteTransferHandler$RemoteTransfer.resolvePath(RemoteTransferHandler.java:912)
Mar 06 18:46:27 dcache.farm.particle.cz dcache@doorsDomain[309618]:         at org.dcache.webdav.transfer.RemoteTransferHandler$RemoteTransfer.start(RemoteTransferHandler.java:942)
Mar 06 18:46:27 dcache.farm.particle.cz dcache@doorsDomain[309618]:         at org.dcache.webdav.transfer.RemoteTransferHandler.acceptRequest(RemoteTransferHandler.java:687)
Mar 06 18:46:27 dcache.farm.particle.cz dcache@doorsDomain[309618]:         at org.dcache.webdav.transfer.CopyFilter.processThirdPartyCopy(CopyFilter.java:438)
Mar 06 18:46:27 dcache.farm.particle.cz dcache@doorsDomain[309618]:         at org.dcache.webdav.transfer.CopyFilter.process(CopyFilter.java:276)
Mar 06 18:46:27 dcache.farm.particle.cz dcache@doorsDomain[309618]:         at io.milton.http.FilterChain.process(FilterChain.java:46)
Mar 06 18:46:27 dcache.farm.particle.cz dcache@doorsDomain[309618]:         at io.milton.http.HttpManager.process(HttpManager.java:158)
...

What did I missed? It seems to me to get working HTTP-TPC I stlill have to apply additional workaround

diff --git a/modules/dcache-webdav/src/main/java/org/dcache/webdav/transfer/RemoteTransferHandler.java b/modules/dcache-webdav/src/main/java/org/dcache/webdav/transfer/RemoteTransferHandler.java
index 7a05b62a02..af619572da 100644
--- a/modules/dcache-webdav/src/main/java/org/dcache/webdav/transfer/RemoteTransferHandler.java
+++ b/modules/dcache-webdav/src/main/java/org/dcache/webdav/transfer/RemoteTransferHandler.java
@@ -909,7 +909,7 @@ public class RemoteTransferHandler implements CellMessageReceiver, CellCommandLi
 
                         var attrs =  msg.getFileAttributes();
                         // for CUSTODIAL files on upload client might provide extra information that should be passed to the tape system
-                        if (attrs.getRetentionPolicy() == RetentionPolicy.CUSTODIAL) {
+                        if (attrs.isDefined(FileAttribute.RETENTION_POLICY) && attrs.getRetentionPolicy() == RetentionPolicy.CUSTODIAL) {
                             var archiveMetadata = ServletRequest.getRequest().getHeader("ArchiveMetadata");
                             if (archiveMetadata != null) {
                                 attrs.getStorageInfo().setKey("archive_metadata", archiveMetadata);