Data Identifier Methods¶
-
class
rucio.client.didclient.
DIDClient
(rucio_host=None, auth_host=None, account=None, ca_cert=None, auth_type=None, creds=None, timeout=600, user_agent='rucio-clients')¶ Bases:
rucio.client.baseclient.BaseClient
DataIdentifier client class for working with data identifiers
-
ARCHIVES_BASEURL
= 'archives'¶
-
DIDS_BASEURL
= 'dids'¶
-
add_container
(scope, name, statuses=None, meta=None, rules=None, lifetime=None)¶ Add data identifier for a container.
Parameters: - scope – The scope name.
- name – The data identifier name.
- statuses – Dictionary with statuses, e.g.g {‘monotonic’:True}.
- meta – Meta-data associated with the data identifier is represented using key/value pairs in a dictionary.
- rules – Replication rules associated with the data identifier. A list of dictionaries, e.g., [{‘copies’: 2, ‘rse_expression’: ‘TIERS1’}, ].
- lifetime – DID’s lifetime (in seconds).
-
add_containers
(cnts)¶ Bulk add containers.
Parameters: cnts – A list of containers.
-
add_containers_to_container
(scope, name, cnts)¶ Add containers to container.
Parameters: - scope – The scope name.
- name – The dataset name.
- dsns – The content.
-
add_containers_to_containers
(attachments)¶ Add containers_to_containers.
Parameters: attachments – The attachments. attachments is: [attachment, attachment, …] attachment is: {‘scope’: scope, ‘name’: name, ‘dids’: dids} dids is: [{‘scope’: scope, ‘name’: name}, …]
-
add_dataset
(scope, name, statuses=None, meta=None, rules=None, lifetime=None, files=None, rse=None)¶ Add data identifier for a dataset.
Parameters: - scope – The scope name.
- name – The data identifier name.
- statuses – Dictionary with statuses, e.g.g {‘monotonic’:True}.
- lifetime – DID’s lifetime (in seconds).
- files – The content.
- rse – The RSE name when registering replicas.
Meta: Meta-data associated with the data identifier is represented using key/value pairs in a dictionary.
Rules: Replication rules associated with the data identifier. A list of dictionaries, e.g., [{‘copies’: 2, ‘rse_expression’: ‘TIERS1’}, ].
-
add_datasets
(dsns)¶ Bulk add datasets.
Parameters: dsns – A list of datasets.
-
add_datasets_to_container
(scope, name, dsns)¶ Add datasets to container.
Parameters: - scope – The scope name.
- name – The dataset name.
- dsns – The content.
-
add_datasets_to_containers
(attachments)¶ Add datasets_to_containers.
Parameters: attachments – The attachments. attachments is: [attachment, attachment, …] attachment is: {‘scope’: scope, ‘name’: name, ‘dids’: dids} dids is: [{‘scope’: scope, ‘name’: name}, …]
-
add_did
(scope, name, type, statuses=None, meta=None, rules=None, lifetime=None, dids=None, rse=None)¶ Add data identifier for a dataset or container.
Parameters: - scope – The scope name.
- name – The data identifier name.
- statuses – Dictionary with statuses, e.g.g {‘monotonic’:True}.
- lifetime – DID’s lifetime (in seconds).
- dids – The content.
- rse – The RSE name when registering replicas.
Paran type: The data identifier type (file|dataset|container).
Meta: Meta-data associated with the data identifier is represented using key/value pairs in a dictionary.
Rules: Replication rules associated with the data identifier. A list of dictionaries, e.g., [{‘copies’: 2, ‘rse_expression’: ‘TIERS1’}, ].
-
add_did_meta
(scope, name, meta)¶ Insert metadata to the json column of a did, updates key if already present
Parameters: - scope – the scope of did
- name – the name of the did
- meta – the metadata to be inserted or updated(in json format)
-
add_dids
(dids)¶ Bulk add datasets/containers.
-
add_files_to_archive
(scope, name, files)¶ Add files to archive.
Parameters: - scope – The scope name.
- name – The dataset name.
- files – The content.
-
add_files_to_dataset
(scope, name, files, rse=None)¶ Add files to datasets.
Parameters: - scope – The scope name.
- name – The dataset name.
- files – The content.
- rse – The RSE name when registering replicas.
-
add_files_to_datasets
(attachments, ignore_duplicate=False)¶ Add files to datasets.
Parameters: - attachments – The attachments. attachments is: [attachment, attachment, …] attachment is: {‘scope’: scope, ‘name’: name, ‘dids’: dids} dids is: [{‘scope’: scope, ‘name’: name}, …]
- ignore_duplicate – If True, ignore duplicate entries.
-
add_temporary_dids
(dids)¶ Bulk add temporary data identifiers.
Parameters: dids – A list of dids.
-
attach_dids
(scope, name, dids, rse=None)¶ Attach data identifier.
Parameters: - scope – The scope name.
- name – The data identifier name.
- dids – The content.
- rse – The RSE name when registering replicas.
-
attach_dids_to_dids
(attachments, ignore_duplicate=False)¶ Add dids to dids.
Parameters: attachments – The attachments. attachments is: [attachment, attachment, …] attachment is: {‘scope’: scope, ‘name’: name, ‘dids’: dids} dids is: [{‘scope’: scope, ‘name’: name}, …] :param ignore_duplicate: If True, ignore duplicate entries.
-
close
(scope, name)¶ close dataset/container
Parameters: - scope – The scope name.
- name – The dataset/container name.
-
create_did_sample
(input_scope, input_name, output_scope, output_name, nbfiles)¶ Create a sample from an input collection.
Parameters: - input_scope – The scope of the input DID.
- input_name – The name of the input DID.
- output_scope – The scope of the output dataset.
- output_name – The name of the output dataset.
- account – The account.
- nbfiles – The number of files to register in the output dataset.
-
delete_did_meta
(scope, name, key)¶ Delete a key from the metadata column
Parameters: - scope – the scope of did
- name – the name of the did
- key – the key to be deleted
-
delete_metadata
(scope, name, key)¶ Delete data identifier metadata
Parameters: - scope – The scope name.
- name – The data identifier.
- key – the key.
-
detach_dids
(scope, name, dids)¶ Detach data identifier
Parameters: - scope – The scope name.
- name – The data identifier name.
- dids – The content.
-
get_dataset_by_guid
(guid)¶ Get the parent datasets for a given GUID. :param guid: The GUID.
Returns: A did
-
get_did
(scope, name)¶ Retrieve a single data identifier.
Parameters: - scope – The scope name.
- name – The data identifier name.
-
get_did_meta
(scope, name)¶ Get all metadata for a given did :param scope: the scope of did :param name: the name of the did
-
get_metadata
(scope, name)¶ Get data identifier metadata
Parameters: - scope – The scope name.
- name – The data identifier name.
-
list_archive_content
(scope, name)¶ List archive contents.
Parameters: - scope – The scope name.
- name – The data identifier name.
-
list_associated_rules_for_file
(scope, name)¶ List the associated rules a file is affected from..
Parameters: - scope – The scope name.
- name – The file name.
-
list_content
(scope, name)¶ List data identifier contents.
Parameters: - scope – The scope name.
- name – The data identifier name.
-
list_content_history
(scope, name)¶ List data identifier contents history.
Parameters: - scope – The scope name.
- name – The data identifier name.
-
list_did_rules
(scope, name)¶ List the associated rules of a data identifier.
Parameters: - scope – The scope name.
- name – The data identifier name.
-
list_dids
(scope, filters, type='collection', long=False, recursive=False)¶ List all data identifiers in a scope which match a given pattern.
Parameters: - scope – The scope name.
- filters – A dictionary of key/value pairs like {‘name’: ‘file_name’,’rse-expression’: ‘tier0’}.
- type – The type of the did: ‘all’(container, dataset or file)|’collection’(dataset or container)|’dataset’|’container’|’file’
- long – Long format option to display more information for each DID.
- recursive – Recursively list DIDs content.
-
list_dids_by_meta
(scope=None, select={})¶ Gets all dids matching the values of the provided metadata keys :param scope: the scope of the search :param select: the key value pairs to search with(query in json format)
-
list_files
(scope, name, long=None)¶ List data identifier file contents.
Parameters: - scope – The scope name.
- name – The data identifier name.
- long – A boolean to choose if GUID is returned or not.
-
list_parent_dids
(scope, name)¶ List parent dataset/containers of a did.
Parameters: - scope – The scope.
- name – The name.
-
resurrect
(dids)¶ Resurrect a list of dids.
Parameters: dids – A list of dids [{‘scope’: scope, ‘name’: name}, …]
-
scope_list
(scope, name=None, recursive=False)¶ List data identifiers in a scope.
Parameters: - scope – The scope name.
- name – The data identifier name.
- recursive – boolean, True or False.
-
set_metadata
(scope, name, key, value, recursive=False)¶ Set data identifier metadata
Parameters: - scope – The scope name.
- name – The data identifier name.
- key – the key.
- value – the value.
- recursive – Option to propagate the metadata change to content.
-
set_status
(scope, name, **kwargs)¶ Set data identifier status
Parameters: - scope – The scope name.
- name – The data identifier name.
- kwargs – Keyword arguments of the form status_name=value.
-