3.5. Replica Management

The replica management module provides an interface to (distributed) data replication services, like for example iRODS.

The basic usage of the replica module is as follows:

myfile = saga.replica.LogicalFile("irods://localhost/"+TEMP_FILENAME)
myfile.add_location("irods:////data/cache/AGLT2_CE_2_FTPplaceholder/whatever?resource=AGLT2_CE_2_FTP")

mydir = saga.replica.LogicalDirectory("irods://localhost/" + IRODS_DIRECTORY)
mydir.make_dir("anotherdir")

Like all SAGA modules, the replica module relies on middleware adaptors to provide bindings to a specific resource manager. Adaptors are implicitly selected via the scheme part of the URL, e.g., local:// in the example above selects the local replica adaptor.

Note

A list of available adaptors and supported resource managers can be found in the Developer Documentation part of this documentation.

The rest of this section is structured as follows:

3.5.1. Flags

The following constants are defined as valid flags for logical file and directory methods:

saga.filesystem.OVERWRITE
saga.filesystem.RECURSIVE
saga.filesystem.CREATE
saga.filesystem.CREATE_PARENTS
saga.filesystem.LOCK
saga.filesystem.EXCLUSIVE
saga.filesystem.DEREFERENCE

3.5.2. Logical File – saga.replica.LogicalFile

class saga.replica.LogicalFile(url=None, flags=512, session=None, _adaptor=None, _adaptor_state={}, _ttype=None)[source]

Bases: saga.namespace.entry.Entry, saga.attributes.Attributes

__init__(url=None, flags=READ, session=None)[source]

url: saga.Url flags: flags enum session: saga.Session ret: obj

is_file()[source]

ttype: saga.task.type enum ret: bool / saga.Task

get_size()[source]

Return the size of the file.

ttype: saga.task.type enum ret: int / saga.Task

Returns the size of the physical file represented by this logical file (in bytes)

Example:

# get a file handle
lf = saga.replica.LogicalFile("irods://localhost/tmp/data/data.bin")

# print the logical file's size
print lf.get_size ()
add_location(name)[source]

Add a physical location.

name: saga.Url ttype: saga.task.type enum ret: None / saga.Task

remove_location(name)[source]

Remove a physical location.

name: saga.Url ttype: saga.task.type enum ret: None / saga.Task

update_location(old, new)[source]

Updates a physical location.

old: saga.Url new: saga.Url ttype: saga.task.type enum ret: None / saga.Task

list_locations()[source]

List all physical locations of a logical file.

ttype: saga.task.type enum ret: list [saga.Url] / saga.Task

replicate(name)[source]

Replicate a logical file.

name: saga.Url flags: flags enum ttype: saga.task.type enum ret: None / saga.Task

upload(name, tgt=None, flags=None)[source]

Upload a physical file.

name: saga.Url tgt: saga.Url flags: flags enum ttype: saga.task.type enum ret: None / saga.Task

download(name, src=None, flags=None)[source]

Download a physical file.

name: saga.Url src: saga.Url flags: flags enum ttype: saga.task.type enum ret: None / saga.Task

3.5.3. Logical Directory – saga.replica.LogicalDirectory

class saga.replica.LogicalDirectory(url=None, flags=512, session=None, _adaptor=None, _adaptor_state={}, _ttype=None)[source]

Bases: saga.namespace.directory.Directory, saga.attributes.Attributes

__init__(url, flags=READ, session=None)[source]

Create a new Logical Directory instance.

url: saga.Url flags: flags enum session: saga.Session ret: obj

is_file(tgt=None)[source]

tgt: saga.Url / string ttype: saga.task.type enum ret: bool / saga.Task

open(tgt, flags=READ)[source]

tgt: saga.Url flags: saga.namespace.flags enum ttype: saga.task.type enum ret: saga.namespace.Entry / saga.Task

open_dir(tgt, flags=READ)[source]
Parameters:
  • tgt – name/path of the directory to open
  • flags – directory creation flags

ttype: saga.task.type enum ret: saga.namespace.Directory / saga.Task

Open and return a new directoy

The call opens and returns a directory at the given location.

Example:

# create a subdir 'data' in /tmp
dir = saga.namespace.Directory("sftp://localhost/tmp/")
data = dir.open_dir ('data/', saga.namespace.Create)
get_size(tgt)[source]

tgt: logical file to get size for ttype: saga.task.type enum ret: int / saga.Task

Returns the size of the physical file represented by the given logical file (in bytes)

Example:

# get a logical directory handle
lf = saga.replica.LogicalFile("irods://localhost/tmp/data/")

# print a logical file's size
print lf.get_size ('data.dat')
find(name_pattern, attr_pattern=None, flags=RECURSIVE)[source]

name_pattern: string attr_pattern: string flags: flags enum ttype: saga.task.type enum ret: list [saga.Url] / saga.Task