The “remote mover” concept describes an HPSS installation which includes a mover node at a remote site. NERSC and ORNL have tested two configurations -- one in which an ORNL node is part of a NERSC HPSS installation and the converse -- a NERSC node is part of an ORNL HPSS installation. We have also established a configuration in which a single node is “remote” to both installations. The benefit of the remote mover concept is that files are transferred between the sites under the control of HPSS software as a “migration” from one level of storage to a lower (and remote) level. The user does not have to wait for the transfer to complete it takes place “behind the scenes”.

The tests have been successful and the concept is valuable. At the moment the two production HPSS installations are at different HPSS releases, which may lead to operational difficulties, so we have agreed to wait until we are both at the same HPSS release level to revisit deploying remote movers in production.

The initial testing above had an ORNL mover sited at NERSC; this project reversed that configuration.

In the earlier project, only one stripe was used in the network transfer, and it proved to be rather slow because the transfer uses the "mover protocol". The mover protocol includes several handshakes which prove to be very expensive when network latency is high (as it is between East Tennessee and Oakland, California).

In this project we allowed the mover to send several stripes in parallel. Although no individual stripe was fast, the aggregate demonstrated much-improved speed.

Back to Projects