Data Transfers in the ALCF
This guide outlines methods for transferring data to and from the Argonne Leadership Computing Facility (ALCF). Options include using secure protocols like SFTP, SCP for smaller files, as well as GridFTP and Globus Online for larger transfers. Discover how to securely access your data, utilize compressing and decompressing tools, and initiate transfers easily. ALCF supports the scientific research community with reliable and efficient file transfer services. Get started with useful links for registration and support.
Data Transfers in the ALCF
E N D
Presentation Transcript
Data Transfers in the ALCF Robert Scott Technical Support Analyst Argonne Leadership Computing Facility
Transferring Data To/From ALCF • sftp and scp (for “small” transfers) • If you must use scp, eureka, and in the future tukey are better systems to scp to/from. All paths will be the same as they are on our other systems. • Eureka is also a better host for compressing and uncompressing large file archives • GridFTP (for large transfers) • Other site must accept our CA • ssh / cryptocard access available • For more information on Grid FTP please go here: http://www.alcf.anl.gov/resource-guides/using-gridftp-intrepid • Globus Online (for large transfers) Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy
Getting Started with Globus Online • You’ll need a Globus Online account • Go here to signup: https://www.globusonline.org/ • Globus Connect: to transfer files to and from your local machine (laptop) • SSH key or crypto card to access an ALCF endpoint • Go here to initiate your transfer: https://www.globusonline.org/xfer/StartTransfer Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy
What is Globus Online? • Reliable File Transfer • Easy: Simple signup, login and use – and minimized intervention • Fast: GO can move large filesets in hours • Secure: No Need to worry about security configs or certs, and one-time passwords work • Research-focused: Globus Online is the only file transfer service with the scientific researcher in mind • No IT required • No client software installation • Works with existing GridFTP servers • Globus Connect solves “last mile problem” for local machines • Consolidated support and troubleshooting Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy
Transferring Data To/From ALCF using GO • Globus Online (for large transfers) • Globus Onlineaddresses the challenges faced by researchers in moving, sharing, and archiving large volumes of data among distributed sites. • ALCF BG/P endpoints: alcf#dtn_intrepid, alcf#dtn_surveyor, alcf#dtn • Check if your laboratory, university or research center has already an endpoint. • Globus Connectto transfer files to and from your local machine. • http://www.alcf.anl.gov/resource-guides/data-transfer Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy
Globus Online Demo • Let’s start the Demo…. Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy
Table of BG/Q File Systems on ALCF Resources NOTE: all “filesystems” are also available via the /gpfs/$filesystem_name path on these systems. Argonne Leadership Computing Facility – www.alcf.anl.gov
Thank You! • Questions? Argonne Leadership Computing Facility - supported by the Office of Science of the U.S. Department of Energy
