Transferring and Managing Project Files

Transferring Files

Uploading and Downloading Local Files Using the Files Tab

Users can upload and download local files by clicking on the "Files" tab to the right of the Workbench.

To upload a file, click on the "Upload File" button next to the "Actions" button. To upload a folder, click on the down arrow to the right of the "Upload File" button, which will reveal the option to upload a folder:

Currently, only one file or folder up to 5GB can be uploaded/downloaded at a time by users. To upload multiple and/or large files, one option is to compress them locally and then extract them on the RCP. Similarly, you can compress files on the RCP prior to downloading them localling. Detailed instructions on can be found in our Compressing and Extracting Files Tutorial. Otherwise, please use one of the methods outlined in the following sections.

Special Note on Uploading Files from the HBSGrid to the RCP Using the Files Tab

As the HBSGrid prepares for retirement in FY27, we recommend creating RCP project spaces to replace projects currently hosted on the HBSGrid. For large data transfers, we recommend using Globus. For smaller transfers, you can use the RCP "Files" interface to upload data directly from the HBSGrid.

1.Log in to the RCP

In NoMachine, open a browser and log in to the RCP.

2.Select the RCP Project Space

Navigate to the RCP project space that will receive the data. Select the "Files" tab. To upload a folder, click the arrow next to "Upload File", then select "Upload Folder".

3.Select and Upload the HBSGrid Folder

Navigate to your HBSGrid project space and select the folder you want to transfer. Click "Upload", then review the selected files and click "Upload" again to confirm.

Using Globus

Important

Directories created via Globus may initially appear as 0-byte files. This happens because folders transferred through Globus are stored in Amazon S3 (your project space) with the content type application/x-globus-dir-placeholder, which S3 treats as a file rather than a directory. After transferring folders through Globus, open each folder once from the RCP Files tab so its content type is updated and recognized as a directory, or, if using an active utility session, restart the session.

Globus, a data transfer service, can be used to transfer multiple files at once and/or files exceeding the 5GB limit. Globus can be used to transfer data into a project space on the RCP or out of the RCP to another location. In order to use Globus, you will need to establish your project as an end point for Globus.

1.Log in to Globus

Log into the Globus transfer application at www.globus.org. You will need to select Harvard University as your institution followed by entering your Harvard Key credentials and password. Please note that only HBS credentials will have access to HBS resources. Credentials from other schools will not work properly.

2.Open the File Manager

Navigate to the file manager tab and locate the Harvard Business School S3 Storage Collection by searching for this collection in the Collection dialog box.

3.Select the HBS S3 Storage Collection and Authenticate

Select the "Harvard Business School S3 Storage" Collection, authenticate (if needed), and then you should see the following dialog:

4.Enter Your AWS IAM Credentials

🛑 Stop! Please treat your AWS Keys as you would a password. Do not store them, share them with others, or expose them to AI.

Enter the AWS IAM Access Key ID and AWS IAM Secret Key. These keys can be generated via the Files tab by clicking on the key button at the top of the page (detailed instructions here).

Click the continue button and you will be brought to this dialog box:

Important

Keys are only valid for 90 days. If your keys expired or you rotated them, follow our instructions for generating new ones.

5.Access Your Project Files

After entering the Key IDs, click on "File Manager," and then you should be able to see the directory listing of your project, as well as be able to upload and download files as shown below.

If you are transferring very large files, we recommend transfer from another Globus endpoint (such as the Harvard Business School DTN, attached to the HBSGrid, or by creating your own endpoint using Globus Personal).

6.Accessing Mulitple Project Folders

If you want to access multiple project folders within RCP using Globus, you will need to add multiple key sets, one for each project storage location. After the default location has been added, use the + button in the "manage credentials" view to add additional keys.

In addition to entering the Access Key ID and Secret Key you need to enter the S3 bucket name.

You can copy the bucket address from the Files tab within RCP.

Only the bucket name is needed instead of the entire bucket URN. So for example, the bucket with the URN of "s3://rcp-project-rcptesting-owzh-projectstorage/ProjectStorage/" would only require "rcp-project-rcptesting-owzh-projectstorage". Also note that bucket name will not be the same as the storage location name within RCP. Entering several project storage locations will result in multiple entries in the "manage credentials" tab.

and each project storage location will be visible as a separate folder with the File Manager tab.

Special Note on Uploading Files from the HBSGrid to the RCP Using Globus

As the HBSGrid prepares for retirement in FY27, we recommend creating RCP project spaces to replace projects currently hosted on the HBSGrid. For large data transfers from an HBSGrid project space to the RCP we recommend using Globus.

1. Copy your HBSGrid project space folder or files to the Globus staging area

Globus requires files to be staged in the /export/globus HBSGrid folder before they can be transferred to RCP, so you'll first need to copy your HBSGrid files there.

In NoMachine, open the file explorer and navigate to your project space folder. Select the folder you'd like to move to RCP, right click and select "Copy to."

Select the /export/globus folder as your destination, and then either create a new folder to house the data or click on an existing one.

Note: you can also move the data into the /export/globus staging area using Terminal commands (e.g., cp -r /path/to/project-folder /export/globus/<your-folder-name>/

The files must be located in /export/globus/<your-folder-name> before you begin the Globus transfer.

2. Open the HBS DTN collection in Globus

Log in to Globus and navigate to the File Manager tab. In the Collection dialog box, search for and select the Harvard Business School DTN Collection. Click Transfer or Sync to…, then navigate to and select the folder containing your staged HBSGrid data.

3. Connect to the RCP project space

On the right-hand side of the screen, connect to your RCP project space. If you have not already done so, follow the instructions above: search for the Harvard Business School S3 Storage Collection, authenticate, and enter your AWS keys.

Navigate to or create the destination folder in your RCP project space where you would like the files to be transferred.

4. Start the transfer

Select the folder you wish to transfer and click on the Start arrow icon. For this workflow, files should transfer from the Harvard Business School DTN collection to the Harvard Business School S3 Storage collection.

5. Verify the transfer

Monitor the transfer status in Globus and confirm that the files appear in the expected RCP location. Do not delete the files from /export/globus until the transfer has completed successfully and the files have been verified.

If you need help at any point along the way, please contact research@hbs.edu.

Accessing AWS S3 Access Keys

Each RCP project has its own project storage bucket. You must generate separate S3 credentials for each project you are assigned to. The generated AWS S3 Access Keys are valid for 90 days from the time they are generated. After they expire, they can no longer be used to access the project bucket or transfer files through Globus.

🛑 Stop! Please treat your AWS Keys as you would a password. Do not store them, share them with others, or expose them to AI.

Generate S3 credentials

In the RCP project space, select the Files tab. Click on the Key icon next to the bucket name.

The S3 Credentials window opens.

Select Generate Credentials. The system generates an Access Key ID and Secret Access Key.

Important

The Secret Access Key is displayed only once. Copy both credentials before closing the window. You cannot retrieve the Secret Access Key afterward.

Add the credentials to Globus

Copy the Access Key ID and Secret Access Key directly into Globus.

The credentials window also displays the expiration date. RCP provides status notifications as credentials approach expiration.

Credentials may appear in the following states:

• Active — More than 30 days left
• Active — 8 to 30 days left
• Active — 7 days or fewer left
• Expired credentials

When credentials are close to expiring, select Rotate Credentials to generate a new set. Rotating credentials immediately invalidates the previous credentials, so update Globus and any other tools that use them.

Troubleshooting

If an error occurs while generating credentials, select Try Again after a short wait

If the error persists, please contact RCS.

Update expired or rotated credentials

RCP S3 credentials expire after 90 days. They also become invalid immediately when credentials are rotated.

When the credentials expire or are rotated: 1. Generate a new Access Key ID and Secret Access Key in RCP. 2. Return to the Globus collection’s Credentials tab. 3. Add the new credentials and the project bucket name. 4. Remove the old credentials. 5. If the old credentials are not updated, Globus may be unable to access the project folder or transfer files.

Managing Source Code - Syncing to Git

All launchers for IDEs (including Stata, VSCode, RStudio, Jupyter and Spyder) have git installed on their base images. To use git for sessions using these launchers, please use the terminal window for each to execute git commands.

For a list of common git commands see the Git Hub Cheat Sheet.

If you are brand new to code code version control with git, please visit this useful guide.

If you are using Harvard’s Github enterprise for your repositories the standard login and password do not work and you will need to use either ssh keys or tokens. Please see 1. https://github.com/settings/tokens for using tokens
2. https://github.com/settings/ssh for ssh

If you have any questions or run into issues, please reach out to research@hbs.edu.