There are two ways to transfer data to and from the Radboud Data Repository (RDR). Up- and downloading data via the web portal is the easiest option, since it does not require installing additional software and feels most intuitive. However, if you find that data transfer via the web portal is too slow, consider using a WebDAV client instead.
Please note that collaborating on a collection from a WebDAV client involves certain risks. When multiple people work on the same data files at the same time, data entered may be lost. As such, it is important that only a single user works with the data of a collection at any point in time. We recommend making a single researcher responsible for working with data in the RDR, or to communicate well with your colleagues on who works with the data when.
Transferring data with the web portal
The easiest method to transfer data to or from the Radboud Data Repository (RDR), is via the web portal in the Files tab of the collection's landing page. You can only upload and manage a collection if you are a manager or contributor to the collection in question.
Uploading data
- Go to the collection's landing page and click on the Files tab.
- You now have three options to upload your files (see also the screenshot below):
- You can drag-and-drop files or entire folders into the area under the Files tab buttons.
- Click the Upload files button to upload one or several files.
- Click the Upload folder button to upload a single folder (use option a to upload multiple folders at once).
- The RDR displays the progress of your upload per file. Once all files have been uploaded correctly, a message is displayed that all files have been uploaded. If your upload failed, the RDR lists all files that failed to upload. Simply try again: the RDR will resume the upload from where it was disrupted. If your upload failed because you tried to upload more files than your collection quota allows, you should either clean up your collection by deleting files or request your research administrator to increase the collection quota.
There is an upload limit of 300 GB for a single file. If you want to upload files that are larger than 300 GB, try using WebDAV or contact us at [email protected] for Radboud University employees or [email protected] for Radboudumc employees.
Managing data
To rename an uploaded file or folder:
- Click on the rename icon behind the file or folder name, provide the new file name and click Rename file:
To remove an uploaded file or folder:
- Click on the remove icon behind the file of folder name and press Delete if you are sure you want to delete it:
To create a new folder:
- Click on the button Create folder.
- Enter the folder name and click Create folder:
Downloading data
There are two options to download data from collection.
- To download the entire collection as a tar.gz file click the button Download collection;
- To download individual files or folders click the download icon behind the file or folder name.
Transferring data with a WebDAV client
A number of WebDAV tools are available for transferring data to and from the repository, depending on your operating system. For Windows and MacOS users, we recommend using Cyberduck. Alternatively, Windows users can use YodaDrive. Linux users can mount a network drive. For users with experience in command-line tools that want to perform batch processing on large datasets, we recommend using the command-line tool Repocli, which can be used on most operating systems.
Please note that collaborating on a collection from a WebDAV client involves certain risks. When multiple people work on the same data files at the same time, data entered may be lost. As such, it is important that only a single user works with the data of a collection at any point in time. We recommend making a single researcher responsible for working with data in the RDR, or to communicate well with your colleagues on who works with the data when.
Please do not use Cyberduck versions older than version 4.8.4
Step 1: Install Cyberduck
If you are a Radboud University employee and have access to the Software Centre, you can download CyberDuck there. Otherwise, you can download the installer from the CyberDuck website. Once you have downloaded it, follow the instructions in the installer.
Step 2: Configure Cyberduck
- Open Cyberduck and click on the ‘+’ sign on the bottom left of your screen to create a new bookmark.
- A pop-up window will appear. In this window you must configure the bookmark as follows:
- Select ‘WebDAV (HTTPS)’ from the dropdown menu.
- Nickname: Add a name of your choice for the new connection.
-
Server: Enter webdav.data.ru.nl to access all your collections.
Optional: under More Options in the Path field you can enter the specific path to a collection you are working on. - Port: Select or enter port 443.
- Username and Password: Fill in your data access credentials. You can find these credentials on the top right of the RDR under [Your name] > Data access credentials.
- The configurations are saved to the bookmark automatically. Close the configuration window when you are done.
Step 3: Connect to the RDR
Double-click on the saved bookmark to connect to your RDR collections.
If the WebDAV client (i.e. Cyberduck) is inactive for two hours, you will be logged out automatically by the server. In this case, you will have to connect again.
Step 4: Open your RDR collection
Click on the folder of your Organisational Unit, and then on the folder of your collection. The name of your collection is based on the collection identifier. This identifier can be found in the RDR underneath the title and abstract of your collection:
Step 5: Uploading and downloading files
After opening a collection's directory you can upload or download data as follows:
- Select the file(s) or folder that you want to download or the folder to which you want to upload data.
- Click on the Action button on the menu bar. In the drop-down menu, choose one of the download options (i.e. Download, Download As… or Download To…) or the Upload option to save or upload files:
Step 6: Check uploaded files
You should always check whether your files are uploaded to the RDR. In order to do this, open the RDR in your browser and log in. Find your collection and select it. Under the tab Files you should now see the files you have added. To ensure that the data transfer is complete, you should always perform a checksum prior to deleting files from your original storage system or publication or archiving of your collection in the RDR.
Uploading to the RDR can take some time (up to several hours), especially for large files. Please be patient when handling a lot of and/or large files.
Step 1: Download and install YodaDrive
Download and install YodaDrive from here. You need administrator privileges to complete the installation procedure. If you work on a PC or laptop managed by the Radboud University, please contact the ICT helpdesk to gain temporary administrator privileges.
Step 2: Open YodaDrive
After installation, open YodaDrive by clicking on the system tray icon and choose Manage.
Step 3: Configure YodaDrive
Enter the following information:
- Drive name: Enter a name for your Yoda Network Drive.
- Drive letter: Select a drive letter - any free letter is okay.
- URL: Either enter https://webdav.data.ru.nl to access all your collections or enter the specific WebDAV URL of a collection you are working on.
- Login and Password: Fill in your data access credentials. You can find these credentials on the top right of the RDR under [Your name] > Data access credentials.
- Check the following boxes: Remember Password (do NOT check this box if you are on a publicly accessible PC or laptop), Auto-mount drive and Start with Windows.
Click Save, then Mount and you can now view the Yoda Network Disk in your File Explorer.
Step 4: Open your RDR collection
Click on the folder of your Organisational Unit, and then on the folder of your collection. The name of your collection is based on the collection identifier. This identifier can be found in the RDR underneath the title and abstract of your collection:
Step 5: Work in your Yoda Network Drive
You are currently in the folder that corresponds to the collection you want to work in. You can treat files in this folder just as you would do in ‘regular’ folders in Windows File Explorer.
Step 6: Check uploaded files
You should always check whether your files are uploaded to the RDR. In order to do this, open the RDR in your browser and log in. Find your collection and select it. Under the tab Files you should now see the files you have added. To ensure that the data transfer is complete, you should always perform a checksum prior to deleting files from your original storage system or publication or archiving of your collection in the RDR.
Uploading to the RDR can take some time (up to several hours), especially for large files. Please be patient when handling a lot of and/or large files.
Here we will show you how to configure and use the Radboud Data Repository (RDR) on Linux Ubuntu. We chose Ubuntu as the example, however other desktop distributions of Linux have similar ways of configuring the WebDAV client. If you prefer a command-line based approach, we suggest using Cadaver. When using Cadaver, use the WebDAV URL https://webdav.data.ru.nl instead of the URL mentioned under Step 1.3.
Step 1: Connect to the RDR
- Open the file manager.
- Click on '+ Other Locations' on the left of your screen.
- At the bottom of the screen under Connect to Server, type davs://webdav.data.ru.nl to access all your collections or enter the specific WebDAV URL of a collection you are working on, then click Connect.
Step 2: Enter your credentials
Next, a screen pops up asking for your credentials:
- Username and Password: Fill in your data access credentials. You can find these credentials on the top right of the RDR under [Your name] > Data access credentials.
- Optional: Check the box Remember password until you log out.
Click Connect. You can now view the RDR folder in your file manager.
In order to add data to your RDR collection, simply drop files in the correct folder in your Linux file manager. Steps 3 to 5 will instruct you on how to do this.
Step 3: Open your RDR collection
Click on the folder of your Organisational Unit, and then on the folder of your collection. The name of your collection is based on the collection identifier. This identifier can be found in the RDR underneath the title and abstract of your collection:
Step 4: Work in Ubuntu file manager
You are currently in the folder that corresponds to the collection you want to work in. You can treat files in this folder just as you would do in ‘regular’ folders in Linux.
Step 5: Check uploaded files
You should always check whether your files are uploaded to the RDR. In order to do this, open the RDR in your browser and log in. Find your collection and select it. Under the tab Files you should now see the files you have added. To ensure that the data transfer is complete, you should always perform a checksum prior to deleting files from your original storage system or publication or archiving of your collection in the RDR.
Uploading to the RDR can take some time (up to several hours), especially for large files. Please be patient when handling a lot of and/or large files.
We do not recommend to use Repocli unless you are familiar with command-line tools and want to work with large or many data files. The tool is the standard solution at DCCN's HPC Cluster to transfer data from the central storage to the RDR.
Step 1: Install Repocli
Repocli is already installed on the DCCN HPC Cluster for transferring data between the central storage and the RDR. You can therefore skip this step if you are using DCCN's HPC Cluster.
Go to this site to download the latest version of Repocli: 'repocli' for Linux, 'repocli.darwin' for MacOS and 'repocli.exe' for Windows users.
Next, run the file via a command terminal. See Github for more detailed information.
Step 2: Configure Repocli
The credentials of the RDR should be provided in a configuration file. In order to generate (or overwrite) this file, use the following command:
$ repocli configThe command asks for the following credentials:
- repo baseurl: Either enter https://webdav.data.ru.nl to access all your collections or enter the specific WebDAV URL of a collection you are working on.
- username and password: Fill in your data access credentials. You can find these credentials on the top right of the RDR under [Your name] > Data access credentials.
- save credential [y/N]: depending on if you would like to save your credentials type 'y' (yes) or 'N' (no). Do NOT save your credentials if you are on a publicly accessible PC or laptop.
Step 3: Open your RDR collection
Click on the folder of your Organisational Unit, and then on the folder of your collection. The name of your collection is based on the collection identifier. This identifier can be found in the RDR underneath the title and abstract of your collection:
Step 4: Work in Repocli
Use the repocli commands to manage files in your RDR folder. See https://github.com/Donders-Institute/dr-tools/tree/main/cmd/repocli/here for a list of repocli commands.
Step 5: Check uploaded files
You should always check whether your files are uploaded to the RDR. In order to do this, open the RDR in your browser and log in. Find your collection and select it. Under the tab Files you should now see the files you have added. To ensure that the data transfer is complete, you should always perform a checksum prior to deleting files from your original storage system or publication or archiving of your collection in the RDR.
Uploading to the RDR can take some time (up to several hours), especially for large files. Please be patient when handling a lot of and/or large files.