Table of Contents

Creating and managing projects at VSC

Creating a new project

Login to service page

Go to our service page and on the left side (Login for project managers), choose your organization. If it is not there, please contact us for an external access. Then log in using the credential from your organization.

Note: If your organization is not in our list, go to https://idp.vsc.ac.at/simplesaml/module.php/selfregister/ and choose 'Register new account'. After you have done that, please tell us the username you created and we will add it to our system. Then you can use “IdP VSC” as organization to log in.

Request new project

Click on 'request project' in the left menu bar.

Project types

Add project details

After the creation, you get assigned a unique project ID and a corresponding project page. You affiliation (the organization you are from) will be automatically added, and you will be added as the manager of the project. You will also get an email about the project creation that includes further steps to do.

Now you need to choose a resource pool at the item 'Request resources from a resource pool'. This basically means the hardware your project will be running on. The project can run on several of these pools at the same time. The current options are:

Click on 'request new project part and enter the requirements:

Then click 'Submit Query'. This will create a project part for the selected resource pool.

Resource types

  1. cpu contingent: The CPU core hours that are estimated to be needed for the project.
    This can be extended at any time during the project runtime. For testprojects, this is fixed to 100000, with extensions only under special circumstances. In private projects, the core-h are not accounted for, so the number is not relevant. In external projects, this must not be higher than the number purchased.
  2. project end: The runtime of the project in days, counted from the beginning date.
    This can be extended at any point, even after project expirations (to reactivate a project). The default is 366 days, and projects are only granted 1 year at a time. Private projects are an exception and are usually granted for several years. Testprojects are set to 30 days, with a limited amount of extension possible.
  3. $DATA inodes: The number of inodes in the DATA storage volume.
    This correspondents to the number of files. The default is one million, which can be extended at any time.
  4. $DATA size: The size, in GB, of the DATA storage volume.
    The default is 10TB, and it can be extended to 100TB without any query. Higher values need to be discussed with us.
  5. $HOME inodes: The number of inodes in the HOME storage volume.
    This correspondents to the number of files. The default is one million, which can be extended at any time.
  6. $HOME size: The size, in GB, of the HOME storage volume.
    The default is 100GB, which will never be extended except in some special situations.

Upload files

Next, unless it is a testproject or a private project, you need to upload some files at the section 'Files':

For every project type except the mentioned ones, upload a short project description (2-4 pages).
For a funded project, upload a proof of funding.
For an internal project, upload a list of at least 3 possible reviewers. They can be from anywherere except your department.
For an external project, upload either a quota for core hours or an appropriate contract about calculation times.
Additionally, if you chose yes for the GDPO data, upload a data handling contract with the TU Vienna.

You get a confirmation mail for every uploaded file. Once this is done (or directly, if no files are needed), click on the red bar at the top of the new project part to request technical review.

Review process

tech approval

All projects are first checked by the VSC team to assess if it can actually be run on the systems, if all project data are set correctly or if any other administrative or technical issue is still open. At that stage, the VSC team also creates the files directories for the project.

peer review (optional)

Projects of the type internal go through a peer review - the project description is sent out to the mentioned reviewers, ans their answers are then incorporated. If they are positive, the project goes to the next stage.

SC (steering committee) review

Projects of the type funded, internal (after peer review), external and application are sent to the SC for review. Depending on the requested amount of core-h, this is either just the SC members of the requesting organization (if below 2 Mio core-h) or the entire SC (above 2 Mio core-h). Projects of the type test or private only have the tech review.

Once the project is granted, you will get a mail notification

Adding users

Once a project is granted, user can be created for it. These are the actual cluster suers and independent of any users on the service page.

Click on 'create cluster user account' and fill in First and Last name, an email address and a phone number. The phone number can be from any country, but it has to be able to receive text messages. If such a number is not available, please contact us
Then click 'submit' and enter a username and a password. If you have project parts on several resource pools, you can activate or deactivate the user there via checkboxes. Then click 'create account'.
The username must be unique within the entire VSC. The password must be 10 charactes, with at least one lower case letter, one upper case letter, one digit and one special character.

The user gets an email with some informations and instructions. The password is not contained in that mail and has to be conveyed to the user in some other way.

A project can contain any number of users, even multiple one for the same real person, but email and phone number have to be unique within the project.

As soon as users are created, these can log into the cluster and start working on the project.

Project changes and extensions

At any time while the project is running, various resources can be changed or extended

Changes in the project

The following items can be changed by the project manager via the project page:

Project extensions

At 'Request resources from a resource pool' the project can be extended to another part of our systems. See the drop-down list for the possible options

To extend some of the resources of an existing resource pool, click on 'request more resources' at the top of that part. On the opened page, set those resource types you want to extend to 'yes' (leave all others on 'no') and enter the value you want to have ADDED to the current amount. You can add an explanatory comment as well. Afterwards click on submit.
The following resource types can be extended:

Changes done by the VSC team

Some changes in the project can only be done by the VSC team. Please contact us if that is needed.
These changes are for example the project manager, project name, project group memberships (access to certain software or volumes for the entire project)
The VSC team can also add secretaries to a project - these are persons that have similar rights as the project manager. In contrast to the manager, several secretaries are possible.

Project lifecycle

During the runtime of a project, you will or might get various reminder or warning mails:

core hours

There will be a first warning mail when the core hours in your project are 90% used up. Please note that the mail is sent at a fixed time of the day, which means in the time between crossing the 90% threshold and the mail being sent, the use might already be higher.

When the core-h are used up to 100%, there will be another mail. This will also result in your project being set to idle priority. This means only the idle queues on the cluster are available. These queues have a low priority and only allow jobs to run, when no one else needs the respective nodes. Which in turn greatly impacts the chance for running jobs.

It is then highly recommended to request an extension. Funded and internal projects can always extend as long as they are running, external projects need an appropriate quote. In testprojects, core-h are only extended in special circumstances. Note that private projects do not count core-h at all.

number of files

When the number of files (which equals the number of inodes) in either \$HOME or \$DATA reaches 90% of the allowed quota, a warning mail is sent. As with core-h, the actual value can be higher as the mail is sent some time after the threshold is reached.

Another mail is sent upon reaching 100% - at that point, no more files can be created in the affected volume, preventing further work or sometimes even login. In that case, delete any files not needed and if that is not sufficient, request an extension. Any type of project can get an extension in files.

Note: the volumes \$HOME and \$FILE are handled independently, so mails can be received for both.

volume size

When the size of the data in either \$HOME or \$DATA reaches 90% of the allowed quota, a warning mail is sent. As with core-h, the actual value can be higher as the mail is sent some time after the threshold is reached.

Another mail is sent upon reaching 100% - at that point, no more files can be created in the affected volume, preventing further work or sometimes even login. In that case, delete any files not needed and if that is not sufficient, request an extension. Any type of project can get an extension in size on \$DATA up to 100TB. Higher values are possible after discussion with us. \$HOME cannot be extended, except for very special circumstances, such as more than 20 users in a project. Therefore, put all research data in $DATA.

Note: the volumes \$HOME and \$FILE are handled independently, so mails can be received for both.

Project expiry

Expiry: About a month before a project expires, you are sent a notification mail about said expiry. Another mail is sent on the day of expiry. As soon as a project is expired, it is still possible to work normally, including job submission. However, new user can no longer be created.

Job submission disabled: One month after project expiry, a mail is sent about job submissions being disabled. From that point on, jobs can no longer be submitted to the clusters. Jobs that are running at that time are not broken off.

Backup relevant project data: Two month after project expiry, a mail is sent to warn you to backup any data from your project that you want to keep. The mail says data retention is only guaranteed until three months after project expiry. For details about data see the next paragraph.

Login disabled: Three month after job submission, a mail is sent informing about login being disabled. At that point, all user logins to the clusters belonging to that project are disabled. So far, all data belonging to a project was kept indefinitely after the project ran out - but this can not be guaranteed as total storage space might run out. It is therefore highly recommended to back up all relevant data before that point.

If the project shall continue, an extension can be requested. That can even be done after the project has run out. Testprojects can only be extended for short times (30 - 60 days usually), other projects for a maximum of one year at a time.

Auto-extension: If a project (that is neither test nor private) has a large amount of core-h left available, it is automatically extended once for 6 month.