Quick Guide

Tha following rule applies to all users working on the GDC Euler share.

If you are wasting too many resources (CPU, memory and/or runtime), the Cluster Support will contact you. We will contact you if we detect excessive filesystem overhead on the GDC volume. After the second warning, the system will put you into the sustained mode until you comply with our rules.

GDC rules

  1. Respect the terms of use that you accepted when you first logged in to Euler.
  2. Follow the instructions of the GDC bioinformaticians.
  3. Your working directory is the scratch (${SCRATCH}). We do not allow you to do heavy computing (e.g. generating large amounts of data (>100Gb) and/or many inodes (>1000)) in GDC home or GDC projects to keep the filesystem overhead low. Always output data to scratch first, and copy only important files to the GDC volume: Make sure that the files (A) are compressed and (B) many small files (> 500) are archived (tar or zip). Do not install software (e.g. conda environments) on the GDC volume. Do not use more than 13,000 inodes per 1Tb of storage. Keep your folders tidy and do not store redundant data. Remember that there is a storage charge at the end of the year.
  4. Inform the GDC bioinformaticians before copying large amounts of data (> 1 Tb) to GDC home or GDC projects.
  5. Make sure you have a backup of your raw data and scripts. Euler should not be used as a data archive.
  6. We recommend using CPUs and memory according to our proposed usage profiles.
  7. Regularly monitor the resource usage of your jobs and optimze your jobs (running jobs: myjobs, finished jobs: WebGUI, overview about several days: /cluster/software/bin/get_inefficient_jobs). If necessary, stop jobs and adjust the resources required. Be careful if you request a lot of memory (> 60 Gb) and multiple CPUs (> 4 CPUs). On average, CPU, memory and run-time should be at least 50%. Minimise the overhead of job submission by avoiding sending jobs that take less than 5 minutes to complete, and run tests before submitting many jobs.
  8. Consult with the bioinformatician if you plan to submit numerous (> 1,000 jobs), heavy (> 200 Gb of memory or > 24 CPUs), or very long (>120 hours running time) jobs.