This version (2020/10/20 09:13) is a draft.
Approvals: 0/1

Best Practices

  • Article written by Bernhard Hermann (VSC Team) <html><br></html>(last update 2017-04-27 by bh).

Don’t run expensive calculations on login nodes

onedoesnotsimply.jpg

Don’t run expensive calculations on login nodes

  • resources shared by all users
    • expensive tasks slow down everybody
  • tasks will be killed automatically
  • use batch jobs instead
  • very small tasks are OK
    • 500 MB mem, 600 s, 1 core

Make sure your jobs do useful stuff

  • don’t explore the whole parameter space just because…
    • …you can
    • …computing time on VSC is so cheap (it really isn’t!)
  • check your results before submitting heaps of jobs
  • calculate only what will be used

Don’t use I/O

grumpy.jpg

Don’t overuse I/O

  • use few large files over many little ones whenever possible
  • open(), close() outside loops
  • limit seek()
  • use HDF5
  • many short-lived and/or small files → use ${TMPDIR}
    • if you need to keep them, store as archives
  • limit number of simultaneous jobs for I/O-bound calculations
  • pandoc/introduction-to-vsc/11_best_practices/best_practices.txt
  • Last modified: 2020/10/20 09:13
  • by pandoc