User Tools

Site Tools


doku:deep_learning

Deep Learning (DL)

DL and DataLAB community

The DLDC platform is designed to foster the exchange of scientists from different fields in DL. There you will find community meeting announcements and the slides of past meetings. In addition, it is a space to host moderated special interest groups in the field of DL.

DL modules on VSC-3

Tensorflow

Typically on some GPU node,

[someone@n372-003 ~]$  module purge 
[someone@n372-003 ~]$  module load gcc/5.3 mpioff/1.0 bazel/0.12.0 cuda/9.0.176 intel-mkl/2018 hdf5/1.8.18-SERIAL python/3.6 numpy/1.14.3 h5py/2.7.0 scipy/0.18.0 tensorflow/1.8.0.ttest 

Keras

Typically on some GPU node,

[someone@n372-003 ~]$  module purge 
[someone@n372-003 ~]$  module load gcc/5.3 mpioff/1.0 bazel/0.12.0 cuda/9.0.176 intel-mkl/2018 hdf5/1.8.18-SERIAL python/3.6 numpy/1.14.3 h5py/2.7.0 scipy/0.18.0 tensorflow/1.8.0.ttest keras/2.1.6.ttest

Pytorch

Typically on some GPU node,

[someone@n372-003 ~]$  module purge 
[someone@n372-003 ~]$  module load gcc/5.3 cuda/9.1.85 intel-mkl/2018 cmake/3.9.6 python/3.6 numpy/1.14.3 pytorch/1.1.0b

Theano

Typically on some GPU node,

[someone@n372-003 ~]$  module purge 
[someone@n372-003 ~]$  module load gcc/5.3 cuda/9.1.85 intel-mkl/11 cmake/3.8.2 python/2.7 numpy/1.12.0 libgpuarray/0.7.6 scipy/0.17.1 Theano/0.9.0
[someone@n372-003 ~]$  export MKL_THREADING_LAYER=GNU
[someone@n372-003 ~]$  THEANO_FLAGS=device=cuda,floatX=float32 python whatevermyscript.py

Many users nowadays prefer conda-based installs of DL software. For example on some GPU node,

[someone@n372-003 ~]$  module purge 
[someone@n372-003 ~]$  module load anaconda3/5.3.0
[someone@n372-003 ~]$  module show anaconda3/5.3.0 
[someone@n372-003 ~]$  source /opt/sw/x86_64/glibc-2.17/ivybridge-ep/anaconda3/5.3.0/etc/profile.d/conda.sh
[someone@n372-003 ~]$  conda activate base
(base) [someone@n372-003 ~]$  conda list   ( just to see what packages are currently around )
(base) [someone@n372-003 ~]$  conda search tensorflow
(base) [someone@n372-003 ~]$  conda create -n mytf "tensorflow 1.14.0 gpu_py37h4491b45_0"
                                 --> y   ( say yes to the suggested list of packages to install )
(base) [someone@n372-003 ~]$  conda deactivate    ( to get out of the 'base' environment again )
[someone@n372-003 ~]$  conda info --envs
                          --> mytf      /home/lv123456/someone/.conda/envs/mytf
[someone@n372-003 ~]$  conda activate mytf
(mytf) [someone@n372-003 ~]$  python
Python 3.7.blablabla
>>> import tensorflow as tf    ( lots of FutureWarning... but probably not harmful !? )
>>> hello = tf.constant('Hello, TensorFlow!')
>>> sess = tf.Session()
>>> print(sess.run(hello))
>>> <CTRL>D
(mytf) [someone@n372-003 ~]$  conda deactivate
doku/deep_learning.txt · Last modified: 2020/04/24 10:37 by ir