Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision | |||
doku:copy [2021/05/13 17:48] goldenberg removed |
— (current) | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | < | ||
- | ################################################ | ||
- | ##### start copying process | ||
- | ################################################ | ||
- | |||
- | |||
- | |||
- | #temporary directory on nodes: | ||
- | tmp_dir=$TMPDIR/ | ||
- | |||
- | #naming of the tar files that should be distributed to | ||
- | #the nodes. Each tar file should contain 8 subdirs. | ||
- | |||
- | #file names are completed by appending a number without leading | ||
- | #zeros + tar.gz | ||
- | input_tar_file_base=processes_ | ||
- | |||
- | #naming of the outputfiles; | ||
- | output_tar_base=output_ | ||
- | #define which directories and files should be put into the output tar.gz | ||
- | # not tested yet | ||
- | PACKING=data\* | ||
- | |||
- | start_dir=`pwd` | ||
- | #save output data to this directory | ||
- | data_save=" | ||
- | |||
- | |||
- | |||
- | nodes_uniq=$(cat $TMPDIR/ | ||
- | |||
- | |||
- | #copy files per node | ||
- | #tar.gz files contain 8 subdirectories for each process | ||
- | |||
- | j=0 | ||
- | |||
- | for i in $nodes_uniq | ||
- | do | ||
- | tar_file_name=" | ||
- | echo " | ||
- | ssh $i mkdir -p $tmp_dir | ||
- | echo " | ||
- | ssh $i cp ${start_dir}\/ | ||
- | ssh $i cp -r ${start_dir}\/ | ||
- | echo " | ||
- | ssh $i "cd ${tmp_dir} ;tar -zxf ${tmp_dir}\/ | ||
- | j=$(echo " | ||
- | done | ||
- | |||
- | #command to run: | ||
- | cd ${tmp_dir} | ||
- | time mpirun -mca btl_openib_ib_timeout 20 -machinefile $TMPDIR/ | ||
- | |||
- | #cp files per node back to start directory of job | ||
- | echo " | ||
- | j=0 | ||
- | for i in $nodes_uniq | ||
- | do | ||
- | tar_file_name=" | ||
- | output_tar_file=" | ||
- | echo " | ||
- | echo "ssh $i \" cd ${tmp_dir} ;tar -zcf ${output_tar_file} $PACKING\"" | ||
- | ssh $i " cd ${tmp_dir} ;tar -zcf ${output_tar_file} $PACKING" | ||
- | |||
- | echo " | ||
- | mkdir -p ${data_save} | ||
- | ssh $i cp ${tmp_dir}\/ | ||
- | j=$(echo " | ||
- | done | ||
- | |||
- | # ----------------------------------------------------------------- end-of-file | ||
- | |||
- | |||
- | </ |