Small bug in line 658 in src/backprojector.cpp:
change:
double myfsc = XMIPP_MAX(0., DIRECT_A1D_ELEM(fsc, i));
to:
double myfsc = XMIPP_MAX(0.001, DIRECT_A1D_ELEM(fsc, i));
on VSC-1 only gcc works currently well. Intel 10.x and 11.x have problems with some part of the code.
CPPFLAGS="-I/usr/include " \ CFLAGS="-L/usr/lib64 -lpthread" \ LDFLAGS="-L/usr/lib64 -lpthread" \ CXXFLAGS="-L/usr/lib64 -lpthread" \ MPICXX="mpicxx -DMPICH_IGNORE_CXX_SEEK " \ CXX=g++ \ CC="gcc" CPPFLAGS="-I/usr/include " \ CFLAGS="-L/usr/lib64 -lpthread" \ ./configure --enable-mpi \ --prefix=/home/lv70172/blaas/software/relion-1.1
Configure: (Note '\' characters are on all lines, i.e. following paragraph is treated like one single line by bash):
With INTEL MPI + IntelCompiler:
CPPFLAGS="-I/opt/fftw/3.3/intel_12.1_O3_pic/include " \ CFLAGS="-L/opt/fftw/3.3/intel_12.1_O3_pic/lib -lpthread" \ LDFLAGS="-L/opt/fftw/3.3/intel_12.1_O3_pic/lib -lpthread" \ CXXFLAGS="-L/opt/fftw/3.3/intel_12.1_O3_pic/lib -lpthread" \ MPICXX="mpicxx -DMPICH_IGNORE_CXX_SEEK " \ CXX=icpc \ CC="icc" CPPFLAGS="-I/opt/fftw/3.3/intel_12.1_O3/include " \ CFLAGS="-L/opt/fftw/3.3/intel_12.1_O3/lib -lpthread" \ ./configure --enable-mpi \ --prefix=/home/lv70172/blaas/software/relion-1.1
With gcc and intelmpi, first use INSTALL.sh to compile fftw for relion itself. FFTW on VSC-2 althoug configured very similarly, did not work here:
CPPFLAGS="-I/home/lv70172/blaas/software/relion-1.1_gcc/include " \ CFLAGS="-L/home/lv70172/blaas/software/relion-1.1_gcc/lib -lpthread" \ LDFLAGS="-L/home/lv70172/blaas/software/relion-1.1_gcc/lib -lpthread" \ CXXFLAGS="-L/home/lv70172/blaas/software/relion-1.1_gcc/lib -lpthread" \ MPICXX="mpigxx -DMPICH_IGNORE_CXX_SEEK" \ CXX=g++ \ CC="gcc" CPPFLAGS="-I/home/lv70172/blaas/software/relion-1.1_gcc/include " \ CFLAGS="-L/home/lv70172/blaas/software/relion-1.1_gcc/lib -lpthread" \ ./configure --enable-mpi \ --prefix=/home/lv70172/blaas/software/relion-1.1_gcc
Then:
make make install
For installing the GUI edit INSTALL.sh:
BUILD_FFTW=false BUILD_FLTK=true BUILD_RELION=false BUILD_RELION_GUI=true
Finally execute:
./INSTALL.sh
When running jobs on the queue, place a file 'qsub.bash' in the 'gui' directory of relion:
#!/bin/bash #$ -pe XXXqueueXXX XXXmpinodesXXX #$ -e XXXerrfileXXX #$ -o XXXoutfileXXX #$ -cwd #$ -V #needed on VSC-1: export IPATH_NO_CPUAFFINITY=1 mpirun -machinefile $TMPDIR/machines -np XXXmpinodesXXX XXXcommandXXX
Settings in the GUI:
Compile FFTW from relion sources, because VSC FFTW showed some errors:
in INSTALL.sh set:
BUILD_FFTW=true BUILD_FLTK=false BUILD_RELION=false BUILD_RELION_GUI=false
execute:
#might be needed to remove path to pgi because fftw does not compile, at least on vsc-1 PATH=`echo $PATH | sed -e s?/opt/sw/pgi/linux86-64/13.7/bin:??` ./INSTALL.sh
Somewhere a missing 'include' directive. Add to 'src/mpi.cpp' at the very beginning of the file:
#include <stdio.h>
Configure and install Relion: RELION_PREFIX should be the source directory of the current version
For gcc use:
#export RELION_PREFIX=/home/lv70172/blaas/software/relion-1.2_gcc
export RELION_PREFIX=`pwd`
CPPFLAGS="-I${RELION_PREFIX}/include " \
CFLAGS="-L${RELION_PREFIX}/lib -lpthread" \
LDFLAGS="-L${RELION_PREFIX}/lib -lpthread" \
CXXFLAGS="-L${RELION_PREFIX}/lib -lpthread" \
MPICXX="mpigxx -DMPICH_IGNORE_CXX_SEEK" \
CXX=g++ \
CC="gcc" CPPFLAGS="-I${RELION_PREFIX}/include " \
CFLAGS="-L${RELION_PREFIX}/lib -lpthread" \
./configure --enable-mpi \
--prefix=${RELION_PREFIX}
make
make install
For the Intel compiler first compile and install the FFTW libraries by editing the following lines in INSTALL.sh:
BUILD_FFTW=true BUILD_FLTK=false BUILD_RELION=false BUILD_RELION_GUI=false ... #################### FFTW ########################### if $BUILD_FFTW; then echo -e "$GREEN Compiling $VFFTW ...$ENDC" echo -e "See $RELION_HOME/external/fftw_build.log for details" cd external tar -zxf $VFFTW.tar.gz cd $VFFTW CC=icc CPP=cpp MPICC=mpiicc F77=ifort ./configure --enable-openmp --enable-shared prefix=$PREFIX > $RELION_HOME/external/fftw_build.log make $N_THREADS >> $RELION_HOME/external/fftw_build.log make install >> $RELION_HOME/external/fftw_build.log cd ../.. fi
Then run:
./INSTALL.sh
Then execute
export RELION_PREFIX=`pwd`
CPPFLAGS="-I${RELION_PREFIX}/include "\
CFLAGS="-L${RELION_PREFIX}/lib -openmp -lpthread -liomp5" \
LDFLAGS="-L${RELION_PREFIX}/lib -lpthread" \
CXXFLAGS="-L${RELION_PREFIX}/lib -openmp -lpthread -liomp5" \
MPICXX="mpiicpc -DMPICH_IGNORE_CXX_SEEK" \
CXX=icpc \
CC="icc" \
CPPFLAGS="-I${RELION_PREFIX}/include " \
CFLAGS="-L${RELION_PREFIX}/lib -openmp -lpthread -liomp5" \
./configure --enable-mpi --prefix=${RELION_PREFIX}
make
make install
Finally recompile ctf by editing the appropriate Makefile in software/ctf:
all: ctffind3.exe ctftilt.exe
ctffind3.exe: ctffind3.o iof.o ioc.o
ifort -xHost -openmp -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE \
ctffind3.o iof.o ioc.o -o ctffind3.exe
ctftilt.exe: ctftilt.o iof.o ioc.o
ifort -xHost -openmp -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE \
ctftilt.o iof.o ioc.o -o ctftilt.exe
\rm *.o
ctffind3.o: ctffind3.f
ifort -xHost -openmp \
-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE \
-w -c ctffind3.f
ctftilt.o: ctftilt.f
ifort -xHost -openmp \
-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE \
-w -c ctftilt.f
iof.o: iof.f
ifort -xHost -openmp \
-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE \
-w -c iof.f
ioc.o: ioc.c
icc -DPROTOTYPE -Dalliant -D_FILE_OFFSET_BITS=64 \
-D_LARGEFILE_SOURCE -w -c ioc.c
Then execute make -f Makefile
For installing the GUI edit INSTALL.sh:
BUILD_FFTW=false BUILD_FLTK=true BUILD_RELION=false BUILD_RELION_GUI=true
Finally execute:
./INSTALL.sh
When running jobs on the queue, place a file 'qsub.bash' in the 'gui' directory of relion:
#!/bin/bash #$ -pe XXXqueueXXX XXXmpinodesXXX #$ -e XXXerrfileXXX #$ -o XXXoutfileXXX #$ -cwd #$ -V #needed on VSC-1: export IPATH_NO_CPUAFFINITY=1 mpirun -machinefile $TMPDIR/machines -np XXXmpinodesXXX XXXcommandXXX
The generated submit script should be modified to look like this:
#!/bin/bash #$ -pe mpich 128 #$ -e particles.err #$ -o particles.out #$ -cwd #$ -V export I_MPI_DAT_LIBRARY=/usr/lib64/libdat2.so.2 export OMP_NUM_THREADS=1 export I_MPI_FABRICS=shm:dapl export I_MPI_FALLBACK export I_MPI_CPUINFO=proc export I_MPI_PIN_PROCESSOR_LIST=0-15 export I_MPI_JOB_FAST_STARTUP=0 unset PE_HOSTFILE mpirun -machinefile $TMPDIR/machines -np $NSLOTS `which relion_preprocess_mpi` --o particles --ctffind "Micrographs/*.mrc" --ctfWin -1 --CS 2 --HT 300 --AmpCnst 0.1 --XMAG 79372 --DStep 15 --Box 512 --ResMin 100 --ResMax 7 --dFMin 5000 --dFMax 50000 --FStep 500 --ctffind3_exe /home/lv60000/jz/users/blaas/software/ctf/ctffind3.exe --coord_files "Micrographs/*.poscorr" --extract --coord_format xmipp2 --extract_size 240 --norm --bg_radius 100 --white_dust 3.5 --black_dust 3.5 --invert_contrast
Settings in the GUI:
Installation Procedure with Intelcompilers and Intelmpi:
umask 0022 my_src=/opt/src/relion/relion-1.3 my_inst=INSTALL_vsc.sh my_prefix=/opt/sw/relion/1.3-intel/ cp_dirs="bin include lib share" cd $my_src sed '3 i export MPICXX=mpiicpc ; export CC=icc; export CXX=icpc ' INSTALL.sh > $my_inst bash $my_inst mkdir -p $my_prefix for d in $cp_dirs do cp -r $my_src/$d $my_prefix done mkdir -p $my_prefix/gui echo ' #!/bin/bash #$ -pe XXXqueueXXX XXXmpinodesXXX #$ -e XXXerrfileXXX #$ -o XXXoutfileXXX #$ -cwd #$ -V #needed on VSC-1: #export IPATH_NO_CPUAFFINITY=1 mpirun -machinefile $TMPDIR/machines -np XXXmpinodesXXX XXXcommandXXX '> $my_prefix/gui/qsub.bash
Add this to .bashrc:
RELION_PREFIX=/opt/sw/relion/1.3-intel
export PATH=${RELION_PREFIX}/bin:$PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${RELION_PREFIX}/lib
export RELION_QSUB_TEMPLATE="$RELION_PREFIX/gui/qsub.bash"
export RELION_CTFFIND3_EXECUTABLE="/opt/sw/ctf/ctffind3.exe"