Procedure:
----------
Recommendations:
----------
The disk space used by my files were multiplied by 2, possibly due to backups.
Since the quota disk is not so high, it is possible to ask for a temporary space on the disk "scratch", which does NOT offer automatic backup service.
In this configuration, a proper sge script shall copy your working space on scratch, launch the experiments, copy back the results and clean.
It is also possible to launch a python script from the sge script if your prefer python to bash.
Additionnal remarks (1):
----------
Use the following commands to adjust your environment:
'module avail' - show available modules
'module add <module>' - adds a module to your environment for this session
'module initadd <module>' - configure module to be loaded at every login
Or you can install your librairies in your local path.
For instance to install Python 3 & Theano :
$ wget lien Anaconda3 (linux 64) (or a smaller one miniconda3 Minconda3 (linux64))
$ bash Anaconda_downloaded.sh
$ source .anaconda #ajout de la source anaconda ?
That should add this two lines in your .bashrc file I think :
# added by Miniconda3 4.1.11 installer
export PATH="/home/tao/USERNAME/miniconda3/bin:$PATH"
$ git clone https://github.com/Theano/Theano
$ cd Theano
$ python setup.py develop
If you are experiencing technical difficulties, you can contact the administrators at https://helpdesk.inria.fr/. (3)
References (2)
----------
- Contact https://helpdesk.inria.fr/ to register your public ssh key to ssh.saclay.inria.fr
- Connect to the Inria frontend: ssh -A your_username@ssh.saclay.inria.fr (-A is for agent transfer, necessary for the next step)
- Connect to the SGE frontend: ssh -A 195.83.212.209
- Copy your installation directory from your personal space to the cluster, using scp or svn
- Load the available modules, if needed (cf.1)
- Compile your project
- Access to an arbitrary node with "qlogin" and test that the execution is correct
- Write a sge script (bash script but with additionnal syntax for SGE) for running your experiments (cf.2)
- From the SGE frontend, submit your job with qsub your_script_name
- Monitor the execution of your job with qstat (qstat -u "*" for all users)
- If something goes wrong, delete the job with qdel
- Copy back the results to your personal space
Recommendations:
The disk space used by my files were multiplied by 2, possibly due to backups.
Since the quota disk is not so high, it is possible to ask for a temporary space on the disk "scratch", which does NOT offer automatic backup service.
In this configuration, a proper sge script shall copy your working space on scratch, launch the experiments, copy back the results and clean.
It is also possible to launch a python script from the sge script if your prefer python to bash.
Additionnal remarks (1):
Use the following commands to adjust your environment:
'module avail' - show available modules
'module add <module>' - adds a module to your environment for this session
'module initadd <module>' - configure module to be loaded at every login
Or you can install your librairies in your local path.
For instance to install Python 3 & Theano :
$ wget lien Anaconda3 (linux 64) (or a smaller one miniconda3 Minconda3 (linux64))
$ bash Anaconda_downloaded.sh
$ source .anaconda #ajout de la source anaconda ?
That should add this two lines in your .bashrc file I think :
# added by Miniconda3 4.1.11 installer
export PATH="/home/tao/USERNAME/miniconda3/bin:$PATH"
$ git clone https://github.com/Theano/Theano
$ cd Theano
$ python setup.py develop
If you are experiencing technical difficulties, you can contact the administrators at https://helpdesk.inria.fr/. (3)
References (2)
- http://gridscheduler.sourceforge.net/htmlman/manuals.html
- http://star.mit.edu/cluster/docs/0.92rc2/guides/sge.html
- https://wikis.utexas.edu/display/CCBB/sge-tutorial