Date: Fri, 29 Mar 2024 13:17:16 +0100 (CET) Message-ID: <653847650.2642.1711714636554@atlassiancin01.private.cineca.it> Subject: Exported From Confluence MIME-Version: 1.0 Content-Type: multipart/related; boundary="----=_Part_2641_929363929.1711714636548" ------=_Part_2641_929363929.1711714636548 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Location: file:///C:/exported.html
The bulk of the cineca-ai package, p= rovided by the deeplrn profile, is based on the Open Cognitive Environment = (Open-CE) tool, which includes (for example) Tensorflow, Pytorch, XGBoost, = and other related packages and dependencies.
This cognitive environment has been personalised by CINECA AI experts an= d will be published in a public channel<= /a>. You can find several versions of the cineca-ai module in profile/deepl= rn, differing in the versions of their main components (pytorch, tensorflow= etc.). The module help reports the versioning for these components. For a = complete list load the module and launch the "pip list" command.
The CINECA AI project can be used in several ways, depending on the meth= od more suited to your needs and on the availability of conda/pip packages.=
module = load profile/deeplrn module load autoload cineca-ai/<version> # see all available packages pip list # enjoy the environment
If you prefer working in a conda environment, you can activate the cinec= a-ai env via "conda activate $CINECA_AI_ENV", and explore the environment w= ith "conda list".
If you need to use a package not included in the list of those provided = by the cineca-ai modules, you can always rely on the cineca-ai environment = for the dependencies and install what you need within a personal virtual en= vironment and/or a conda environment.
If your package is available to pip:
module = load profile/deeplrn module load autoload cineca-ai/<version> python -m venv <myvenv> --system-site-packages source <myvenv>/bin/activate pip install PACKAGE
NOTES:
Example: torch-scatter
It requires torch, which will be taken from = the cineca-ai env (i.e., no need to install it):
$ module load profile/deeplrn
$ module = load autoload cineca-ai/2.1.0
$ python -m venv torch_venv --system= -site-packages
$ source torch_venv/bin/activate
$ pip install torch-s= catter
module = load autoload cineca-ai/<version>=20 conda create --prefix <mycondaenv> -y conda activate <mycondaenv> # see all packages in the channel conda search -c $CINECA_AI_CHANNEL --override-channels # install a package from the channel conda install -c $CINECA_AI_CHANNEL PACKAGE
# after= activating <mycondaenv> and installed the needed prerequisites of PA= CKAGE from CINECA channel python -m venv <myvenv> --system-site-packages source <myvenv>/bin/activate pip install PACKAGE
# after= activating <mycondaenv> and installed the needed prerequisites of PA= CKAGE from CINECA channel, install the additional packages conda install PACKAGE1 conda install -c conda-forge PACKAGE2
NOTES:
# after= activating <mycondaenv> and installed the needed prerequisites of PA= CKAGE from CINECA channel, install the additional conda packages conda install PACKAGE # create a virtual env and install the additional pip packages python -m venv <myvenv> --system-site-packages source <myvenv>/bin/activate pip install PACKAGE
NOTES:
Example: netket
It requires some of the conda packages provided by the cineca-ai environ= ment, and netket is only available via pip.
$ module load autoload cineca-ai/2.1.0
$ conda create --prefix <= mycondaenv> -y
$ conda activate <mycondaenv>
$ conda install= -c $CINECA_AI_CHANNEL mpi4py jaxlib jax dm-tree mpi4jax
$ conda i= nstall cmake h5py -y
$ conda install -c conda-forge llvmlite=3D0.38.0 or= json -y
$ python -m venv <myvenv> --system-site-packages
$ sour= ce <myvenv>/bin/activate
$ pip install 'git+https://github.com/netket/netket.git#egg=3Dnetket[all]'
conda c= reate --prefix <mycondaenv> --clone /cineca/prod/opt/libraries/cineca= -ai/<version>/none/cineca-ai-conda-env-py3.8-cuda-openmpi-11.0 -y conda activate mycondaenv # see all packages in the channel conda search -c <mycondaenv> --override-channels # install a package from your personal channel conda install -c <mycondaenv> PACKAGE