Managing python virtual environments

It is always recommended to use a virtual environment while developing Python applications. They essentially allow you to create a “virtual” isolated Python installation and install packages into that virtual installation. When you switch projects, you can simply create a new virtual environment and not have to worry about breaking the packages installed in the other environments.

Create a new environment with Conda

One method to create a Python virtual environment is through the Conda package manager. Conda is an open-source package management system and environment management system. With conda, you can create, export, list, remove, and update environments that have different versions of Python and/or packages installed in them. Switching or moving between environments is called activating the environment.

Warning

Ensure that Anaconda or Miniconda is installed on your OPH Cluster. Check if you can loaded the module (module load anaconda3/xx); otherwise, you may need to install it in the cluster (visit the Anaconda website)

Once you load a version of anaconda, you can create and manage Python environments using the conda command. To create an environment, use the following conda command:

conda create --name my_env python=3.9

This will create a conda environment named whatever value you put in place for [my_env], using Python version 3.9. This Python version can be modified to a specific version of Python that you want your environment to use.

You have also the option to create a Conda environment from a YAML file by executing the following command:

conda env create --file envname.yml

where the YAML file lists all the packages and their versions that you want to include in your Conda environment. Here’s an illustrative example:

name: my_env
channels:
  - defaults
dependencies:
  - python=3.8
  - numpy=1.19.2
  - pandas=1.1.3

This method ensures consistent package versions across different environments and makes it easier to reproduce the environment on different systems.

Activiating the Environment

After creating an environment, you need to activate it before use. You can activate it using the following command:

conda activate my_env

Once your environment is activated, you will see your command line prompt change:

(my_env)[username@ophfe2 ~]$

It will start with your environment name in parenthesis at the start of the prompt.

Installing Packages Inside An Environment

Once inside your environment, you can install additional packages with the conda install command. Some packages must be installed from a specific channel, such as conda-forge. To install a package from a specific channel, such as conda-forge, use the command:

conda install -c conda-forge PACKAGE

You can also install a specific version of a package by using the command:

conda install PACKAGE=VERSION