Even if you’re new to Python (or other programming languages), there’s a good chance you’ll be aware of the importance of using libraries and packages. You might have also come across the ‘laboratory equipment analogy’, which compares using packages in your application or script with a scientist installing only the equipment in their lab that is required for each particular experiment, to avoid things getting cluttered and over complicated. The Software Carpentry have an excellent tutorial about this.
If your code relies on particular versions of libraries or packages, then it is useful to be able to record your environment setup, and enable others to create the same setup, when re-using your code.
Back to the laboratory analogy: what if other researchers wish to replicate the scientist’s experiments in other labs, despite recent equipment upgrades and improvements? For successful replication, our scientist would need to provide all equipment/versions used, along with any other important experimental conditions. That is, for each experiment (or Python application/script in our case), the full environment must be documented. Thankfully, with Python the ability to create and share such environments is baked in, using virtual environments (venv) and pip.
Virtual environments are also important for isolating your code from your system version of Python. The operating system that you work on will, by necessity, usually be using an older (and more stable) version of Python - which tends to only get updated for security reasons, not feature updates. If you want to use the latest features of a language such as Python, or other libraries, then it is useful to be able to use virtual environments to isolate these from the system libraries.
So how do we know what packages are available and how do we install, upgrade or uninstall them?
pip
One way to add, upgrade or remove packages into your virtual environments is by using pip, a built-in and easy to use Python package management program. Full instructions on how to install pip (if not already included as part of your Python install) are in https://pip.pypa.io/en/stable/installing/. Then to find out how to to use pip, see: https://docs.python.org/3/tutorial/venv.html#managing-packages-with-pip
conda
The above functionality has proved so useful that it has been extended for packages which are not pure Python (or, indeed, are not Python based at all), using the conda environment manager, and conda-forge software channel. For example, there are packages available for other languages (e.g. R), as well as libraries (e.g. HDF5 and BLAS), and complete software programs (e.g. prismatic, ncview).
Instructions for installing conda are available as are instructions for working with the conda environment manager.
To get the most out of conda it is recommended to use the conda-forge channel, which contains a lot of community-contributed packages.
Further reading
venv + pip
- Description of virtual environments and pip
- How to install pip
- Repository of all current python packages in pip
- How to create your own package