Setting Up Python for Scientific Computing

Introduction

This is a guide on installing and setting up Python for scientific computing. For those of you new to Python, this setup may feel somewhat longer and more cumbersome than say, Matlab. This is largely due to the freedom and flexibility that general-purpose programming languages provide, in contrast to a product like Matlab which makes most decisions for you, such as which tools to use and how to use them.

This setup aims to properly setup Python for scientific computing in a way that minimizes the amount of trouble and confusion which may arise later on during development or deployment, by taking extra steps during the installation and setup portion to ensure proper management of dependencies and compatibility.

This guide can essentially be omitted by using all of the links provided within it, as those links cover all of the information discussed in this guide, and more.

Here is the FAQ section of the scipy website, which helps explain common questions such as the main differences between Python and other alternatives.

Download Python

Visit https://www.python.org/downloads/ - This is the official website for Python - Download the version for your operating system ( Windows, Mac, etc.) - You will want to get version 3.x.x, as 2.x.x will soon be phased out - For installing Python on Mac or Linux, simply launch the installer and go through all the steps. You can safely ignore the following section

Install Python (Windows)

This guide will cover all steps, but refer to this guide for official documentation on how to install and run Python.

  • Run the installer until you get to the following image

python installation

  • At this point, Make sure to select “Add Python 3.x to PATH”!
  • Once you have done so, continue through with the installation until completion

(Optional) Setup VSCode (a text editor)

For those of us who have a preferred text editor for programming, feel free to use that for your programming needs. However, for those of us who are either unsure of which text editor / IDE they should use, or are curious to see what this guide’s recommended setup is, keep reading

Download / install VSCode

VSCode can be downloaded at this link. The installation process should require no extra steps, and can essentially be sped through.

Setup VSCode

Here is the accompanying guide.

Once VSCode has been installed, you should start by clicking on the extensions button on the left side, and installing the Python package. Live Share is also useful if you wish to collaborate on a project in realtime, such as to debug code, or to help explain a certain concept remotely.

Using VSCode

Open your project

To start programming within VSCode, start by opening your python project folder by selecting file, and then open. You should see your project files populate the left column if you have the file tab open. If not, click the icon that looks like a file in the top-left corner, and you should see the project’s contents in the left-side tab (if you have any)

Use the integrated terminal

Since software development relies so heavily on a terminal (Command Prompt in Windows), it is a relief to know that VSCode includes an integrated terminal, which can be opened by pressing the control (aka ctrl on Windows) and ` key at the same time. You should see a terminal appear at the bottom. Here, you can enter the same commands you would normally enter in a terminal, such as pipenv shell.

Note: After everything is set up, you will most likely notice that depending on which project you have opened in VSCode, the terminal will have automatically loaded the virtual environment for that project, since the terminal’s prompt displays the name of your project in parentheses before the rest, and entering pipenv --venv will give you a “Courtesy Notice” saying something about automatically using an environment.

Using the command palette

If you aren’t sure where to access a command, pressing ctrl (command on Mac), shift, and p at the same time will open the Command Palette; a centralized access point for almost all functionality offered by VSCode.

Run an example Python file

As per tradition, a Hello World script can be created by putting the following in a file whose extension ends with .py:

print( "Hello World!")

A Python file can be run by navigating to the directory within which it is situated, and then entering the following:

python <name of file>.py
  • If on a Mac / Linux machine, replace python with python3

You should see the expected Hello World! outputted in the terminal.

Package management

pip and PyPI

Packages are installed system-wide using pip, which is a terminal-based tool for installing libraries and tools from PyPI. Later on we will introduce pipenv, which will be the preferred way of managing dependencies.

Update and setup pip

First, ensure pip / pip3 is up to date.

python -m pip install --upgrade pip setuptools wheel
  • If on a Mac / Linux machine, replace python with python3

Managing environments

While creating a program using Python, you will inevitably end up using many 3rd party packages. While it is quite easy to download and use said packages to create programs on your own machine, once that program is distributed to other users for their own usage, it is rather tricky to try and find out what packages the program depends on, and which version of these packages was compatible with the program. This is why virtual environments such as pipenv are used, which aim to containerize and automate the dependencies of your program.

Here is a great tutorial on pipenv, which is short and to the point.

Install pipenv

Install pipenv using the following command:

pip install pipenv

clone / create a new project

  1. create a new project, for now it could be as simple as a single .py file in a folder.
  2. navigate to that folder in your terminal ( Command Prompt for Windows), and then enter the following command:

    pipenv install
    
  • This will setup a virtual environment, which completely isolates all of the packages and things you might work with within this project, which means things you install and work with in this project will not interfere with other projects on your computer.
  • If the project was cloned from somewhere else, such as GitHub, pipenv will look at the Pipfile in the root directory of the project, and automatically install and setup any dependencies listed there. This solves the problem of finding and tracking packages.

Access the pipenv environment

Run many commands

To access your isolated pipenv environment, simple navigate to the directory in your terminal, and then enter the following command:

pipenv shell

This will now allow you to use the terminal as you normally would, but instead all your Python-related commands will depend on your project’s Python setup and packages.

Run a single command

If all you need to do is run a single command, such as a Python file, you can instead run

pipenv run <command>

for example, if you wish to run the Python file main.py in the current directory, run

pipenv run python main.py
  • If on a Mac / Linux machine, replace python with python3

Exit an environment

To exit, simply enter

exit

Packages to install

While there are many packages that are useful for specific use cases, the four standard scientific computing packages are

They will cover most use cases, and offer a solid collection of fast and easy to use data structures and functions.

Again, using pipenv, first ensure that you are in the root directory of the project you are working on. Then, using scipy as an example, enter

pipenv install scipy

If you then open the project’s Pipfile, you will see it has been updated to show the package scipy and its compatible version as a dependency of the project.

Example project:

You can access an example Python project at this repository. Simply click the green button with the text “Clone or download”, and choose “Download ZIP”. Once the download has finished, unzip the project and open it in VSCode or your editor of choice. Then, open a terminal, navigate to the project ( VSCode’s integrated terminal will already be in the project), and then enter pipenv install to install all of its dependencies. Then, finally, enter pipenv run python example.py. You should see a graph pop up. If you do not, make sure to verify that you have taken the correct steps above, or to Google the error code you received to see if there is a solution online ( almost guaranteed, considering how popular Python is).

Jupyter Notebooks

Quoting the official Jupyter website, “The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more.”

To install Jupyter Notebook using pip, enter the following command:

python3 -m pip install --upgrade pip
python3 -m pip install jupyter

Then, to run Jupyter Notebook, enter

jupyter notebook

Refer to this installation guide for more details.

Troubleshooting VSCode

If VSCode does not appear to properly work, such as improperly displaying which packages have been installed, or error messaging keep appearing in the bottom-right corner, there are a few solutions which will likely fix the problem:

Error Warning: Python 3.x was not found on your system… when running pipenv install

Here is the relevant post for MacOS / Linux users.

This means your machine does not have the version of Python required by the project. You can rectify this by installing and running pyenv. See this repository for details.

Windows has no easy way of installing Python (there is a port of pyenv for Windows, but it did not work on my machine), so the best option currently is to download and install that version of python from the official website.

Error saying package needs to be installed

If there is a notification saying a package needs to be installed, and it shows an install button, click it. Worse case scenario, you can simply uninstall it later

Choose the correct interpreter

Open the Command Palette, and enter Python: Select Interpreter. Then, choose the interpreter which has the same name as your project, which should be the interpreter that was installed in that project’s environment.

Correct interpreter not showing / other venv problems

Here is the relevant guide

Any problem such as an error mentioning pipenv –venv, can usually be rectified by adding the path of your virtual environments to VSCode’s venv variable in the settings. For example, the default path to the virtual environments on Mac is

/Users/(USERNAME)/.local/share/virtualenvs/
  • You can run the pipenv –env command in either VSCode’s terminal or the default one (such as Command Prompt) to find the path of the virtual environment of the project you are currently in. This path is essentially your virtual environment path, plus the actual name of the virtual environment for your current project.
Avatar
Philippe Nadon
Student majoring in Computing Science, and Math/Physics

My academic focus is in data science, scientific computing, and distributed networks.

Related