11/24/2021»»Wednesday

Install Python In Docker

Here are the steps to install python 3 on docker. First download the appropriate docker image: docker pull centos. Start the docker container: docker run -it -name boto3-centos centos. Install Docker Desktop. The next step is installing Docker Desktop. So Docker Desktop is an application that connects to our default WSL2 machine and manages Docker Containers. It is not required to run containers on your virtual Linux OS but highly recommended. Because it makes the management of Linux Containers on Windows 10 very easy. ADD is a really useful instruction, it can add remote files to you Docker image. The ENTRYPOINT specifies the entry point for any command, in our case python app.py, pretty much like running /tini - python app.py. Update pip, setuptools and wheel. Python is an interpreted, interactive, object-oriented, open-source programming language.

If you ever wanted to create your own Python application in Docker VSCode, here is what you need to do. We will cover all necessary steps, the VSCode extensions you need, and how debugging works. Because with the right VSCode extensions it’s very easy and comfortable to develop an application in a container.

I’m developing my Python Applications in Docker Containers on Windows 10. But you can also follow this tutorial if you’re running Mac OS or Linux, just skip the following section for Windows.

Set up our VSCode environment on Windows 10

First, we need to install and set up our Development Environment. Because we want to run our Python application in a Linux Docker container with VSCode. Linux containers are lightweight and highly customizable. And they can run in very large and scaled environments, such as Kubernetes or Cloud Provider Services.

Install Windows Subsystem for Linux

Because we can’t run Linux containers natively on a Windows Operating System, we need to install our development environment with WSL2 (Windows Subsystem for Linux in version 2). It is a small virtual machine that runs a full Linux OS inside Windows 10. So, you can use it to develop, run and test your Docker Container locally, which makes development pretty easy.

To install Windows Subsystem for Linux on your Windows 10 machine, just follow the instructions in the official documentation.

Install Docker Desktop

The next step is installing Docker Desktop. So Docker Desktop is an application that connects to our default WSL2 machine and manages Docker Containers. It is not required to run containers on your virtual Linux OS but highly recommended. Because it makes the management of Linux Containers on Windows 10 very easy. With the right extensions in VSCode, we can later add some features that will make the build and debugging process for our Containers so much easier.

Install python package in docker container

Again, just follow the official instructions to install Docker Desktop.

Add the right VSCode extensions for Python and Docker

There are also some VSCode Extensions, which I highly recommend. You can simply just search for them in VSCode Extensions, download and install them.

  • Docker – Makes it easy to create, manage, and debug containerized applications.
  • Remote WSL – Open any folder in the Windows Subsystem for Linux (WSL) and take advantage of Visual Studio Code’s full feature set.
Dockerfile

Connect to WSL2 in VSCode

It’s very easy to run VSCode remotely on WSL2. You can just click on the Remote Connection Icon in VSCode, or open VSCode with the “code” command from your Windows Terminal Application. Let’s try the second method and create a new project folder for the application and open this remote workspace in VSCode.

If you execute the following commands in the WSL terminal, VSCode should open on your Windows 10 and automatically connect to your remote WSL workspace.

If you’re successfully connected to your WSL machine, you’ll see it in the bottom statusbar.

Build a simple Python application

Finally, we can now start developing our application. This should be the easy part for you. I don’t have any good example projects, so we will write a very simple application that will just add two numbers and output the result.

Create a new file and selection our Python interpreter

Let’s create a new Python file called app.py and place it in our workspace folder. If you get a message to select your Python Interpreter, you can simply select it. Because we need to tell WSL how we want to run Python programs. If you don’t get a message, but you want to select your standard Python interpreter, only for this workspace folder you can create a new file called .vscode/settings.json inside your workspace folder.

This is our main application script app.py

Set the Python interpreter in .vscode/settings.json

Test our Python application without a container

To test and run our application without a container, we can simply execute the following command in the terminal. Or, if you have set up your Python interpreter correctly, you can also run it with F5 in VSCode and debug it.

Build our first Docker Container

But of course, we want to deploy this Python application inside a Docker container with VSCode. The Docker extension in VSCode has a pretty comfortable function to generate all necessary files automatically for us. Don’t worry I will explain all the things the extension does, so you could also create all these files manually.

Generate Dockerfiles

To generate all files, press F1 and select “Add Docker Files to Workspace“. Simply follow the instructions, add your app.py as your main script and skip Docker-Compose.

How our Docker container is structured

Then, you should see some new files in your workspace folder. Because the Python Docker extension creates a complete environment for you in VSCode. It creates a Dockerfile, a requirements.txt, a .dockerignore, and some .vscode configuration files for you.

  • Dockerfile – This is the main file that contains all instructions how to generate and build the Docker image file
  • requirements.txt – Includes all necessary libraries and plugins that should be installed inside the Docker image
  • .dockerignore – All files that should not be included inside the Docker image
  • .vscode/launch.json – How to launch the application, debug it, etc.
  • .vscode/tasks.json – Any tasks that should be run by VSCode before the launch tasks

Let’s take a closer look at the Dockerfile.

The first line with the FROM python:3.8-slim-buster statement defines our base image. Because we want to build our application on top of a base Linux distribution image. In this example, the python:3.8-slim-buster is a small and lightweight Debian Image with Python 3.8 installed.

The ENV PYTHONDONTWRITEBYTECODE=1 will stop Python from generating .pyc files in the container which we don’t need.

COPY requirements.txt and RUN python -m pip install -r requirements.txt will copy our requirements.txt file inside the container and install all packages that are described here. This is just a plain text file where you can list all pip packages with a specific version you need inside your container.

WORKDIR /app and COPY . /app set’s our work directory inside the container and copies the entire project folder (except files that are defined in the .dockerignore) into the container.

RUN adduser -u 5678 –disabled-password –gecos “” appuser && chown -R appuser /app and USER appuser is highly recommended for security reasons. Because it limits the permissions and user id that the application is using inside the container. This can help to prevent privilege escalation if our application has security vulnerabilities.

CMD [“python”, “app.py”] will execute the python command to run our application inside the container. The container automatically exits, when our python script exits.

Build the Docker image file and run it

The Docker extension in VSCode allows us to simply build the Docker image with a right click on the Dockerfile and select “Build Image”.

Open a new Terminal and type docker image list. Because then we can see a new entry called vscodedockerpython (the project folder name). We can also simply run this container and see that our application is running successfully!

What about debugging inside the container

The Docker extension in VSCode is absolutely beautiful! Because it also allows us to debug our application inside the container with no effort. In the past, I needed to install and use debugging libraries and extensions in Python, but this is not needed anymore. The extension is smart enough to rewrite our entry point file with a debugger automatically.

You simply just need to click on “Start debugging” and it works!

This is only possible, because the Docker extensions created the two files .vscode/launch.json and .vscode/tasks.json.

Set breakpoints and analyzing variables

This also supports breakpoints and analyzing variables natively inside the Container! Which is absolutely powerful and key to write complex applications in Python.

Question or problem about Python programming:

Can you give me an example of a Dockerfile in which I can install all the packages I need from poetry.lock and pyproject.toml into my image/container from Docker?

How to solve the problem:

Solution 1:

There are several things to keep in mind when using poetry together with docker.

Installation

Official way to install poetry is via:

This way allows poetry and its dependencies to be isolated from your dependencies. But, in my point of view, it is not a very good thing for two reasons:

  1. poetry version might get an update and it will break your build. In this case you can specify POETRY_VERSION environment variable. Installer will respect it
  2. I do not like the idea to pipe things from the internet into my containers without any protection from possible file modifications

So, I use pip install 'poetry$POETRY_VERSION'. As you can see, I still recommend to pin your version.

Also, pin this version in your pyproject.toml as well:

It will protect you from version mismatch between your local and docker environments.

Caching dependencies

We want to cache our requirements and only reinstall them when pyproject.toml or poetry.lock files change. Otherwise builds will be slow. To achieve working cache layer we should put:

After the poetry is installed, but before any other files are added.

Virtualenv

The next thing to keep in mind is virtualenv creation. We do not need it in docker. It is already isolated. So, we use poetry config virtualenvs.create false setting to turn it off.

Development vs Production

If you use the same Dockerfile for both development and production as I do, you will need to install different sets of dependencies based on some environment variable:

This way $YOUR_ENV will control which dependencies set will be installed: all (default) or production only with --no-dev flag.

You may also want to add some more options for better experience:

  1. --no-interaction not to ask any interactive questions
  2. --no-ansi flag to make your output more log friendly
Result

You will end up with something similar to:

You can find a fully working real-life example here: wemake-django-template

Update on 2019-12-17
  • Update poetry to 1.0

Solution 2:

Multi-stage Docker build with Poetry and venv

Do not disable virtualenv creation. Virtualenvs serve a purpose in Docker builds, because they provide an elegant way to leverage multi-stage builds. In a nutshell, your build stage installs everything into the virtualenv, and the final stage just copies the virtualenv over into a small image.

Use poetry export and install your pinned requirements first, before copying your code. This will allow you to use the Docker build cache, and never reinstall dependencies just because you changed a line in your code.

Do not use poetry install to install your code, because it will perform an editable install. Instead, use poetry build to build a wheel, and then pip-install that into your virtualenv. (Thanks to PEP 517, this whole process could also be performed with a simple pip install ., but due to build isolation you would end up installing another copy of Poetry.)

Here’s an example Dockerfile installing a Flask app into an Alpine image, with a dependency on Postgres. This example uses an entrypoint script to activate the virtualenv. But generally, you should be fine without an entrypoint script because you can simply reference the Python binary at /venv/bin/python in your CMD instruction.

Dockerfile

docker-entrypoint.sh

wsgi.py

Solution 3:

That’s minimal configuration that works for me:

Note that it is not as safe as @sobolevn’s configuration.

As a trivia I’ll add that if editable installs will be possible for pyproject.toml projects, a line or two could be deleted:

Solution 4:

Here’s a stripped example where first a layer with the dependencies (that is only build when these changed) and then one with the full source code is added to an image. Setting poetry to install into the global site-packages leaves a configuration artifact that could also be removed.

Solution 5:

Install

TL;DR

Install Python Package In Docker Container

I have been able to set up poetry for a Django project using postgres. After doing some research, I ended up with the following Dockerfile:

This is the content of entrypoint.sh:

Detailed Explanation

Some points to notice:

  • I have decide to use slim instead of alpine as tag for the python image because even though alpine images are supposed to reduce the size of Docker images and speed up the build, with Python, you can actually end up with a bit larger image and that takes a while to build (read this article for more info).

  • Using this configuration builds containers faster than using the alpine image because I do not need to add some extra packages to install Python packages properly.

  • I am installing poetry directly from the URL provided in the documentation. I am aware of the warnings provided by sobolevn. However, I consider that it is better in the long term to use the lates version of poetry by default than relying on an environment variable that I should update periodically.

  • Updating the environment variable PATH is crucial. Otherwise, you will get an error saying that poetry was not found.

  • Dependencies are installed directly in the python interpreter of the container. It does not create poetry to create a virtual environment before installing the dependencies.

In case you need the alpine version of this Dockerfile:

Notice that the alpine version needs some dependencies postgresql-dev gcc python3-dev musl-dev openssl-dev libffi-dev to work properly.

Solution 6:

This is a minor revision to the answer provided by @Claudio, which uses the new poetry install --no-root feature as described by @sobolevn in his answer.

In order to force poetry to install dependencies into a specific virtualenv, one needs to first enable it.

Install Python In Docker Example

Therefore adding these into @Claudio’s answer we have

If you need to use this for development purpose, you add or remove the --no-dev by replacing this line

to something like this as shown in @sobolevn’s answer

after adding the appropriate environment variable declaration.

Install Python In Docker Image

The example uses debian-slim’s as base, however, adapting this to alpine-based image should be a trivial task.

Install Python In Docker Image

Hope this helps!

Most Viewed Posts