Using .env Files for Environment Variables in Python Applications

Applications are made to be deployed. At some point during development you will need to think about the environment in which your application will run and the potentially sensitive or environment specific information your application will need to perform its tasks.

Environment variables are one of the key ways software developers provide an application with this kind of information, however running and testing an application locally that is dependent on environment variables can be a hassle if you are setting those variables on your local machine’s environment.

The process of setting or changing an environment variable is time consuming and over time the number of environment variables you have to manage grows out of control. Eventually naming conflicts becomes an issue and every new variable requires a lengthy prefix to distinguish itself from similar variables.

.env File

Using a .env file will enable you to use environment variables for local development without polluting the global environment namespace. It will also keep your environment variable names and values isolated to the same project that utilizes them.

A .env file is a text file containing key value pairs of all the environment variables required by your application. This file is included with your project locally but not saved to source control so that you aren’t putting potentially sensitive information at risk.

# environment variables defined inside a .env file
GCP_PROJECT_ID=my-project-id
SERVICE_ACCOUNT_FILE=path/to/serviceAccountCredentials
STORAGE_BUCKET_NAME=my-super-important-data

python-dotenv

Nearly every programming language has a package or library that can be used to read environment variables from the .env file instead of from your local environment. For Python, that library is python-dotenv. Once the library is installed, an average use case for python-dotenv only requires adding two lines of code to your project.

from dotenv import load_dotenv

load_dotenv()

load_dotenv() will first look for a .env file and if it finds one, it will load the environment variables from the file and make them accessible to your project like any other environment variable would be.

import os
from dotenv import load_dotenv

load_dotenv()

GCP_PROJECT_ID = os.getenv('GCP_PROJECT_ID')
SERVICE_ACCOUNT_FILE = os.getenv('SERVICE_ACCOUNT_FILE')
STORAGE_BUCKET_NAME = os.getenv('STORAGE_BUCKET_NAME')

If an environment variable is not found in the .env file, load_dotenv will then search for a variable by the given name in the host environment. This means that when your project is running locally and the .env file is present, the variables defined in the file will be used. When your project is deployed to a host environment like a virtual machine or Docker container where the .env file is not present, the environment variables defined in the host environment will be used instead.

By default load_dotenv will look for the .env file in the current working directory or any parent directories however you can also specify the path if your particular use case requires it be stored elsewhere.

from dotenv import load_dotenv
from pathlib import Path

dotenv_path = Path('path/to/.env')
load_dotenv(dotenv_path=dotenv_path)

For most applications, that’s all the information you should need to be productive with python-dotenv however there are a few additional features which you can read about in the python-dotenv documentation.

A quick example

.
├── .env
└── settings.py

import os
from dotenv import load_dotenv

#automatically gets the .env file from the root directory
load_dotenv()

SECRET_KEY = os.environ.get("SECRET_KEY")
DATABASE_PASSWORD = os.environ.get("DATABASE_PASSWORD")

The .env file contains the following

SECRET_KEY=abcdefghh
DATABASE_PASSWORD=123456

Each line in the .env file is the environment variable name along with its associated values. Here we have set up 2 environment variables.

To get the same result without using the .env file we would have to setup the environment variables either by using the environment variable manager in Windows or by using export SECRET_KEY=abcdefghh in Linux-based systems.

Using .env file for configuration

Another common use case is for Project configuration. For example – You want to choose between the production database and test database when developing or deploying your application, we can simply use a debug variable in the .env file instead of changing it in the code. This makes it easier for everyone working on the project to quickly test and push their changes after testing.

Let’s look at the code below, I have mocked the actual database connection with a string for demonstration purposes.

.env file

DEBUG=True

import os

from dotenv import load_dotenv

load_dotenv()

DEBUG = os.environ.get("DEBUG")

if DEBUG:
    db="Test Database"
else:
    db="Production Database"

print(db)

Result

Test Database

Working, Yayy!! 😄
Let’s do some testing by changing the DEBUG=False in the .env file and re-running the script.

Result

Test Database

Whatt!! 🤯
This is a very naive approach as the IF statement is only checking “if the variable is defined or not” and the “actual value of the variable“.

Let’s makes some changes in the code and set DEBUG=True in the .env file.

import os

from dotenv import load_dotenv

load_dotenv()

DEBUG = os.environ.get("DEBUG")

if DEBUG==True:
    db="Test Database"
else:
    db="Production Database"

print(db)

what do you think will be the output of the code? Let’s run the script to find out.

D:\LEARNING\Python\dotenv>setting.py
Production Database

😲what just happened? We accidentally made changes to the production database, but we clearly used the debug variable to choose the debug database.

To understand what happened we see the return type of the os.environ.get() function. It returns an “str” and we are comparing it to a boolean True, so this “if” statement will never be true.

There is a quick fix to this => Compare the Debug variable with the string “True”

if DEBUG=="True":
    db="Test Database"
else:
    db="Production Database"

There is another way to fix this. we can convert the variable to a bool using strtobool from distutils library. I don’t prefer this approach because it involves installing an external library.

I remember making this mistake in my early days. I got really frustrated because things were working fine locally as I had debug=True in my .env file but when I pushed it to production with debug=False, It was still using the test database.

Conclusion

One of the benefits of using a .env file is that it becomes much easier to develop with environment variables in mind early on. Thinking about these things at an earlier stage of the development process will make it easier for you to get your application ready for deployment.

If you are deploying your application in a Docker container, you can seamlessly transition to running and testing your application in a locally run container by using the flag --env-file .env with your docker run command. Docker will then set the same environment variables you’ve been using through python-dotenv in the container’s environment.

Without a .env file, environment variables are a cumbersome albeit necessary part of testing your application locally. By using a .env file and the python-dotenv library, working with environment variables becomes much more manageable and will get you developing with deployment in mind right from the start.

Download Source Code

$ git clone https://github.com/favtuts/python-best-practices.git
$ cd python-dotenv-examples