Test Django app using Postgres + Redis with GitHub Actions

Introduction

Hello there! App testing plays a crucial role in application development, regardless of the framework or language used. This article will demonstrate how to utilize GitHub actions to test a Django app that utilizes Redis and Postgres, with test cases that require these databases.

Set up project

Assuming that you already know how to set up a Django project and create new apps within it. By the way, I will share the GitHub repo of this project with you.

Environments

As most of you know, we may have several environments for our project, such as development, test, staging, and finally production. Each of these environments must have its variables to work with. Every environment has to have its variables and database images to ensure that it does not affect any other environment. This helps to maintain the integrity of the project and ensures that any issues can be identified and resolved in a controlled manner.

To utilize environment variables, I recommend using the Decouple package. You can find installation instructions and usage guidelines on their website.

Redis and Postgres connections

To utilize Postgres and Redis, we need to modify the configurations in _base/settings.py.

# in _base/settings.py
from decouple import config

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': config('DATABASE_NAME'),
        'USER': config('DATABASE_USER'),
        'PASSWORD': config('DATABASE_PASS'),
        'HOST': config('DATABASE_HOST'),
        'PORT': config('DATABASE_PORT'),
    }
}

# redis
CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": config('REDIS_LOCATION'),
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            "PASSWORD": config('REDIS_PASSWORD'),
        }
    }
}

As you may notice, we make use of environment variables for location, password, host, and other important data in our databases and cache systems, specifically in redis. In addition, we utilize django_redis for our redis client.

Implement Models

In this section we are going to create our basic models.

# in bookstore/models.py
from django.db import models


class Author(models.Model):
    first_name = models.CharField(max_length=200)

    last_name = models.CharField(max_length=200)


class Book(models.Model):
    name = models.CharField(max_length=200)

    author = models.ForeignKey(
        to=Author,
        on_delete=models.DO_NOTHING
    )

Next, we must execute python manage.py makemigrations to generate migration files for the models and apply them to our default database.

Cache books data

Now, It’s time to write a utility to cache our data in Redis.

# in bookstore/utils.py
from django.core.cache import cache

from bookstore.models import Book

BOOKS_CACHE_KEY = 'books'


def cache_all_books():
    all_books = Book.objects.all().select_related('author').values(
        'id',
        'name',
        'author__first_name',
        'author__last_name',
    )
    cache.set(BOOKS_CACHE_KEY, list(all_books))


def get_cached_books():
    return cache.get(BOOKS_CACHE_KEY)

We are currently utilizing the Django cache system, which uses Redis to store all book data, including their respective author information.

Write Test

Now we need to test this cache utility to see if our function cache_all_books will store all of our data in Redis or not.

# in bookstore/tests.py
import random
from django.test import TestCase

from bookstore.models import Author, Book
from bookstore.utils import cache_all_books, get_cached_books


class TestCacheSystem(TestCase):

    @classmethod
    def setUpTestData(cls):
        cls.author_1 = Author.objects.create(first_name='Ehsan', last_name='Movaffagh')
        cls.author_2 = Author.objects.create(first_name='Banana', last_name='Movaffagh')
        authors = [cls.author_1, cls.author_2]

        cls.books = Book.objects.bulk_create([
            Book(name=f'number {_}', author=random.choice(authors)) for _ in range(10000)
        ])

    def test_cached_books(self):
        initial_book_ids = [_.id for _ in self.books]

        cache_all_books()
        cached_books = get_cached_books()
        cached_book_ids = [_['id'] for _ in cached_books]

        self.assertListEqual(initial_book_ids, cached_book_ids)

In this test case, we first prepare our data for the test. Then, we verify that we have correctly stored all the books that were initially created.

Write Github workflow

It’s time to create our workflow to test our project on GitHub servers.

Create a file named .github/workflows/test.yml. In this file, we will configure our test environment and create a workflow to test our project.

name: Test

on:
  pull_request:
    branches: [ "main" ]

jobs:

  test:
    runs-on: ubuntu-20.04

    container: python:3.8

    services:
      postgres:
        image: postgres
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 5432:5432

      redis:
        image: redis
        options: >-
          --health-cmd "redis-cli ping"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 6379:6379

    steps:
      - name: Check out repository code
        uses: actions/checkout@v3

      - name: Install Dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

      - name: Run Tests
        run: |
          python manage.py test
        env:
          DATABASE_NAME: postgres
          DATABASE_USER: postgres
          DATABASE_PASS: postgres
          DATABASE_HOST: postgres
          DATABASE_PORT: 5432
          REDIS_LOCATION: redis://redis:6379/?db=0
          REDIS_PASSWORD:

Simply put, this file sets up the testing process for Ubuntu 20.04 whenever a pull request is made to the main branch. Docker images of Redis and PostgreSQL are used as services to test our application. The process involves installing project dependencies and running project tests using the aforementioned environments.

For more information about services, you can see this page.

Create new PR

Now, we must create a new pull request to test our workflow and see the results. You may notice that we forgot to store the author_id in the cache, so we need to make that change.

# in bookstore/utils.py
def cache_all_books():
    all_books = Book.objects.all().select_related('author').values(
        'id',
        'name',
        'author_id',
        'author__first_name',
        'author__last_name',
    )
    cache.set(BOOKS_CACHE_KEY, list(all_books))

Then, We need to create our PR:

Create PR.

As you can see, Our test action has been initiated.

The test workflow has succeeded.

Check if we can merge the PR.

Then we can simply merge this branch to main.

Merge the PR.

Conclusion

And we’re done!

We have completed the necessary steps to test our Django project (which utilizes Postgres and Redis) using GitHub Actions. Testing is an essential aspect of software development, and in this article, we have outlined how to automate testing with GitHub workflows.

Additionally, you can find the git repository for this project through this link. Hope you all enjoyed it!