How to Test a Django Application: the Thought Process Behind Testing

In the following tutorial you’ll learn how to test a Django application and the thought process behind testing and refactoring

When it comes to backend my tools of choice are Django and Python. Speed of development (particulary useful for rapid prototyping) and the excellent test coverage I can get almost out of the box are the major boons for me.

How to Test a Django Application: the Thought Process Behind Testing

I use Django for training aspiring developers too and when I run a workshop I don’t go straight to the “right implementation”. What I like instead is guiding newcomers through the initial “working implementation” to refactoring into a more complex application.

In this tutorial you’ll build a small application with the exact same mindset. We’ll make it work first and them we will refactor for adding more functionalities.

Enjoy the reading!

How to test a Django application: what you will learn

In the following tutorial you’ll learn:

  • how to build a simple Django website
  • how to test a Django application
  • how to measure test coverage

How to test a Django application: requirements

To follow along with the tutorial you should have:

  • a basic understanding of Python and Django
  • a newer version of Python installed on your system (Python 3 is nice to have)
  • a basic understanding of both unit testing and functional testing

How to test a Django application: setting up a Python virtual environment and the project

Before getting started with the tutorial make sure to set up a Python virtual environment. Starting from Python 3 there is a native module for that: venv.

Move to a directory of choice, (for me it’s ~/Code) and create the new virtual environment:

python3 -m venv TestingDjangoVenv

Move inside the new folder and activate the environment with:

cd TestingDjangoVenv && source bin/activate

Once inside the environment install Django by running:

pip install django

When the installation ends you’re ready to create a new Django project:

django-admin startproject djangoproject

And you’re all set! Ready to roll!

How to test a Django application: building a Django application

I love Django for two main reasons: it’s easy to reach high test coverage, and there is a lot already built out of the box. On top of that Django really shines in modularization: a Django project is built around many applications. If the community has built something for that particular use case you can plug in the app and you’re good to go.

If that’s not the case you can write your own Django application and go from there. For this project we will create a simple Django application for storing and displaying links.

Move inside the project folder:

cd djangoproject

and run the handy command line tool that Django gives us:

django-admin startapp links

After a second you’ll see a new directory called links inside ~Code/TestingDjangoVenv/djangoproject.

Now let’s tell Django how to use the new app. This is the same thing as when you install a Drupal module or a WordPress plugin. In Django there is a configuration file to modify.

Open up ./djangoproject/settings.py and add the app in INSTALLED_APPS:

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'links', # load the new application
]

So far so good!

In the next section we’ll add our first model.

How to test a Django application: planning a Django applications (and our first model)

How to test a Django application: planning a Django application (and our first model)

NOTE: From now on the tutorial assumes you’re in ~Code/TestingDjangoVenv/djangoproject

We just created our first Django application. Now if you snitch inside ./links you will see a bunch of files:

admin.py
apps.py
__init__.py
migrations
models.py
tests.py
views.py

The most interesting files are indeed models.py, views.py and tests.py. We want to test our Django application and it looks like tests.py is the right place for adding test code.

models.py and views.py on the other hand suggests that Django relies on the Model-View-Controller pattern. And that’s true! Django follows a variation of MVC (Model-View-Controller) called MVT (Model-View-Template). If you want to learn more about the Django philosophy check out this FAQ.

But what’s a model? In brief, a model is a Python class representing a table in your database. For this simple application I can imagine a Link model, without thinking too much, with a bunch of fields. Every model in Django has fields which correspond to columns in the table.

A link can have a title, a url, and maybe a list of tags. At this moment I’m not concerned about model relationship but I’m already thinking about an interesting functionality: when I click a tag I can see a list of related links. Take note because it sounds like a nice user story!

For our Link model we’re going to add a timestamp field as well because Django does not have a created_at column by default. And with this basic info we can create a simple Django model for storing links. Open up ./links/models.py and create the model:

from django.db import models

class Link(models.Model):
    title = models.CharField(max_length=300, unique=True)
    url = models.URLField(max_length=300, unique=True)
    tags = models.CharField(max_length=200)
    created_at = models.DateTimeField(auto_now_add=True)

There’s not a lot going in this code but the most important thing is the shape of Link: we have a title which is a CharField with a max length of 300 characters; we gave Link a field named tags, which is just a string for now; we have a url which is a URLField. And there’s also a created_at field which is the timestamp for our model.

On top of that I’ve added unique=True to title and url. That takes care of duplicate links.

And with the model in place let’s create a migration by running:

python manage.py makemigrations links

and finally migrate the database with:

python manage.py migrate

At this point you should be ready to launch the website but … hold on for a second. We don’t have any test!

How to test a Django application: getting serious with testing

NOTE: The tutorial assumes you’re in ~Code/TestingDjangoVenv/djangoproject

Up until now we didn’t test anything in our Django application. Time to add some unit tests? Maybe! With the basic structure in place we are ready for adding a lot of links. But our application is flawed. Remember when I said “in this application I want the user to click a tag and see a list of related links“?

Well, as the links app is designed there is no way to click a link and retrieve all the related articles from the database. At this moment the tags field in our model is a simple string with no mapping to related models.

We need to unleash one of the most powerful Django feature: model relationships (the concept is not exclusive to Django, by the way). But before doing so we need to lay down the basis for testing our application.

This is the moment I advise adding tests to your code: you (I) made a silly design mistake and the application is flawed from the ground. But now you want to refactor and turn your prototype into something more serious. At this point don’t do refactoring without tests.

First, how do I know exactly what to test? Is there a tool for highlighting un-tested code? Yes, and that tool is called “test coverage” which is also the general idea behind a lot of powerful tools in the same category.

In Python we can install a tool called coverage:

pip install coverage

And run coverage check with:

coverage run --source='.' manage.py test

Now run the report with:

coverage report

and you’ll see a lot of interesting informations. But for now there is not so much to see. Time to add some tests, for real!

How to test a Django application: creating our first (flawed link)

If I were to add a new link right now in my application the tags field would be a regular string. No way to relate tags with links. You can confirm that by registering your link in the Django admin panel. Open up ./links/admin.py and register the model like so:

from django.contrib import admin
from .models import Link

admin.site.register(Link)

Create a Django super user for logging into the admin panel:

python manage.py createsuperuser

and once done launch the website:

python manage.py runserver

Now head over http://127.0.0.1:8000/admin, log in, and try to add a new Link inside the app. You will see that the tags field is a simple string:

How to test a Django application: creating our first (flawed link)

Not so useful! At this point we’re ready to refactor and move our prototype to the next level.

How to test a Django application: setting up LiveServerTestCase

We have a good user story to start with:

“as a user when I click a tag I can see a list of related links”.

In this phase we have to choices: either we start with a functional test (LiveServerTestCase/Selenium or Cypress) or we can also go the easy way with a unit test.

When the pages do not require JavaScript and you’re just interested in testing the response there’s no better tool than TestCase, a Django class already included in the framework. There is also SimpleTestCase, a Django class for testing apps without a database.

But for testing our functionality we need to interact and click within the page so we’ll pick LiveServerTestCase. LiveServerTestCase relies on Selenium so we need to install it before getting started:

pip install selenium

Also, make sure to have Geckodriver in your PATH. That means downloading Geckodriver from Geckodriver releases, extracting the binary and moving it to ~/.local/bin (I’m talking about Linux here):

cd ~
wget https://github.com/mozilla/geckodriver/releases/download/v0.24.0/geckodriver-v0.24.0-linux64.tar.gz
tar zxf geckodriver-v0.24.0-linux64.tar.gz
mv geckodriver ~/.local/bin/

At this point we’re ready to write our first test. But let’s think a moment about it. What do we want to test? Remember the user story? As a user when I click a tag I can see a list of related links.

That means we are expecting at least a tag in the page (the url could be links), associated with a link. The tag should be clickable and when I click the tag I should see a list of related links.

Now, excuse me for this long function name but I would write this user story like so:

def test_as_a_user_when_click_tag_can_see_list_of_related_links(self):
    # visit the /links url
    # find tags inside the page
    # click on the first tag
    # check that url matches /tag/tagname
    # check that the page contains a list of related articles

But before launching the actual test we need to set up a couple of things inside our test file. We will import LiveServerTestCase, WebDriver, and our models. Then we will use two methods: setUpClass for instantiating WebDriver and populating the database on every test run, tearDownClass for quitting Selenium at the end of every test.

Let’s create the actual test now! Open up ./links/tests.py and create the following class:

from django.test import LiveServerTestCase
from selenium.webdriver.firefox.webdriver import WebDriver
from links.models import Link, Tag

class LinksApp(LiveServerTestCase):

    @classmethod
    def setUpClass(cls):
        super().setUpClass()
        cls.selenium = WebDriver()

    @classmethod
    def tearDownClass(cls):
        cls.selenium.quit()
        super().tearDownClass()


    def test_as_a_user_when_click_tag_can_see_list_of_related_links(self):
        pass
        # visit the /links url
        # find tags inside the page
        # click on the first tag
        # check that url matches /tag/tagname
        # check that the page contains a list of related articles

Save and close the file. In the next section we’ll finally getting to refactor!

How to test a Django application: when I click a tag I can see a list of related links (refactoring to relationships)

As it is, the test fails right away for one reason: we don’t have a Tag model. This new model can have just one string field. But this won’t solve our main requirement: every Link should be related to a Tag. This sounds like a one to many relationship right? In reality it’s not. We’re looking for a many to many relationship: a Link can have many tags and a single Tag can have many Links.

In Django we model this many to many relationship with a ManyToManyField. Check out the doc for more: Many-to-many relationships. So for fixing our model we need to: add a new Tag model; add a new ManyToManyField in our Link model, pointing to Tag.

Open up ./links/models.py and create the new model/relationship:

from django.db import models

class Tag(models.Model):
    tag = models.CharField(max_length=200, unique=True)

class Link(models.Model):
    title = models.CharField(max_length=300, unique=True)
    url = models.URLField(max_length=300, unique=True)
    tags = models.ManyToManyField(Tag)
    created_at = models.DateTimeField(auto_now_add=True)

In brief, we created a new model named Tag with a tag field. Second, we changed the tags field of Link to point at Tag. This refactor should be enough to make things work!

Don’t forget to apply the new migration:

python manage.py makemigrations links

and finally migrate the database with:

python manage.py migrate

If you register the new Tag model inside the admin panel you’ll be able to add new tags and to point every new Link to a bunch of tags. But we’re not interested in testing our app by hand: we will continue testing with LiveServerTestCase in the next section!

How to test a Django application: testing with Selenium and LiveServerTestCase

Now it’s really time to get serious with testing. With our models in place we can fill our database and start testing. First in our ./links/tests.py we want to populate the database before running tests. This is done in the setUp method:

def setUp(self):
    # Populate the database
    self.redux_link = Link.objects.create(
        title='Redux tutorial for beginners',
        url='https://www.valentinog.com/blog/redux'
    )
    self.redux_tag = Tag.objects.create(tag='redux')
    self.javascript_tag = Tag.objects.create(tag='javascript')
    self.redux_link.tags.add(self.redux_tag)
    self.redux_link.tags.add(self.javascript_tag)

    self.djangorest_link = Link.objects.create(
        title='Tutorial: Django REST with React (Django 2.0 and a sprinkle of testing)',
        url='https://www.valentinog.com/blog/tutorial-api-django-rest-react/'
    )
    self.djangorest_link.tags.add(self.javascript_tag)

As you can see we create two new links and a couple of tags. Those tags are added to our new links with .add(). Of course this is not the most efficient way for creating models during testing but it’s enough for getting you started.

PRO TIP: LiveServerTestCase accepts a class member named fixtures which is useful for loading models before testing. If you don’t want to use fixtures then FactoryBoy is your friend.

Next up we implement the actual test. This is done in that very long function test_as_a_user_when_click_tag_can_see_list_of_related_links. I know it’s too long. If you find yourself writing those long names I suggest splitting the test in two separate functions.

Now if I were to write this test in plain english I would say:

  1. visit the /links/ url
  2. find at least a tag inside the page
  3. click on that tag
  4. check that the new url matches /tag/tagname
  5. check that the page contains a list of related articles

And here’s how it looks like with Selenium:

def test_as_a_user_when_click_tag_can_see_list_of_related_links(self):
    self.selenium.get(f'{self.live_server_url}/links/')
    tag = self.selenium.find_element_by_partial_link_text('javascript')
    first_tag_text = tag.text
    tag.click()
    self.assertEqual(f'{self.live_server_url}/tag/{first_tag_text}/', self.selenium.current_url)
    titles = self.selenium.find_elements_by_tag_name('h2')
    self.assertEqual(self.redux_link.title, titles[0].text)
    self.assertEqual(self.djangorest_link.title, titles[1].text)

Open up ./links/tests.py and modify your class so it looks like the following:

from django.test import LiveServerTestCase
from selenium.webdriver.firefox.webdriver import WebDriver
from links.models import Link, Tag

class LinksApp(LiveServerTestCase):

    @classmethod
    def setUpClass(cls):
        super().setUpClass()
        cls.selenium = WebDriver()

    @classmethod
    def tearDownClass(cls):
        cls.selenium.quit()
        super().tearDownClass()

    def setUp(self):
        # Populate the database
        self.redux_link = Link.objects.create(
            title='Redux tutorial for beginners',
            url='https://www.valentinog.com/blog/redux'
        )
        self.redux_tag = Tag.objects.create(tag='redux')
        self.javascript_tag = Tag.objects.create(tag='javascript')
        self.redux_link.tags.add(self.redux_tag)
        self.redux_link.tags.add(self.javascript_tag)

        self.djangorest_link = Link.objects.create(
            title='Tutorial: Django REST with React (Django 2.0 and a sprinkle of testing)',
            url='https://www.valentinog.com/blog/tutorial-api-django-rest-react/'
        )
        self.djangorest_link.tags.add(self.javascript_tag)


    def test_as_a_user_when_click_tag_can_see_list_of_related_links(self):
        self.selenium.get(f'{self.live_server_url}/links/')
        tag = self.selenium.find_element_by_partial_link_text('javascript')
        first_tag_text = tag.text
        tag.click()
        self.assertEqual(f'{self.live_server_url}/tag/{first_tag_text}/', self.selenium.current_url)
        titles = self.selenium.find_elements_by_tag_name('h2')
        self.assertEqual(self.redux_link.title, titles[0].text)
        self.assertEqual(self.djangorest_link.title, titles[1].text)

Save and close the file. With the test in place you can run:

python manage.py test

and seeing it fail miserably. Turns out we need to wire up a bunch of things in Django before calling us victorious!

Head over the next section! (I promise we’re near the end).

How to test a Django application: fixing the test step 1, setting up the url

Django borrows from the MVC pattern and you’ll be tempted to create a controller somewhere in your code. But that’s not exactly the case because Django works slightly different.

There is a model of course, and in our app we have two of them: Link and Tag. But for making the test pass we need a couple of “routes” too. If you look at the test, Selenium will try to visit self.live_server_url/links/ and self.live_server_url/tag/first_tag_text/.

Those “routes” in Django are called URLconf and we’re going to define our new routes in a moment. Open up ./djangoproject/urls.py and configure a new empty url below admin for including links.urls:

from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('links.urls')), # our new url
]

Now we’re going to create two routes for the link app. We need links/, which is associated with a view for displaying all the links and tag/ which calls the same view for displaying related links. Now create a new file named ./links/urls.py and configure everything as follows:

from django.urls import path
from links import views

urlpatterns = [
    path('links/', views.index ),
    path('tag/<tag>/', views.index ),
]

When called, both urls will look for a view named index which we’re going to create in the next section.

How to test a Django application: fixing the test step 2, setting up the view and the template

So where are the controllers we were talking about? Well, in Django the are no controllers. Fetching data from the database and presenting it to the user is a job for Django views. There are many types of views in Django: function views, class based views, and generic views. With time you’ll learn to pick the right type of view for your use case but in its simplest incarnation a view is just a Python function.

Let’s create our view then! Open up ./links/views.py and create a function named index:

from django.shortcuts import render
from links.models import Link, Tag

def index(request, tag = ''):
    if tag:
        links = Tag.objects.filter(tag=tag).get().link_set.all()
    else:
        links = Link.objects.all()
    return render(request, 'links/index.html', { 'links': links } )

The role of this view is twofold.

It fetches a list of links related to a particular tag when the path contains tag, that is, when the user requests tag. This is done by checking that tag is not an empty string. Then the Django ORM takes care of getting the requested tag, alongside with a list of related links.

If the tag is empty the view falls back to getting a list of all the links. The default argument tag = ” makes sure that the route links/ will never error out.

In both cases the view returns a template, which has a context object attached (third argument of the render function). The context object will contain our data.

With this structure in place we’re ready to complete our application with a template, the last missing piece! A template in Django is a special kind of HTML file which gets interpred by the framework. Inside the template we can access our context object, that is, we can iterate over links and display data to our users.

Create a new folder named ./links/templates/links and inside this folder create a new file named index.html:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Links App</title>
</head>
<body>
<div>
    {% for link in links %}
        <div>
            <h2>{{ link.title }}</h2>
        {% for tag in link.tags.all %}
            <a href="/tag/{{ tag.tag }}/" class="tag">{{ tag.tag }}</a>
        {% endfor %}
        </div>
    {% endfor %}

</div>
</body>
</html>

As I promised we’re done. Save and close the file and finally run your test suite:

python manage.py test

The test should now pass!

How to test a Django application: fixing the test step 2, setting up the view and the template

You can confirm that everything is working fine by logging into admin, adding some links/tags and then checking http://127.0.0.1:8000/links/ and http://127.0.0.1:8000/tag/tagname/

And what about test coverage? You can try removing some lines of test from test_as_a_user_when_click_tag_can… and running coverage. You will see how the coverage varies depending on the lines you actually test.

Wow! We’re done! That was quite a lot but if you followed the tutorial you should now be able to test your Django websites with Selenium and LiveServerTestCase. In the process you learned the thought process behind testing and refactoring. You learned many to many relationship in Django, how Django works with urls, views, templates and how to wire up everything together.

At this point you’re ready to begin the testing journey … but not before checking the resources in the next section!

How to test a Django application: summary and resources

When developing a new project you will rarely nail the “right implementation” at the first try. Programming is an experiment: your design will have flaws and that class you were so fond of will be rewritten many times.

In this tutorial we went from a simple Django model to a many to many relationship between models. In Django we shape this kind of bond with a ManyToManyField.

It’s in the refactoring process that you can help yourself with tests. If you’re building a website you can start with a functional test. In Django you would use LiveServerTestCase and Selenium for testing simple webpages (no JavaScript).

In this project we tested two “routes”: links/ and tag/tagname/. The first route would show a list of links while the second displays a list of related links, given a single tag. The ManyToManyField in Django gives use the power to follow backward relationship between models.

If the frontend has a lot of JavaScript I suggest looking at Cypress (here’s an introduction to Cypress in Django REST). A functional test is just the starting point and you may want to pair that with unit testing. In Django there are TestCase and SimpleTestCase for unit testing your code. If you want to learn more about testing in Django check out this Testing tools FAQ.

Test coverage is the key if you want a clear hint of what code needs to be tested. In Python we have the coverage package.

Last but not least, I’m barely scratching the surface of testing in Django with this guide. If you want to learn more from a pro I cannot recommend enough Obey the testing goat by Harry Percival. I use his book whenever I need to refresh my testing skills.

Oh and if you want to expand from this tutorial then I recommend (shameless plug) another article of mine: Django REST with React (Django 2.0 and a sprinkle of testing)

Thanks for reading and happy testing!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.