Decoupling Django with Django REST (and a sprinkle of React) @ Pycon Italia 2019

I gave a talk about Decoupling Django with Django REST at Pycon Italia 2019. Here are the slides and the transcript!

Decoupling Django with Django REST (and a sprinkle of React) @ Pycon Italia 2019

It was a pleasure to attend and speak at the tenth edition of Pycon Italia. I gave a talk about Decoupling Django with Django REST. Here’s a loose transcript of the talk which you can use as a tutorial too, alongside with the slides and some pictures!

Who am I

I’m Valentino Gagliardi, JavaScript developer. I have a blog where I share everything I know about JavaScript and Django. I love Python and I use it whenever I can. Starting this year I’m serving as a coach for Django Girls and I encourage you spreading the word. We need more women in our field.

Decoupling Django with Django REST (and a sprinkle of React) @ Pycon Italia 2019

Agenda and goals of the talk

Let’s see the plan of this talk. There is a quick introduction to decoupling. Then we’ll see what is Django REST framework and what it can do for us. Next up we’ll see how to setup a JavaScript front-end (namely React). There is a section on testing too, both for the front-end and for the API. Finally we will see some resources.

Goal of this talk: in this talk I want to give you a loose guideline for structuring your Django project with React. Anyway, I don’t want to endorse React by any means. You can use Django and Django REST with any library of framework: VueJS, Angular. The concepts we’ll going to see apply everywhere.

In the talk I’ll also outline challenges and tradeoffs of a decoupled approach. By the end of the talk you should be able to configure a Django REST project with a front-end library.

The talk aims mostly at beginners. If you’re a more experience developer consider it as a re-introduction to Django and Django REST.

Introduction, what is decoupling?

A bit of backstory. Last year, while trying to scratch my clients hitches, I wrote a blog post on setting up Django REST with React. The article has been well received by the community. Brad Traversy created a series of videos based on my article.

But now let’s introduce decoupling. What is decoupling? Decoupling in Django means exposing data and contents outside Django templates, with the help of a REST API.

Decoupling in Django means exposing Django models as JSON entities. Let’s say you have a model named Link with a bunch of fields. If you expose that model as JSON you can end up with the following result:

Decoupling Django with Django REST: introduction, what is decoupling?

As you can see we have a JSON array with a bunch of objects. In the picture you can see the tags field which is a relationship to another Django model. We will see that in detail later.

Decoupling Django with Django REST: why decoupling?

Now let’s see why you may want to decouple. Because of flexibility for example. Back-end and front-end developers can work in isolation. Also, no need to fight build tools and we will see what it means later.

But what are the challenges of a decoupled approach? A decoupled approach can give a lot of headaches! Code duplication, more testing, more developers. The tragedy … ehm the rising of JavaScript imposes challenges to more traditional web frameworks.

In particular you should care about search engine optimization (SSR or die). If your Django application has a lot of textual contents and you rely on SEO then you may be better with a traditional approach (no single page application).

Authentication is an important topic too and when decoupling you lose the fantastic Django’s built-in authentication. Logic duplication is another issue. Django gives a lot out of the box (form validation for example), and if your front-end is pure JavaScript you will have to rewrite that logic.

With a decoupled front-end there is more work to do, more testing, and of course more developers.

Decoupling Django with Django REST: when to decouple and when NOT to decouple

Before getting started always ask yourself if decoupling is the right thing to do. These are good signs that you will benefit from decoupling:

  • the application has a lot of JS-driven interactions
  • you’re building a mobile app
  • you’re building a dashboard for internal use

The last case is interesting because if there are no textual contents exposed to the internet you can forget SEO and build a single page application. Another neat thing is that if your client uses a modern browser you can skip the “transpilation” process and serve modern JavaScript code.

In general always ask yourself when and if it makes sense to decouple. These are good signs that you don’t need to decouple:

  • the application has little or no JS-driven interactions, that is, content is mostly static
  • you’re concerned about SEO and you don’t want to introduce the complexity of a SSR
  • you want to decouple just because everyone is doing the same, that’s what I call “Medium driven development”

Decoupling Django with Django REST: Django REST framework and Django models

Decoupling Django with Django REST: Django REST framework

What is Django REST framework? Django REST framework is a Django application for building REST APIs on top of Django.

You can install the package inside a Django application with:

pip install djangorestframework

But first let’s talk a bit about Django models. For exposing JSON from your Django application you need at least some Django Models. Imagine a Link model with a bunch of fields. In the following example tags is a related ManyToManyField to another model called Tag:

from django.db import models


class Tag(models.Model):
    name = models.TextField(max_length=50, unique=True)
    created_at = models.DateField(auto_now_add=True)


class Link(models.Model):
    title = models.TextField(max_length=200, unique=True)
    url = models.URLField(max_length=250, unique=True)
    tags = models.ManyToManyField(Tag)
    created_at = models.DateField(auto_now_add=True)

With a couple of models in place we can move to create a Serializer.

Decoupling Django with Django REST: Django REST serializers

Serializers are a layer between Django models and the outside world (they mirror Django forms). With a serializer we can take complex Python objects and transform them to JSON. It works the other way around too: we can save JSON to Python objects.

ModelSerializer is the equivalent of a ModelForm. Of course you can also serialize simpler Python objects. In the following example we have a simple serializer extending from ModelSerializer:

from rest_framework import serializers
from .models import Link


class LinkSerializer(serializers.ModelSerializer):
    tags = serializers.StringRelatedField(many=True)

    class Meta:
        model = Link
        fields = ('title', 'url', 'tags')

The attribute named tags make possible to expose related models. Moreover we can specify what model to serialize and what fields to expose. One can also expose all the fields like so (although it is considered bad practice for obvious reasons):

from rest_framework import serializers
from .models import Link


class LinkSerializer(serializers.ModelSerializer):
    tags = serializers.StringRelatedField(many=True)

    class Meta:
        model = Link
        fields = '__all__' # Don't do that

And with a serializer and some models we’re ready to create a Django REST view.

Decoupling Django with Django REST: Django REST views and urls

With Django REST we can write views as plain functions, like in the following example:

from rest_framework.decorators import api_view
from rest_framework.response import Response
from .models import Link
from .serializers import LinkSerializer


@api_view(['GET'])
def link_list(request):
    links = Link.objects.all()
    serializer = LinkSerializer(links, many=True)
    return Response(serializer.data)

We can use the api_view decorator and specify which HTTP methods our view should accept. Now, that’s a contrived example, you might also have a view accepting both POST and GET requests. In that case you’ll create the logic for handling both methods. In the view we will specify what QuerySet to pass to the serializer, which in turn gives us a data attribute that will be returned to the user. And that will be a JSON response.

There is another and more concise way for creating views in Django REST: with a generic view. Consider this:

from rest_framework.generics import ListAPIView
from .models import Link
from .serializers import LinkSerializer


class LinkList(ListAPIView):
    serializer_class = LinkSerializer
    queryset = Link.objects.all()

With the above view we can expose a list of models to the consumer of the API. The generic view takes at least two attributes: serializer_class and queryset. I like a lot more this style in contrast to function views. Of course both have its own purpose. In general function view can serve specific requirements when there are a lot of customization to do.

Finally after creating a serializer and a view we can expose the API endpoint:

from django.urls import path
from .views import LinkList
PREFIX = 'api'

urlpatterns = [
    path(f'{PREFIX}/link/', LinkList.as_view()),
]
Decoupling Django with Django REST (and a sprinkle of React) @ Pycon Italia 2019

Setting up the front-end: What is React?

Now comes the hard part. But first let’s see what is React. React is a front-end library for building composable, reusable, user interfaces. You can install the library in a JavaScript project with:

npm install react react-dom

React builds upon the concept of components. With React Hooks we can also have function components with their own state. React components are simple as JavaScript functions. Consider the following example:

import React from "react";

function Form(props) {
  return <form onSubmit={props.onSubmit}>
        {props.children}
        </form>;
};

<Form onSubmit={handleSubmit}>
    <Input/>
    <Button/>
</Form>

We have a Form component for rendering an HTML form. The component takes some props, like onSubmit for handling the submit event. Then there’s children which is a prop used for passing child components.

React components are composable because as you can see I can pass whatever child component I need inside Form. That way I can compose components at will.

Now let’s glue things together. There are many ways for pairing up a JavaScript front-end to a Django REST API. We’ll take a closer look at two practical approaches.

Setting up the front-end: approach 1, create-react-app

Setting up the front-end: approach 1, create-react-app

I know this approach might look silly. It consists of a Django app with a React project made with create-react-app. A lot of beginners have the very same idea. Only to find out that when building the project you get a single HTML file with a bunch of script tags:

<!-- How about these? -->

<script src="/static/js/2.3310f33a.chunk.js"></script>
<script src="/static/js/main.c82d5dba.chunk.js"></script>

Turns out there is no way to pair that with Django templates for obtaining something like:

<!-- We do need these instead -->
{% load static %}
<script src="{% static "links/js/2.3310f33a.chunk.js" %}"></script>
<script src="{% static "links/js/main.c82d5dba.chunk.js" %}"></script>

But not everything is lost. We can use a custom webpack configuration to overcome the problem.

Setting up the front-end: approach 2, custom webpack configuration

Setting up the front-end: approach 2, custom webpack configuration

The second approach is based on a custom webpack project inside a Django app.

By doing that we now have control over the bundle path, which is totally fine for development:

// webpack.config.js
const path = require("path");

module.exports = {
  entry: "./index.js",
  output: {
    filename: "main.js",
    path: path.resolve(__dirname, 
            "../static/custom_webpack_conf_2/js")
  },
  module: {
        // other stuff
  }
};

This way we can say “webpack, spit out the bundle in that folder!”. Now we can reference the bundle file in our Django template:

<!-- Seems good! -->

{% load static %}
<script src="{% static "links/js/main.js" %}"></script>

And how about Django views? Since I suppose we’re building a single page application we can have a single view loading our bundle:

def index(request):
    return render(request, 'custom_webpack_conf_2/index.html')

Everything sounds good. But what if our applications grows? What if there are dozens of components? What if the bundle gets bigger? (200 KB of JavaScript is still to much for the first render). Obviously our approach won’t scale.

There is an option in webpack 4 called splitChunks. With splitChunks you can aggressively split the bundle. Ending up with a lot of files …

django webpack splitChunks

How do we inject these files into our Django template? Seems like we have the same problem again. There is a package that could help us with this madnes and it’s called django-webpack-loader. Except that it does not work with webpack 4 and splitChunks.

webpack is a moving target and it’s not always possible to keep up for open source mantainers. So what? I wish we had a “django-webpacker” like Rails has rails-webpacker. It’s a neat package for injecting chunks into templates.

Maybe we can build something similar together?

Decoupling Django with Django REST: fully decoupled

At the end of the story the winner is a fully decoupled front-end. Best of both worlds, you don’t fight the tools. You might also need django-cors-headers when hosting the back-end far from the front-end. And in that case authentication plays an important role in a decoupled setup. You have two options:

  • good ol’ Django session authentication
  • token based authentication (JWT)

If the JavaScript front-end is in the same context of Django then you can use the fantastic built-in authentication. If instead the front-end is outside Django then you should resort to a token based authentication with JWT. One of the best packages at the moment is djangorestframework_simplejwt:

pip install djangorestframework_simplejwt

And now let’s talk about testing.

Decoupling Django with Django REST: testing the front-end

Testing is an important topic. I’m a bit paranoiac about testing my code. There are a lot of options for testing the front-end:

  • Selenium
  • Cypress
  • Splinter

Selenium works fine but it cannot stub AJAX requests. Also there is no automatic awaiting for JavaScript interactions. Both are solved problems with Cypress which can stub the API and more important is able to wait for elements. There is also Splinter which is an abstraction over Selenium but suffers its same problems.

This is an example of testing in Cypress:

it("Test description (user story)", () => {
    cy.server();
    cy.visit('/');
    cy.route({
      method: "POST", url: "**/api/user/token/", status: 401,
      response: { access: undefined, refresh: undefined }
    });
    const emailInput = 'input[name="username"]';
    const passwordInput = 'input[name="password"]';
    cy.get(emailInput).type("xsasasa");
    cy.get(passwordInput).type("changeme");
    cy.get("[data-testid=submit]").click();
    cy.get("[data-testid=error-message]")
           .contains("Unauthorized");
});

We start a fake server, we give a fake JSON response and we assert that the frontend gives the “Unauthorized” message. It’s easy to see the boons of this approach. A front-end developer can work in isolation without the real backend and iterate faster.

Decoupling Django with Django REST: testing the back-end

We have a lot of options for testing the API too:

  • APISimpleTestCase
  • APITestCase
  • APILiveServerTestCase

This is how an API test would look like:

from rest_framework.test import APITestCase
from django.contrib.auth.models import User
from ..models import Link, Tag

class SimpleTest(APITestCase):

    def setUp(self):
        # CREATE SOME MODELS HERE

    def test_guest_cannot_see_links(self):
        response = self.client.get(self.path)
        self.assertEqual(response.status_code, 401)


    def test_authenticated_user(self):
        user = User.objects.create_user(username='valentino',
                                        password='changeme')
        self.client.force_authenticate(user=user)
        response = self.client.get(self.path)
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.data, [{"id": 1,
                                          "title": "Some title",
                                          "url": "someurl",
                                          "tags": ["django", "python"]}])

Here we’re testing both a login failure with the 401 error code and a successful login which should give some data back to the user. Of course you’ll most likely use some kind of model factory in the setUp method.

(Demo time! There were no time for a longer demo and I just showed a failed login in the front-end and a quick test with Cypress).

Demo time! There were no time for a longer demo and I just showed a failed login in the front-end and a quick test with Cypress

Decoupling Django with Django REST: conclusions and resources

Django REST framework is great! There is a lot out of the box like:

  • built-in pagination
  • authentication and permissions
  • throttling

And really, DRF is configuration over coding, you can wire up and API in minutes. Now, where to go from here this talk? You may want to investigate more advanced topics like DRF router and DRF viewset.

And here are some resources:

Thank you!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.