Valentino Gagliardi Blog

Building accessible web forms in Django

2025-03-15T11:36:39Z

A step by step reference to building accessible web forms in Django.

The complete source code can be found on GitHub.

Introduction

Accessibility in Django is taken very seriously. However, as with almost everything, there is no size that fits all, and there is only so much Django can offer by default without resulting too opinionated.

Recent version of Django are already improving the accessibility compartment.

Starting from Django 5.0 for example, form fields receive the aria-describedby attribute pointing to their help text if the latter is present. Also, invalid fields expose aria-invalid="true".

Starting from Django 5.1, fieldsets receive aria-describedby as well.

When dealing with web forms, we need to apply techniques to augment accessibility if needed, and be also very careful not to break existing capabilities.

This guide offers a collection of techniques for building accessible web forms in Django, by improving the already good defaults.

Disclaimer

Keep in mind that this reference is non-exhaustive and that the changes proposed in this post might not suit your project! There is no "one size fits all" in web accessibility: every project has its requirements and constraints.

Progressive enhancement

In this post we work our way by progressively enhancing the form by first using only standard HTML features. This is know as progressive enhancement.

This tecnique suggests that you should make your page work with HTML first, and only later add JavaScript as needed.

Sources:

Using progressive enhancement

Setting the ground: customizing form field group template project-wide

Then main requirement for this reference, in order to apply the techniques presented, is to customize the form renderer of your Django project so that you can provide your own form field group template project-wide.

Please follow Rendering form fields as group in Django before moving forward.

Provide descriptive field errors with aria-describedby

Technique: associate form fields and their respective errors with aria-describedby.

At the time of writing using aria-describedby to associate invalid fields with their error message is the most robust way for announcing errors to screen readers (see sources below).

To apply this technique in Django, we augment the default Django field group template by taking control of error rendering:

{% if field.use_fieldset %}
  <fieldset{% if field.help_text and field.auto_id and "aria-describedby" not in field.field.widget.attrs %} aria-describedby="{{ field.auto_id }}_helptext"{% endif %}>
  {% if field.label %}{{ field.legend_tag }}{% endif %}
{% else %}
  {% if field.label %}{{ field.label_tag }}{% endif %}
{% endif %}
{% if field.help_text %}<div class="helptext"{% if field.auto_id %} id="{{ field.auto_id }}_helptext"{% endif %}>{{ field.help_text|safe }}</div>{% endif %}
{{ field }}
- {{ field.errors }}
+ <ul id="{% if field.auto_id %}{{ field.auto_id }}_{% endif %}errorlist" class="errorlist">
+  {% for error in field.errors %}
+      <li class="error">{{ error }}</li>
+  {% endfor %}
+ </ul>
{% if field.use_fieldset %}</fieldset>{% endif %}

Here we wrap field errors in a list, the same as Django does by default, but setting in addition the id to the error list so that the error box will have id_fieldname_errorlist.

With the field associated to the relevant error list, we set aria-describedby in the Django form:

# books/forms.py

from django import forms
from books.models import Book


class BookForm(forms.ModelForm):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        for field_name, field in self.fields.items():
            auto_id = self.auto_id % field_name
            field.widget.attrs.update(
                {
                    "aria-describedby": f"{auto_id}_errorlist {auto_id}_helptext",
                }
            )

    class Meta:
        model = Book
        fields = ["title", "publication_date", "author"]
        help_texts = {
            "title": "Enter the title of the book.",
            "publication_date": "Enter the publication date.",
            "author": "Select the author of the book.",
        }

If a field becomes invalid after being validated on submit, Django will render the following markup :

<label for="id_title">Title:</label>
<div class="helptext" id="id_title_helptext">Enter the title of the book.</div>
<input
  type="text"
  name="title"
  maxlength="200"
  aria-describedby="id_title_errorlist id_title_helptext"
  required=""
  aria-invalid="true"
  id="id_title"
/>
<ul class="id_title_errorlist">
  <li>This field is required.</li>
</ul>

Here aria-describedby="id_title_errorlist id_title_helptext" points both to the help text and to the error box. This ensures that screen reader will properly announce the field errors to the user.

Testing field errors and aria-describedby

Testing field errors described with aria-describedby can be exercised with unbrowsed as follows (the example uses pytest):

import pytest

from unbrowsed import parse_html, get_by_role

@pytest.mark.django_db
def test_book_create(client):
    """Test post request to the book create view."""
    response = client.post(
        "/books/create/",
        {
            "title": "",
        },
    )
    dom = parse_html(response.content)
    get_by_role(dom, "textbox", description="This field is required. Enter the title of the book.")

Sources:

Disabling the default browser validation

Disabling the default browser validation might seem incomprehensible to most developers, but as experience teaches, and confirmed also by multiple sources, the default browser validation makes more harm than good. Therefore it is reasonable to disable it altogether.

What to do instead?

rely on server-side validation on submit as the default
progressively enhance with custom JavaScript validation if needed (shown later in the guide)

To disable the default browser validation, just put the novalidate attribute on your form element:

{% extends "common/base.html" %} {% block content %}
<form method="post" novalidate>
  {% csrf_token %}
  <div>{{ form.title.as_field_group}}</div>
  <div>{{ form.publication_date.as_field_group}}</div>
  <div>{{ form.description.as_field_group}}</div>
  <div>{{ form.author.as_field_group}}</div>
  <button type="submit">Submit</button>
</form>
{% endblock %}

Testing the novalidate attribute

Testing form attributes can be exercised with unbrowsed as follows (the example uses pytest):

import pytest

from unbrowsed import parse_html, get_by_role

@pytest.mark.django_db
def test_book_create(client):
    """Test get request to the book create view."""
    response = client.get("/books/create/")
    dom = parse_html(response.content)

    assert get_by_role(dom, "form").to_have_attribute("novalidate")

Sources:

Resources

htmx snippets and examples

2025-01-26T15:05:37Z

htmx event listeners

Colon vs double-colon

Use case

Listening for htmx-specific events and DOM events on the same element.

Code

<dialog hx-on:keydown="// do something here" 
        hx-on::after-swap="// do something here">
  <!-- some other element here -->
</dialog>

Explanation

In htmx we can listen for DOM events with the following syntax:

hx-on:keydown="// do something here"

Notice the colon between hx-on and the event name.

In contrast, listening to htmx-specific events such as afterSwap requires the double colon:

hx-on::after-swap="// do something here"

Resources

htmx with the HTML dialog element

Opening a dialog, removing HTML dialog children elements on Esc, on close

Use case

An HTML <dialog> element modal that gets opened after htmx swap and that is able to remove its children elements on Esc and close.

Code

<!-- Trigger element -->
<a href="/book/{id}/" hx-get="/book/{id}/" hx-target="#dialog" hx-swap="beforeend">The naked sun</a>

<!-- Dialog -->
<dialog id="dialog"
  hx-on:keydown="if (event.key === 'Escape' || event.key === 'Esc') { disposeElements.call(this); };"
  hx-on::after-swap="this.showModal();">
    <form method="dialog" hx-on:submit="disposeElements.call(this);">
        <button>Close the dialog</button>
    </form>
    <div id="content">
        <p>some content</p>
        <div>
            <p>(possibly with other elements nested) here</p>
        </div>
    </div>
</dialog>
<script>
  function disposeElements() {
    return this.closest('dialog').querySelector('#content')?.remove();
  }
</script>

Explanation

This dialog is always present in the DOM, but initially hidden as per the default. When the user clicks the trigger element, htmx swaps the partial HTML received from the endpoint declared in hx-get, and puts the content into the dialog.

After the swap, the modal opens itself:

hx-on::after-swap="this.showModal();"

Please note that using showModal() is preferred over the open attribute of the dialog since modals opened with showModal() can be closed with Esc.

When the user closes the dialog by clicking <button>Close the dialog</button>, the modal closes via method="dialog" on the form element. This triggers a submit event on the form, where we dispose of the content:

hx-on:submit="disposeElements.call(this);"

When the user closes the dialog by pressing the Esc key, this triggers a keydown event where we dispose of the content:

hx-on:keydown="if (event.key === 'Escape' || event.key === 'Esc') { disposeElements.call(this); };"

Resources

Rendering form fields as group in Django

2025-01-24T18:21:43Z

Field groups in Django

A welcome addition in Django 5.0, the concept of field groups makes a lot easier to customize the layout of our Django forms.

In the past, to take control of field layout, developers had to render field labels, help text, errors and fields one by one (or resort to use an external library).

Field group solve that, as we will see in a moment.

In this brief post I base my examples over a Django library app, with a model named Book. You can look up the structure in this repo.

Customizing the form renderer

As a first step to opt-in into Django form field groups, we need to customize the form rendered. This can be done by creating your custom renderer in a file named renderers.py, ideally in a common app in your project.

The custom form rendered must inherit from TemplatesSetting in order to make the field template swappable:

from django.forms.renderers import TemplatesSetting


class FormRenderer(TemplatesSetting):
    field_template_name = "common/field.html"

The renderer can then be used in settings.py:

INSTALLED_APPS = [
    ...
    # Remember always to include django.forms
    'django.forms',
    'common'
]

FORM_RENDERER = 'common.renderers.FormRenderer'

Pay attention: in order to be able to customize the field group template in the following section, django.forms must be in our INSTALLED_APPS!

Customizing the form field group template project-wide

Now that we declared our custom form renderer, we can proceed to customize the form field template by creating a new file named field.html in common/templates/common:

{% if field.use_fieldset %}
  <fieldset{% if field.help_text and field.auto_id and "aria-describedby" not in field.field.widget.attrs %} aria-describedby="{{ field.auto_id }}_helptext"{% endif %}>
  {% if field.label %}{{ field.legend_tag }}{% endif %}
{% else %}
  {% if field.label %}{{ field.label_tag }}{% endif %}
{% endif %}
{% if field.help_text %}<div class="helptext"{% if field.auto_id %} id="{{ field.auto_id }}_helptext"{% endif %}>{{ field.help_text|safe }}</div>{% endif %}
{{ field }}
{{ field.errors }}
{% if field.use_fieldset %}</fieldset>{% endif %}

This template is taken from Django itself.

You can now enrich it with any desired change, or leave it as it is.

Let's now create the view to handle the form.

Rendering fields: the create view

With the template in place, we can now create a Django view to display the form. I use a CreateView in this example, which generates its form automatically. If you need more customization over widgets or over the validation, you can declare your own ModelForm.

Here's the view to create books:

class BookCreateView(CreateView):
    model = Book
    fields = ["title", "authors"]
    success_url = "/"

Note: your custom field template will work well even with a regular Django Form.

With the view and the form in place, we can render the form in our template.

Rendering fields: the form template

We are now ready to render our Django form in a template. Here it is:

{% extends "library/base.html" %}

{% block content %}
    <form novalidate method="post">
        {% csrf_token %}
        <div class="form-group">
            {{ form.title.as_field_group }}
        </div>
        <div class="form-group">
            {{ form.authors.as_field_group }}
        </div>
        <button type="submit" class="btn btn-primary">Save</button>
    </form>
{% endblock %}

As you can see, we no longer need to call field attributes one by one to group the field into a containing HTML element.

We can just call form.field_name.as_field_group:

        ...
        <div class="form-group">
            {{ form.title.as_field_group }}
        </div>
        ...

Isn't that almost magic?

Calling form.field_name.as_field_group will render the label, any help text, the field error list, and the field itself.

Here's the HTML for the title field after a form submit:

<div class="form-group">
    <label for="id_title">Title:</label>
    <ul class="errorlist"><li>This field is required.</li></ul>
    <input type="text" name="title" maxlength="100" required="" aria-invalid="true" id="id_title">
</div>

Note: if you don't see any effect after customizing the field template in your project, check that django.forms is declared in INSTALLED_APPS.

Reusable form templates

Keep in mind that you can not only customize the field template, but also take control of the form template, if you need to do so. Check reusable form templates for more.

Conclusion

Reusable form field groups in Django make a lot easier to take control of form field rendering without reaching for external libraries.

Available since Django 5.0 form fields can render as a group, and have their template overridden in different ways.

This can happen at various levels as described also in the documentation.

Thanks for reading!

Progressive enhancement in Django with htmx

2025-01-22T17:05:12Z

Welcome reader to one of those articles where it seems I'm stating the obvious! But frontend developers constantly forget the most obvious things to do, so here I am.

I've always been a proponent of progressive enhancement, and in general of keeping things simple. However, building things this way might not always be possible for a number of reasons.

When finally the stars align and you get the chance to truly adopt progressive enhancement, patterns start to emerge quickly, and things finally click (no pun intended)!

In Django, for example, adopting progressive enhancement with htmx might be easier than you think.

Let's see!

Progressive enhancement in Django with htmx: the template

Consider the following example. To start with you've got a link augmented with hx-get. The conscious developer should ask itself first of all: what happens if JavaScript doesn't work, or if it's disabled at all in the user browser? For example, this won't work when the user clicks the link:

{% url "book-detail" book.pk as book_detail %}
<a hx-get="{{ book_detail }}">{{ book.name }}</a>

This instead will work, providing progressive enhancement to our users:

{% url "book-detail" book.pk as book_detail %}
<a href="{{ book_detail }}" hx-get="{{ book_detail }}">{{ book.name }}</a>

Magic!

If the user clicks the link and JavaScript is disabled, the browser will happily follow the link, providing the native link experience for the user. If JavaScript is enabled instead, htmx will get into action, fetch the content from the url provided in hx-get, and swap the latter into the given target.

Let's see into the next section how to handle the Django view.

Progressive enhancement in Django with htmx: the view

To make the magic of progressive enhancement happen in Django we need to serve different templates, depending on whether the request comes from JavaScript (via htmx) or from a regular browser HTTP request.

Consider the following Django app structure:

books/templates
└── books
    ├── partials
    │   └── book.html
    ├── books_detail.html

The arrangement here sees books_detail.html including the partial:

{# here you would extend from base.html #}

{% block content %}
  <div>
  {% include "books/partials/book.html" %}
  </div>
{% endblock content %}

Here's instead the partial book.html (where object is a book):

<div>
 {{ object.title }}
</div>

Finally here's the view:

# ... your imports here

class BookDetail(DetailView):
    model = Book
    template_name = "books/book_detail.html"

    def get(self, request, *args, **kwargs):
        if request.htmx:
            self.template_name = "books/partial/book.html"
        return super().get(request, *args, **kwargs)

The way this works is that:

when an full HTTP page request hits the view, Django will serve books_detail.html with a full page refresh like the good ol' days
when an htmx request via JavaScript hits the view, Django will serve the book.html partial, and htmx will do its job of swapping the partial content into the page

I'm using django-htmx here.

Was that hard? Not at all.

Did we create duplication? No, thanks to Django template inheritance.

Did I discover the obvious? Maybe! :-)

See A cheat sheet of common testing recipes for Django applications for suggestions on how to test this Django view.

Conclusion

In this post I presented a very simple technique to ensure that our Django interfaces works even when JavaScript is disabled.

If the request comes from htmx, it means JavaScript is enabled and functioning in the frontend: in our Django views we can serve a partial template that will be swapped into the DOM.

If the request does not come from JavaScript instead, we serve the full page and keep the experience working.

Thanks for reading!

Resources

Exploring techniques for untrusted Python code execution in agentic workflows

2024-12-06T18:41:03Z

If you ask me, one of the most compelling use cases for LLMs is the ability to let them invoke tools.

In LLM parlance, tools are functions that the model can invoke at will.

Type of tools

I like to distinguish between (for now) two type of tools the LLM can invoke:

safe(ish) tools
unsafe tools

Safe tools are functions written by the agent developer. These functions are (hopefully) well tested and devoid of security risks.

Unsafe tools on the other hand are function completely generated by the LLM, and then invoked by the very same. As we can easily imagine, this poses a number of security implications and risks.

Generating and executing code

Large Language Models (LLMs) have shown promise in automated code generation but typically excel only in simpler tasks such as generating standalone code units. That is, for simpler tasks, such as creating a doc or odt files, LLM are able to generate proper code (although not without guidance).

As of today, proper prompting, or the use of few-shot examples is necessary to have the model generate working code through a trial and error process.

Risks

LLMs can generate and execute their own code. At what cost? Here's a non-exhaustive list of the risks:

escaping
data loss
data leak
resource exhaustion

Data leaks and resource exhaustion are particulary terrifying.

Resource exhaustion can manifest itself in a number of ways since the model might generate:

code that gets stuck in infinite loops
regular expressions vulnerable to catastrophic backtracking

Mitigations

monitoring
human supervision
limiting access to data
data anonymization
preemptive testing
sandboxing

Human supervision is particularly challenging. An agent interacting with non technical people cannot be easily scrutinized for insecure code.

Preemptive testing is a technique which can potentially limit damage. The idea is to test the code generated by the LLM before executing it.

Considering the stakes, above all the mitigations proposed, sandboxing of the generated code is the most promising technique. However, sandboxing in Python is notoriously hard.

Executing code, the naive way

It's easy to fall in the trap to directly execute the code generated by an LLM. Think of this example:

@tool("execute_python_code")
def execute_python_code(code_string):
    """Execute the given Python code string and return the local variables."""

    try:
        global_vars = {}
        local_vars = {}
        exec(code_string, global_vars, local_vars)
        
    except Exception as e:
        print(f"Error executing code: {str(e)}")
        return None

This is highly insecure and risky.

Executing code, better ways

There are a number of ways to execute untrusted (Python in this case) code. While sandboxing Python is pratically impossible (see resources), there are a number of ways to control the potential damage.

HuggingFace Python interpreter tool

This Python interpreter from HuggingFace uses ast under the hood. It has the ability to scan the code string with a great degree of accuracy. It prevents infinite loops and has the ability to restrict imports. Here's an example usage:

from transformers.agents import PythonInterpreterTool

python_repl = PythonInterpreterTool(authorized_imports=["odf", "pathlib"])

def generate_odt(code_string: str):
    """Execute the given Python code to save an odt file by using a Python REPL."""
    return python_repl.forward(code_string)

Azure Container Apps dynamic sessions

One interesting technique for running untrusted Python code is offered by Azure Container Apps dynamic sessions.

This service provides strong isolated sandboxing environments, ideal for running LLM-generated code. Here's an example usage in Langchain:

import os

from langchain_azure_dynamic_sessions import SessionsPythonREPLTool
from langchain_core.tools import tool

from dotenv import load_dotenv

load_dotenv()

os.environ["AZURE_TENANT_ID"] = os.getenv("AZURE_TENANT_ID")
os.environ["AZURE_CLIENT_ID"] = os.getenv("AZURE_CLIENT_ID")
os.environ["AZURE_CLIENT_SECRET"] = os.getenv("AZURE_CLIENT_SECRET")

repl = SessionsPythonREPLTool(
    pool_management_endpoint=os.getenv("POOL_MANAGEMENT_ENDPOINT")
)

@tool("generate-odt")
def generate_odt(code_string: str):
    """Execute the given Python code to save an odt file in a Python REPL."""

    output_dir = Path("files")
    properties = repl.execute(code_string)

    if "status" in properties and properties["status"] == "Success":
        filename = properties["result"]
        try:
            output_dir.mkdir(exist_ok=True)

            repl.download_file(
                remote_file_path=filename, local_file_path=output_dir / filename
            )
            return output_dir / filename
        except HTTPError as e:
            return f"Error downloading file: {str(e)}"

    return properties

In this example we use the Azure Container Apps dynamic sessions REPL to run code generated by the LLM. The tool is able to execute the provided code in a sandboxed environment and then download the resulting file.

RestrictedPython

RestrictedPython helps reducing the risk surface when running untrusted code, but it's not enough alone. For example, AgentRun use a combination of Docker and Restricted Python to execute untrusted code.

Resources

LLM and tools

Code execution and sandboxing in Python

The failure of pysandbox
PyPy's sandboxing features
RestrictedPython
Simon Willison's has a lot of great resources about code execution in the context of LLM systems

django-filter: filtering a foreign key model property

2024-09-06T11:00:37Z

django-filter is a fantastic library that makes easy to add complex filters in Django. While it requires very little code to get up and running, it can be a bit tricky to figure out how to do more non-standard things, like for example filtering against a foreign key model property.

In this brief post we'll see with an example how I've achieved this.

The models

Consider two simple Django models, Book and Author:

from django.db import models


class Book(models.Model):
    title = models.CharField(max_length=100)
    author = models.ForeignKey("Author", on_delete=models.CASCADE)

    def __str__(self):
        return self.title


class Author(models.Model):
    first_name = models.CharField(max_length=100)
    last_name = models.CharField(max_length=100)

    @property
    def full_name(self):
        return f"{self.first_name} {self.last_name}"

    def __str__(self):
        return self.full_name

Book has a ForeignKey to Author, and Author has a full_name property that returns the full name of the author.

My use case is to be able to filter the Book list by the Author full_name property.

Django filter would make easy to filter against a model field, that is, things that are actual database columns, like author__first_name or author__last_name, but filtering against a model property like author__full_name is not so straightforward.

In other words, we would like to hit our page at /books/?author_full_name=clarke and get all the books written by Arthur C. Clarke.

The filter

To clarify things better, here's how the filter would look like if we were filtering against a model field:

from django_filters import FilterSet

from .models import Book


class BookFilterSet(FilterSet):
    class Meta:
        model = Book
        fields = ["author__last_name"]

This will work fine for hitting /books/?author_first_name=clarke. What happens instead if we try to filter against a model property? Like this:

from django_filters import FilterSet

from .models import Book

class BookFilterSet(FilterSet):
    class Meta:
        model = Book
        fields = ["author__full_name"]

Django filter will complain with this error:

TypeError: 'Meta.fields' must not contain non-model field names: author__full_name

Let's see how to fix this!

Fixing the filter, wiring up the view and the queryset

To make Django filter work with a model property, first off we need to define a custom filter for that property:

from django_filters import FilterSet, CharFilter

from .models import Book

class BookFilterSet(FilterSet):
    author_full_name = CharFilter(lookup_expr="icontains", label="Author full name")

    class Meta:
        model = Book
        fields = ["other_fields_here"]

Here, the BookFilterSet is instructed to use a CharFilter for the author_full_name property. In this example, we also use the icontains lookup expression.

Next up, we wire up the Django view alongside with the get_queryset method. The "trick" to make Django filter work with a model property is to use an annotation to add a new field to the queryset, that is, the author_full_name field:

from django.db.models import F
from django.db.models import Value as V
from django.db.models.functions import Concat
from django_filters.views import FilterView

from .models import Book

from .filters import BookFilterSet


class BookList(FilterView):
    model = Book
    filterset_class = BookFilterSet
    template_name = "books/book_list.html"

    def get_queryset(self):
        return (
            super()
            .get_queryset()
            .annotate(
                author_full_name=Concat(
                    F("author__first_name"), V(" "), F("author__last_name")
                )
            )
        )

Here's a breakdown of the queryset:

super().get_queryset() gets the base queryset
.annotate() adds a new field to the queryset author_full_name, by concatenating author__first_name and author__last_name fields.

With this change in place, Django filter is now able to filter against the author_full_name property. This way, we can hit /books/?author_full_name=clarke in Django, and have the filter work properly.

Here's the query again:

    def get_queryset(self):
        return (
            super()
            .get_queryset()
            .annotate(
                author_full_name=Concat(
                    F("author__first_name"), V(" "), F("author__last_name")
                )
            )
        )

Notice the use of F() and V() expressions. F() is used to reference a field in the database. The database function Concat() instead is used to concatenate the fields. If you want, you can read more about expressions in Query Expressions, and functions in Database Functions.

Conclusion

django-filter is a fantastic, easy to use, well-documented library for filtering in Django. While it works almost out of the box with model fields, it requires a bit of extra work to handle Django model properties.

In this post, we saw how achieve that by using an annotation to add a new field to the queryset, and then using that field in the filter.

Another alternative, if you don't mind using a third party library, is to use django-queryable-properties.

Thanks for reading!

Implementing the strategy pattern in Django with JavaScript import maps

2023-09-08T14:02:45Z

The strategy pattern is a software design pattern which lets you swap an algorithm at runtime. From wikipedia:

the strategy pattern (also known as the policy pattern) is a behavioral software design pattern that enables selecting an algorithm at runtime. Instead of implementing a single algorithm directly, code receives run-time instructions as to which in a family of algorithms to use

Lately I've been experimenting with a new web feature, import maps, and it turns out, they seem to play nicely in Django to implement the strategy pattern for JavaScript code in the frontend.

With simple examples, we're going to introduce the motivation behind this technique, and the actual implementation.

The need for the strategy pattern

Let's imagine a generic piece of JavaScript code in our Django project which needs to behave differently depending on the context where it's executed.

Let's imagine that if I'm on a page with books/ in the pathname, I want to write "Hello import maps from the books app!", whereas when I'm in the authors/ path I want to write "Hello import maps from the authors app!".

This can be achieved in a number of ways.

The suboptimal way is to have our code take decisions with a series of if statements. Consider the following example:

function renderText() {
    if (window.location.pathname === 'books/'){
        document.querySelector('p').innerText = "Hello import maps from the books app!";
    }

    if (window.location.pathname === 'authors/'){
        // do something else
    }
}

Without going too far, you can easily guess what this code can become. Is there a better solution?

What if instead we can import one or more functions (or variables) from a "swappable" JavaScript import? Consider the following code:

import { renderText } from "strategy";

renderText();

Here we import renderText from the strategy JavaScript module.

The calling code does not know where strategy is coming from. It has only to execute.

It's worth mentioning that here we are talking about JavaScript ES modules, denoted by the syntax import ... from "module-name".

It's also important to understand that ES modules are static, that is, they cannot be changed at runtime, and that up until recently, bare imports in the browser where not supported.

To make things clearer, up until recently, the syntax import { renderText } from "strategy" wasn't valid in the browser, being "strategy" a bare import. This syntax was supported only by module bundlers like webpack.

However, recently, with the introduction of JavaScript import maps, born to allow control over what URL gets fetched by JavaScript import statements, we have a way to use bare imports in the browser, and also to potentially swap the import URL at runtime as we will see in a moment.

Pair that with the flexiblity of the Django templating system, and we can now have a neat way to implement a simple version of the strategy pattern in our JavaScript code.

Let's see this technique in practice!

Implementing the strategy pattern with import maps

Implementing the strategy pattern in Django with JavaScript import maps exploits the ability of Django templates to extend and overwrite a given template block.

In brief, here's how it works:

a child template extends a parent template
the child template overrides the js block to declare its import map

Let's see it in practice. For our example we have a Django project composed of three apps:

core, containing the base template and the base JavaScript file
authors, extending the base template from core
books, extending the base template from core

The core app

The folder structure of the core app:

core
├── admin.py
├── apps.py
├── __init__.py
├── migrations
│   ├── __init__.py
├── models.py
├── static
│   └── core
│       └── js
│           └── Base.js
├── templates
│   └── core
│       └── base.html
├── tests.py
└── views.py

The base Django template core/templates/core/base.html declares Django template blocks that inheritors can override, namely content and js. In addition, it loads a JavaScript module:

{% load static %}
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>The base template</title>
</head>
<body>
{% block content %}{% endblock %}
</body>
{% block js %}
	<script type="module" src="{% static "core/js/Base.js" %}"></script>
{% endblock %}
</html>

In detail:

<script type="module" src="{% static "core/js/Base.js" %}"></script>

This is the base JavaScript file, a piece of logic that will behave differently depending on the page where it's called.

The base JavaScript file core/static/core/js/Base.js imports and executes any function (or constant) it needs from a strategy, without even knowing where this file is. Here's the file content:

import { renderText } from "strategy";

renderText();

Let's now see the two other Django apps.

NOTE: the module import name can be anything you want, not necessarily "strategy", as long as you reference the same name in child templates as well.

The authors app

The authors app is one of the apps using the base template from core.

The folder structure of the authors app:

authors
├── admin.py
├── apps.py
├── __init__.py
├── migrations
│   ├── __init__.py
├── models.py
├── static
│   └── authors
│       └── js
│           └── Authors.js
├── templates
│   └── authors
│       └── list.html
├── tests.py
├── urls.py
└── views.py

The inheritor template in authors/templates/authors/list.html extends core/templates/core/base.html, and then overrides the Django js block to declare its own import via import maps:

{% extends "core/base.html" %}
{% load static %}
{% block content %}
	<p></p>
{% endblock %}
{% block js %}
<script type="importmap">
    {
        "imports": {
            "strategy": "{% static "authors/js/Authors.js" %}"
        }
    }
</script>
{{ block.super }}
{% endblock %}

Now in authors/static/authors/js/Authors.js we can export our strategy function:

function renderText() {
    document.querySelector('p').innerText = "Hello import maps from the authors app!";
}

export { renderText };

The bit of HTML inside the js block is the key:

<script type="importmap">
    {
        "imports": {
            "strategy": "{% static "authors/js/Authors.js" %}"
        }
    }
</script>

What happens now is that by loading this template via a Django view, the browser is smart enough to map the strategy import to the actual JavaScript file authors/static/authors/js/Authors.js so that our code can import renderText properly.

The books app

The books app is one of the apps using the base template from core.

The folder structure of the books app:

books
├── admin.py
├── apps.py
├── __init__.py
├── migrations
│   ├── __init__.py
├── models.py
├── static
│   └── books
│       └── js
│           └── Books.js
├── templates
│   └── books
│       └── list.html
├── tests.py
├── urls.py
└── views.py

Using the same mechanism, the inheritor template in books/templates/books/list.html extends core/templates/core/base.html, and then overrides the Django js block to declare its own import via import maps:

{% extends "core/base.html" %}
{% load static %}
{% block content %}
	<p></p>
{% endblock %}
{% block js %}
<script type="importmap">
    {
        "imports": {
            "strategy": "{% static "books/js/Books.js" %}"
        }
    }
</script>
{{ block.super }}
{% endblock %}

In the file books/static/books/js/Books.js we can export our strategy function:

function renderText() {
    document.querySelector('p').innerText = "Hello import maps from the books app!";
}

export { renderText };

Again, let's take a look at the import map inside the js block:

<script type="importmap">
    {
        "imports": {
            "strategy": "{% static "books/js/Books.js" %}"
        }
    }
</script>

Again, the browser here will map strategy to the actual JavaScript file books/static/books/js/Books.js, so that our code can import renderText properly from this file.

This simple technique is an implementation of the strategy pattern: the theory here is that we can dynamically point our JavaScript import to a different location, depending on the Django template.

Conclusion

By combining Django templates, JavaScript import maps, and a bit of creativity, we can implement the strategy pattern for our JavaScript frontend code.

This technique can lead to high code reusability, and most important, modularity, two important traits of any healthy codebase.

import maps are available in all modern browsers, so you can start using them today!

Thanks for reading!

Suggested resources

Avoiding death by thousands ifs with the strategy pattern

2023-03-04T13:36:23Z

Most of the time, in JavaScript and TypeScript codebases structured as procedural code, design patterns are often neglected in favor of spaghetti code which more often than not leads to an unmaintainable mess.

In this brief post we will see how to apply the strategy pattern to your frontend code, so you can forget those nasty ifs, and sleep at night.

What is the strategy pattern and when it is useful?

The strategy pattern is a software design pattern which lets you swap an algorithm at runtime.

For example, one common application for the strategy pattern is a piece of code which needs to behave differently depending on the context where it's executed.

To keep things practical, let's say you have a UserRegistry which can read and persist users to a storage.

The problem is that you don't know beforehand what the storage is and where it will persist the data. In the context of a frontend application, we can imagine that at some point the storage could be localStorage or even IndexedDB.

UserRegistry should not care what the actual implementation of the storage is, as long as it conforms to a particular interface which exposes save and read methods.

Without the strategy pattern, the calling code might do something like:

if storage === "localStorage" { 
    // save and read logic specific for localStorage
}

else if storage === "IndexedDB" {
   // save and read logic specific for IndexedDB
}

else if storage === "anotherStorage" {
   // save and read logic specific for anotherStorage
}

Without going too far, you can easily guess what this code can become.

Personally, I'm ok with if statements at the tactical level, but not at the strategic level:

Tactical level: a function doing a small computation, or checking if some input is present, or if it's of the expected shape. In this case it's ok to have a couple of if or an if with an early return.
Strategic level: a piece of code needs to behave differently depending on where it's called. In this case it's not feasible to handle conditions with an if.

Using if statements at the strategic level can lead to an explosion of conditional logic. This is when the strategy pattern comes in.

Avoiding death by thousands ifs with the strategy pattern

Here's a simple example of the strategy pattern in TypeScript. I'm not a fan of JavaScript classes, and whenever possible I use objects as follows.

First off, we define an interface for a generic storage strategy:

interface StorageStrategy<TItem> {
    get(id: string): TItem | null;
    save(item: TItem): void;
}

This is an interface that concrete implementations can conform to.

Next up, we use the interface for a more specific strategy which, let's imagine, talks to localStorage:

const LocalStorageStrategy: StorageStrategy<Person> = {
    get(id: string) {
        // Get from local storage
    },

    save(item: Person) {
        // Save to localstorage
    },
};

Let's now define an interface for our program, an imaginary registry to save and read entities from a storage:

interface Registry<TItem> {
    storage: StorageStrategy<TItem> | null;
    init(strategy: StorageStrategy<TItem>): void;
    persist(item: TItem): void;
    read(id: string): TItem | null;
}

Objects conforming to the Registry interface can be initialized with a StorageStrategy. The registry does not care what the concrete implementation of the strategy is, as long as it conforms to the StorageStrategy interface.

Here's the concrete implementation of our program:

const UserRegistry: Registry<Person> = {
    storage: null,

    init(strategy) {
        this.storage = strategy;
    },

    persist(item) {
        this.storage?.save(item);
    },

    read(id) {
        return this.storage?.get(id) || null;
    },
};

Finally, this is how we can use our UserRegistry. Here's the initialization where we pass the concrete implementation of the storage:

UserRegistry.init(LocalStorageStrategy);

How is this better than the if version? Imagine that UserRegistry is used in multiple pages of you application, and depending on where it's used, it should use a different type of storage. In a page you'll do the following:

UserRegistry.init(LocalStorageStrategy);

In another page instead you'll do the following:

UserRegistry.init(IndexedDBStrategy);

Let's also not forget that with this arrangement, you can change the underlying algorithm at runtime.

This is way better than:

if storage === "localStorage" { 
    // save and read logic specific for localStorage
}

else if storage === "IndexedDB" {
   // save and read logic specific for IndexedDB
}

else if storage === "anotherStorage" {
   // save and read logic specific for anotherStorage
}

Also, here's how we read data from the storage:

UserRegistry.read("15");

Here's instead how we persist data to the storage:

UserRegistry.persist({
  id: "12",
  name: "Blaise",
  age: 11,
});

In both cases, read and persist interact with the concrete implementation of the storage.

This can be easily adapted to plain JavaScript if you have the need to do so.

Thanks for reading!

Suggested resources

Practical Python Design Patterns by Wessel Badenhorst
Design Patterns: Elements of Reusable Object-Oriented Software
Fluent Python by Luciano Ramalho shows how to use the strategy pattern with functions in chapter 10

Single-pages without the single-page with django2-tables, django-filter, and htmx

2023-02-12T17:42:06Z

Introduction

I've been meaning to use htmx since it came out, but I've never had time, nor the occasion. Now the opportunity finally came to refactor an old Django view which uses the Datatable jQuery plugin.

I wanted to try something fresh, and htmx seemed the way to go, paired with a couple of great libraries: django2-tables and django-filter.

Let's see how they play well together!

Please, take this post as personal notes, don't expect a step-by-step tutorial :-)

The theory

htmx is a JavaScript library for building dynamic user interfaces which lets you enrich HTML elements with "magic" attributes.

With htmx, any actionable HTML element can make XHR requests. The magic comes from two htmx attributes:

hx-get
hx-target

hx-get basically says: when the user clicks this element, make a GET request to the given URL, then swap the content of hx-target with the partial response. (POST requests are also supported).

The basic principle behind htmx used in the context of server-side web frameworks is the following: if the frontend request comes from htmx, we return a partial HTML fragment instead of the whole document.

In Django, this translates to: if the request comes from htmx, we return a partial template instead of the whole HTML template.

This approach can help give a single-page feeling to traditional web applications.

The recipe

My use case is a sortable/filterable table built with django2-tables and django-filter. For this view I needed a single-page feeling, but zero time to build one!

The building blocks used in this post are:

Django
django2-tables which gives nice datatables out of the box
django-filter which adds filtering capabilities to Django views
htmx
django-htmx, a set of nice tools to integrate htmx into Django

The view

The Django view I used in the project looks more or less like this:

class ProductList(SingleTableMixin, FilterView):
    table_class = ProductTable
    template_name = "products/product_list.html"
    model = Product
    filterset_class = ProductFilter

    def get(self, request, *args, **kwargs):
        if request.htmx:
            self.template_name = "products/htmx/table.html"
        return super().get(request, *args, **kwargs)

    def get_queryset(self):
        return Product.objects.all()

This view uses SingleTableMixin from django2-tables, and subclasses FilterView from django-filters, as described in Filtering data in your table.

For the scope of this post I'll gloss over how ProductFilter is configured: if you want to try things out, a simple configuration as described in the django-filter tutorial will suffice.

What's worth noting in this view is the get method: if the request comes from htmx, it swaps the Django template to return a partial from products/htmx/table.html, instead of the whole products/product_list.html.

Here I'm also using HtmxMiddleware from django-htmx which adds the htmx object to each request. Very nice abstraction.

The templates

Let's now take a look at the templates. To make this work we need two templates:

products/product_list.html
products/htmx/table.html

The first template, products/product_list.html, simply includes the table partial:

{% extends "base.html" %}
{% load static %}
{% load render_table from django_tables2 %}

{% block content %}
    {% include "products/htmx/table.html" %}
{% endblock %}

The second template instead, products/htmx/table.html, is the partial:

{% load render_table from django_tables2 %}
{% render_table table %}

This is the partial template which gets returned from Django for any request coming from a htmx call.

Let's now see how all the pieces fall into place!

Single-page-like sorting

django2-tables adds links to each sortable column. This is how a header cell looks like:

<th class="orderable">
    <a href="?sort=user__name">
        Name
    </a>
</th>

To get this under the control of htmx we need to add the two attributes you saw in the introduction:

hx-get
hx-target

Now, you can add these attributes to the table header cells rendered by django2-tables by subclassing the default template django_tables2/table.html in products/htmx/table.html. This is how the template should look like:

{% extends "django_tables2/table.html" %}

{% load render_table from django_tables2 %}
{% load querystring from django_tables2 %}

{% block table.thead %}
    {% if table.show_header %}
        <thead {{ table.attrs.thead.as_html }}>
        <tr>
            {% for column in table.columns %}
                <th {{ column.attrs.th.as_html }}>
                    {% if column.orderable %}
                        <a hx-get="{% querystring table.prefixed_order_by_field=column.order_by_alias.next %}"
                           hx-target=".table-container"
                           href="#">{{ column.header }} </a>
                    {% else %}
                        {{ column.header }}
                    {% endif %}
                </th>
            {% endfor %}
        </tr>
        </thead>
    {% endif %}
{% endblock table.thead %}
{% render_table table %}

This is the most important part:

<a hx-get="{% querystring table.prefixed_order_by_field=column.order_by_alias.next %}"
   hx-target=".table-container"
   href="#"> {{ column.header }} </a>

These attributes instruct htmx to send a GET request to Django, which will return the partial template, and to swap the content of the element .table-container with the partial response.

Single-page-like filtering

The principle to make django-filter work with htmx is the same you already saw above, but this time we need to change products/product_list.html, which is the container template where the filter form lives.

To give the filter a single-page-like feeling, you can render django-filter form as follows:

{% extends "base.html" %}
{% load static %}
{% load render_table from django_tables2 %}

{% block content %}
    {% if filter %}
        <form hx-get="" hx-target=".table-container" method="get">

            # Your django-filter fields here ...

            <button type="submit">Apply filters</button>
        </form>
    {% endif %}

    {% include "products/htmx/table.html" %}
{% endblock %}

Again, you can see how the form is progressively enhanced with htmx:

<form hx-get="" hx-target=".table-container" method="get">

Now, any filtered request coming from the form will be controlled by htmx, which will swap the target container .table-container with the partial response.

Note: .table-container gets added out-of-the-box by django2-tables

Single-page-like pagination

Coming soon

Is htmx any good?

I should admit that htmx is great for adding the single-page-like feeling to Django views and I quite enjoy working with it. I should also add that at this stage I've still not enough elements to judge its scalability and maintainability, especially for more complex use cases. Time will tell!

Update: after using it on a bunch of projects, I'm more and more convinced that htmx is a great tool for building MVPs, but it's also important to not take it to the extremes.

Thanks for reading and stay tuned!

Today I re-learned: Python function default arguments are retained between executions

2023-01-18T16:01:51Z

This is the story of a tricky bug in a Python application I maintain that was pestering me.

So, the problem is this. Given the following Python code, I expect the method is_ongoing to return False when today is not included between self.start_date and self.end_date, and to return True when today is included between self.start_date and self.end_date. Simple as it is:

def is_ongoing(self, today = datetime.date.today()):
    return (today >= self.start_date) and (today <= self.end_date)

This method is used to respectively show or hide a piece of interface to users. But the code wasn't working as expected.

The comparison would produce False if the user happened to trigger is_ongoing in a day not included between self.start_date and self.end_date, and the function would continue to return False on each subsequent call. Weird!!

Tests say more than a thousand words ...

I couldn't understand why an apparent simple comparison between two dates was not working as expected, so having a bit of spare time I decided to put the code under test, since this is some code I inherited, and not everything there is covered by tests.

Here's the test I set up:

def test_lesson_status(self):
    self.lesson.start_date = datetime.date(2023, 1, 25)
    self.lesson.end_date = datetime.date(2023, 1, 27)

    with time_machine.travel(datetime.datetime(2023, 1, 18)):
        self.assertFalse(self.lesson.is_ongoing())

    with time_machine.travel(datetime.datetime(2023, 1, 27)):
        self.assertTrue(self.lesson.is_ongoing())

To test the passage of time I used the excellent time-machine by Adam Johnson, which by the way seems to be an order of magnitude faster than another competing library, freezegun.

Turns out, by running the test, I surprisingly discovered that the second assertion was always failing: a lesson starting 2023-1-25, ending 2023-1-27 is not ongoing if today is 2023-1-18, but it should be absolutely ongoing if "today" is 2023-1-27.

At this point I quickly understood that someway, somehow, Python was "remembering" the initial state of the default argument today. Here's the function again:

def is_ongoing(self, today = datetime.date.today()):
    return (today >= self.start_date) and (today <= self.end_date)

Writing JavaScript almost every day, I took for granted that function default arguments in Python were recomputed on each call like JavaScript engines do.

Turns out instead that Python’s default arguments are evaluated once when the function is defined, not each time the function is called. Also, relevant: Python Mutable Defaults Are The Source of All Evil.

I guess the Python interpreter creates a closure over the function, effectively retaining its scope.

The funny thing is that this code was written by Python developers more expert than me, nobody ever noticed this bug, and most important, it was an easy fix:

def is_ongoing(self):
    today = datetime.date.today()
    return (today >= self.start_date) and (today <= self.end_date)

Moving today inside the function fixed the thing: what I want is to recompute the date on every invocation, and this function wasn't used anywhere else, thus there is no need for a default argument.

Key takeaways

Today I re-learned that Python function default arguments are retained between executions. I'm sure I fell on this trick in the past, so better write a couple of words about it for my future self.

Know your tools: I kind of know most of the machinery behind JavaScript engines, and I try to stay updated about it. Understanding why and how the interpreter of your main language works is important as writing good code.

Test, test, test: I know this has been said a billion of times, but I cannot stress enough why testing even simple things is important. Sometimes we neglect to test simple implementations because they appear so trivial most of the time, but it's there that silly bugs lie.

Use a linter: fellow developers on Reddit kindly pointed out that flake8-bugbear can catch this type of "bugs". The bugbear rule is B008, which reports the following:

B008 Do not perform function calls in argument defaults.  The call is performed only once at function definition time. All calls to your function will reuse the result of that definition-time function call.  If this is intended, assign the function call to a module-level variable and use that variable as a default value.

Thanks for reading!