For many people, one of Django's best selling points is the Admin app. And as wonderful as it is, there often comes a point where you need/want more. With the changes that landed for Django 1.0 (newforms-admin), adding what you want to the admin area has become a very manageable task.

This post will be the first in a short series of posts on customizing the Admin to your liking. Over the next several days, I plan to cover:

• Customizing Admin Templates
• Adding Multiple Deletes
• Integrating Custom Javascript/CSS
• Altering Forms
• Adding Non-Model Related Functionality

Customizing Admin Templates

Since the admin is just a Django application, if you understand Django's templating, you already know virtually everything you need to know to customize the look and feel of your admin area.

The default templates are located in django.contrib.admin.templates.admin. These templates follow the "three-level" convention suggested in the Django documentation, which means that the base.html defines in very basic terms how the entire admin ("sitewide") looks and is laid out, the base_site.html defines how the specific admin site ("sectionwide") is setup and the individual templates control their specific chuck of output.

A common goal in customizing the look of the admin is to make it match (or at least use similar colors) as the site itself, as well as customizing text to be specific to that site. So that's what we'll do, converting the generic Django admin into something... toastier. Let's get started.

Setup

Within your templates directory (or one of your template directories), you'll want to create a new, top-level directory called admin. If you're OK with Django's default styling, this is the only thing you will have to do for the templates.

If you want even further control of the admin CSS/images/Javascript, you'll have to copy the contents of django.contrib.admin.media to a place where your customized media can be served. Depending on your setup, this may be a new directory in your existing media or setting up a whole virtual host specific to serving this media. In addition, you will have to set your ADMIN_MEDIA_PREFIX to point to this location (path in first case or full URL for the second).

Changing The Text

This is perhaps the easiest customization to make to the admin. To change the "Django administration" text, simply copy the django/contrib/admin/templates/admin/base_site.html template to your admin directory in your templates directory and start editing it. You'll find a {% trans %} tag (which allows this text to be localized) with the "Django administration" text in it.

Before:

{% extends "admin/base.html" %}
{% load i18n %}

{% block title %}{{ title }} | {% trans 'Django site admin' %}{% endblock %}

{% block branding %}
<h1 id="site-name">{% trans 'Django administration' %}</h1>
{% endblock %}

{% block nav-global %}{% endblock %}

Simply replace it with what you want and save. In my case, I've replaced the title text as well because I'm anal-retentive like that.

After:

{% extends "admin/base.html" %}
{% load i18n %}

{% block title %}{{ title }} | {% trans 'Toast Driven Admin' %}{% endblock %}

{% block branding %}
<h1 id="site-name">{% trans 'Toast Driven Admin' %}</h1>
{% endblock %}

{% block nav-global %}{% endblock %}

This new text will now appear in both the admin login screen as well as all the internal admin pages.

Changing The Styling

The Django admin usually follows generally regarded web design best practices, so most of the CSS is kept out of the templates themselves and in an included CSS file. To apply some colors that are more in line with the site (in this case Toast Driven), we'll start by copying over the admin media and editing layout.css to modify the header colors.

Before:

...
#header { background:#417690; color:#ffc; overflow:hidden; }
#header a:link, #header a:visited { color:white; }
#header a:hover { text-decoration:underline; }
#branding h1 { padding:0 10px; font-size:18px; margin:8px 0; font-weight:normal; color:#f4f379; }
#branding h2 { padding:0 10px; font-size:14px; margin:-8px 0 8px 0; font-weight:normal; color:#ffc; }
...

The lines we're concerned with are the #header and #branding h1 declarations. We'll change them like so:

After:

...
#header { background:#FAD9A4; color:#9A6002; overflow:hidden; }
#header a:link, #header a:visited { color:white; }
#header a:hover { text-decoration:underline; }
#branding h1 { padding:0 10px; font-size:18px; margin:8px 0; font-weight:bold; color:#9A6002; }
#branding h2 { padding:0 10px; font-size:14px; margin:-8px 0 8px 0; font-weight:normal; color:#9A6002; }
...

To get the tops of apps on the dashboard, you'll need to edit global.css like so:

Before:

...
.module h2, .module caption, .inline-group h2 { margin:0; padding:2px 5px 3px 5px; font-size:11px; text-align:left; font-weight:bold; background:#7CA0C7 url(../img/admin/default-bg.gif) top left repeat-x; color:white; }
...

After:

...
.module h2, .module caption, .inline-group h2 { margin:0; padding:2px 5px 3px 5px; font-size:11px; text-align:left; font-weight:bold; background-color:#9A6002; color:#9A6002; }
...

We have to remove the background image (as it is a gradient with the blue) in order to affect the color choice.

Altering The Dashboard View

Finally, you can reorganize the main page you're presented with when you login ("Home"). Simply copy the index.html template over to your admin template folder and override it to show apps in the order you want, leave apps out or change it in even more grandiose ways.

Note - Django (before 1.0) used to have a command called adminindex that would generate a snippet corresponding to your current admin. This appears to no longer be present in Django 1.0, so you'll likely have to fall back on making changes to the existing template.

Conclusion

This is a bit outside my standard faire, so what I've presented here is simply what has worked for me in the past. I'd be open to hear different/better ways to handle admin look/feel customizations.

Nick had a nice post about setting DEBUG based on the hostname of the server that you're site is running on. This allows you to set DEBUG to True for your staging site, and False for your production site.

I do something along those lines, but a little bit differently. I can't take credit for this idea, because it came from this snippet. It is a really neat trick, that I have expanded on a little bit.

from django.views.debug import technical_500_response
import sys
from django import settings

class UserBasedExceptionMiddleware(object):
    def process_exception(self, request, exception):
        if request.user.is_superuser or request.META.get('REMOTE_ADDR') in settings.INTERNAL_IPS:
            return technical_500_response(request, *sys.exc_info())

Now simply save this in a file somewhere. Add it to your MIDDLEWARE_CLASSES, and you are good to go. For example, mine looks like:

    'tools.middleware.superuser.UserBasedExceptionMiddleware',

This is a pretty simple middleware that is crazy useful. When you throw this inside of your site, it will give you a normal Django error page if you're a superuser, or if you're IP is inside INTERNAL_IPS.

This makes it really nice, because you can get an error message on your production servers, where your normal users get your normal pretty 500 pages. This makes debugging things that are showing up in production, but won't be reproduced on the staging server possible. Caching behavior is a big one that I know isn't tested when you are using DEBUG = True. This lets you keep DEBUG = False on, but gives you some nice error pages.

Hope this tip is useful.

Pacman

“ The dog was created especially for children. He is the god of frolic. ”

Henry Ward Beecher

Ca Plane Pour Moi

Plastic Bertrand

In addition to metadata everywhere, I'm a big fan of testing1. In my day job working on Ellington, we've made big strides forward in our test coverage and I thought I'd share a couple tips I've learned from that experience as well as my own projects.

I suspect that as the month progresses, Eric will provide quite a bit of information of the unittest-style of testing, so I'll leave that to him and instead cover a few points on doctests.

In my opinion, neither unittest nor doctest are the better testing tool. Each shine in their own way and actually play very nicely together. For intensive testing of low-level functionality, it's hard to beat the organization and customizability that unittests bring to the table. But where doctests shine is in function/integration testing, where you can simulate how the lower-level bits will be used in practice. And in the context of Django, doctests evaluate much quicker, which is a big deal when you're testing a large suite or continuously testing in a TDD manner.

Using doctests do come with their own set of issues. In particular, when a failure occurs, it is sometimes difficult to track down where in the tests it occurred. Failures also prevent further execution, so an early error can cause many more to follow, incorrectly representing the number of failures in the test. Here are some ways to deal with these shortcomings.

1. Locating A Failure

A very common pattern is to make a request with the test client then check to see what the HTTP response code would have been. Unfortunately, this code has a habit of looking very similar over time. For example:

>>> from django.test import Client
>>> c = Client()

>>> r = c.get('/')
>>> r.status_code
200

>>> r = c.get('/blog/')
>>> r.status_code
200

>>> r = c.get('/blog/2008/')
>>> r.status_code
200

A failure on any of the status code checks will result in printing out only the line that failed (i.e. r.status_code). A common way I deal with this is to repeat the URL I'm requesting as a comment on the r.status_code line. So the example would become:

>>> from django.test import Client
>>> c = Client()

>>> r = c.get('/')
>>> r.status_code # /
200

>>> r = c.get('/blog/')
>>> r.status_code # /blog/
200

>>> r = c.get('/blog/2008/')
>>> r.status_code # /blog/2008/
200

Now, when the failure occurs, it's obvious (or more obvious) where it stems from.

2. Use Conditionals

Another common error that can crop up is when a failure occurs when processing a form. The normal pattern in the view would be to check if the form is valid, save then redirect. But an error will fall through, presenting the failures to the user. This will cause multiple failures in the doctest if the form's processing is incorrect. Example:

>>> from django.test import Client
>>> c = Client()

>>> r = c.get('/wall/add/')
>>> r.status_code # /wall/add/
200

>>> r = c.post('/wall/add/', {'name': 'Daniel', 'shout': ''})
>>> r.status_code # /wall/add/
302
>>> r['Location']
'http://testserver/wall/'

A failure in the form will cause both the r.status_code AND the r['Location'] lines to fail, when really there is only one failure causing the problem.

Rather than having to resort to testing this page in the browser, we can provide the programmer with more information to make debugging a failing test here go quickly. We'll conditionally check the r.status_code and supply conditional blocks that make sense based on it.

>>> from django.test import Client
>>> c = Client()

>>> r = c.get('/wall/add/')
>>> r.status_code # /wall/add/
200

>>> r = c.post('/wall/add/', {'name': 'Daniel', 'shout': ''})
>>> r.status_code # /wall/add/
302
>>> r['Location']
'http://testserver/wall/'
>>> if r.status_code != 302:
...     r['context'][-1]['form'].errors # Or r['context'][0]['form'].errors if you're not using template inheritance...

Now, if the form's processing fails (no redirect, so hence no 302), the tests will also output the errors from the form. This occurs because we've introduced a test that we know will fail (no output from the form's errors). This makes debugging the form much easier and faster.

As a warning, conditionals like this are slightly fragile, especially if there are other things in your view that could cause a failure. The point is more to introduce the idea of leveraging conditionals on a case by case basis. I also only use this technique when posting data, as that's when the more difficult errors seem to creep in.

3. Checking Context Variables

A great way to sanity-check what's happened in your views is to check the values that have been put in your context. Some people prefer to use tests or assertions here, like so:

>>> from django.test import Client
>>> c = Client()

>>> r = c.get('/acronyms/')
>>> r.status_code # /acronyms/
200
>>> len(r['context'][-1]['acronym_list']) == 5
True
>>> r['context'][-1]['acronym_list'] == ['Ajax', 'ORM', 'MVC', 'TDD', 'PEBKAC']
True
>>> isinstance(r['context'][-1]['form'], SearchForm)
True

Personally, I prefer to avoid boolean tests and instead output the context variable itself. So this would become:

>>> from django.test import Client
>>> c = Client()

>>> r = c.get('/acronyms/')
>>> r.status_code # /acronyms/
200
>>> len(r['context'][-1]['acronym_list'])
5
>>> r['context'][-1]['acronym_list']
['Ajax', 'ORM', 'MVC', 'TDD', 'PEBKAC']
>>> type(r['context'][-1]['form'])

While this doesn't represent a major difference in the amount of typing when creating the test, this saves a ton of time when running the tests, as the doctest runner will provide what it got instead of what you were expecting, further reducing the amount of time you spend debugging. If left as it was originally, you only know that the test failed (got False instead of True) and would have to dig in further yourself to find out what was actually returned and how.

In combination with fixtures, checking for correct output is a relatively simple, straightforward task.

4. Content Type Relations

One final failing point during testing (which can equally affect both unittests and doctests) is the use of content types when relating two models together. Because of the way things are handled as it stands in Django, the order of content types in testing can be different from that of development (or worse, from developer machine to developer machine). This can also happen when using generic relations (because it uses content types and foreign keys). Assume for this example that your User model has a relation to the Favorite model, and that using this, you recently favorited a Friend model.

>>> from django.test import Client
>>> c = Client()

>>> from django.core.management import call_command
>>> call_command('loaddata', 'myapp_testdata.yaml') #doctest: +ELLIPSIS
Installing yaml fixture 'myapp_testdata' ...
Installed 4 object(s) from 1 fixture(s)

>>> r = c.get('/favorites/')
>>> r.status_code # /favorites/
200
>>> r['context'][-1]['most_recent_favorite']

The way to handle this rather-sticky and sometimes sneaky problem is simple. At run time, simply load the correct content type and reprocess your fixture data, correcting the content type as you go.

>>> from django.test import Client
>>> c = Client()

>>> from django.core.management import call_command
>>> call_command('loaddata', 'myapp_testdata.yaml') #doctest: +ELLIPSIS
Installing yaml fixture 'myapp_testdata' ...
Installed 4 object(s) from 1 fixture(s)

# Fix the CTs.
>>> from django.contrib.contenttypes.models import ContentType
>>> from myapp.models import Favorite
>>> friend_ct = ContentType.objects.get(app_label='myapp', model='Friend')
>>> for fav in Favorite.objects.all():
...     # We thought the CT id was 3, but at run-time it is 5...
...     if fav.content_type_id == 3:
...         fav.content_type = friend_ct.id
...         fav.save()

>>> r = c.get('/favorites/')
>>> r.status_code # /favorites/
200
>>> r['context'][-1]['most_recent_favorite']

Now your tests will pass, regardless of what order apps/models were installed in on the machine running the tests.

Conclusion

Hopefully this gives you some ways to manage complex doctests and to speed up the debugging process when using doctests. For further reading, I highly recommend Django's doctest documentation as well as Python's doctest documentation. There's lots more that can be done with this flexible, simple tool.

---

1 - The name of my site/company/whatever is actually a play on Test Driven Development. I like it that much.

Another quickie tonight in favor of crunch time at work. In a specific use case of Ellington, we have a list of objects that is very expensive to generate and is requested very frequently. To help allievate this pain-point, I wrote a fairly straightforward subclass of the standard Django Paginator that caches that list of objects on a page by page basis.

Usage of the CachedPaginator is almost identical to that of the standard Paginator, with the exception of an extra required argument for the cache key and an optional argument for the cache timeout.

from django.core.paginator import InvalidPage
from django.http import Http404
from django.shortcuts import render_to_response
from myapp.models import ExpensiveModel
from myapp.paginator import CachedPaginator

def my_view(request):
    cache_key = 'wont_someone_please_think_of_the_servers'
    # Cache for 10 minutes by default unless we've specified a different amount.
    cache_timeout = getattr(settings, 'EXPENSIVE_MODEL_TIMEOUT', 600)
    
    expensive_object_list = ExpensiveModel.objects.run_crazy_query()
    
    paginator = CachedPaginator(expensive_object_list, 20, cache_key, cache_timeout)
    
    try:
        page = paginator.page(request.GET.get('page', 1))
        object_list = page.object_list
    except InvalidPage:
        raise Http404("Invalid page requested.")
    
    return render_to_response('awesome_template.html, {
        'paginator': paginator,
        'page': page,
        'object_list': object_list,
    })

The source can be found at http://www.djangosnippets.org/snippets/1173/. Matt Croydon (who happens to be a pretty awesome boss and extremely supportive of open source) gave me permission to release this code under the MIT license, so feel free to use it in your own projects as you see fit.

Check out the NYTimes article on WilsonDailyPrep. We at Imperial Consulting did the application, using Django and Python. It is a great service and a lot of fun to work on (my own vocab and grammar has gotten better, the Math quizzes are too easy). I am glad it is ...

There's a lot of options out there for search. You can defer to the big boys like Google, you can go with the enterprise-y solutions like lucene/solr and there's plenty more smaller options, like full-text search within your database engine or smaller engines like Sphinx.

But in the scale of the some of the small sites I've produced, Django has an often overlooked option, the Q object. The Q object allows you to perform more difficult queries without leaving the comfort of Django's ORM layer. It allows lets you add simple search to your site without have to configure and run a separate daemon.

Adding it is easy. We'll start with the following code:

import datetime
from django.db import models


class NewsPost(models.Model):
    title = models.CharField(max_length=255)
    slug = models.SlugField()
    content = models.TextField()
    posted_date = models.DateTimeField(default=datetime.datetime.now)

To add search, we'll create a custom Manager and add the functionality there. This makes the most sense, as it is consistent with other Manager API usage like filter or get.

import datetime
import operator
from django.db import models
from django.db.models import Q


class NewsPostManager(models.Manager):
    def search(self, search_terms):
        terms = [term.strip() for term in search_terms.split()]
        q_objects = []
        
        for term in terms:
            q_objects.append(Q(title__icontains=term))
            q_objects.append(Q(content__icontains=term))
        
        # Start with a bare QuerySet
        qs = self.get_query_set()
        
        # Use operator's or_ to string together all of your Q objects.
        return qs.filter(reduce(operator.or_, q_objects))


class NewsPost(models.Model):
    title = models.CharField(max_length=255)
    slug = models.SlugField()
    content = models.TextField()
    posted_date = models.DateTimeField(default=datetime.datetime.now)
    objects = NewsPostManager()

Now searching your NewsPosts can be done with a simple call like so:

results = NewsPost.objects.search("quick search")

There's tons of ways to make simple improvements to this (skip words, customizable attribute search, aggregating searches between models, etc.) but it makes for a simple way to provide basic search without hassle.

Found by Mr. Bennett, there was a bug in the CachedPaginator. Newly added is a number = self.validate_number(number) call in the page method. You'll want to grab the latest source from http://www.djangosnippets.org/snippets/1173/ if you used it.

Having been a professional Django developer for Mediaphormedia/The World Company the last 5 months has been a happy experience. The vast majority of my experience, especially the changes leading up to 1.0, has been very good. Most aspects of the framework are well thought out and flexible. And my experience with the code quality is that it is well written and (relatively speaking) fairly bug free.

However, because I spend more time with it than I did as a hobbyist before, I've started to notice things I wish were better or would like to improve. So since I'm in a crappy mood, I'm going to (as politely as I can) rant a little about things that haven't been as pleasant. I intend to do something about many of these things as more time permits toward the end of the year. Onward to the list.

1. Doctesting Is A Little Painful
2. Fixture Hell
3. Testing Suite Speed
4. Generic Views & Pagination
5. SQL Aggregation
6. Documentation

1. Doctesting Is A Little Painful

First off, I think doctesting can be pretty great. It's easy to write and easy to hand test. There's very little overhead in producing them. But there are pain points too.

The worst is the line numbers for failures. I have never seen them point to the correct line in the file where the error occurred, so you have hunt down where the failure occurred before you can debug it.

Additionally, loading fixtures in doctests sucks. The "official" way (stated via a ticket) is as follows:

"""
>>> from django.core.management import call_command
>>> call_command('loaddata', 'my_fixture_here.json') #doctest: +ELLIPSIS
Installing yaml fixture 'my_fixture_here' ...
Installed 4 object(s) from 1 fixture(s)
"""    

Ouch, ouch, ouch. Nevermind the fact that we're essentially simulating manually running the ./manage.py loaddata my_fixture_here, this command dumps output to the screen. And if your fixtures change (add/remove objects), you either use the doctest: ELLIPSIS comment or gain a fresh test failure because it loaded all the data you asked it to load (different fixture count).

2. Fixture Hell

Fixtures make me mad in other ways. For instance, let's say you write two applications, and like a good developer, include tests with fixtures in both. If both apps create a similar model that their data depends on (such as a User object) with the same primary key, when run together, one primary key will overwrite the data and potentially cause a test fail that doesn't occur when the app's tests are run on their own.

Additionally, you better hope you aren't using a GenericForeignKey or relying on a ForeignKey to a ContentType because at run-time, there's no guarantee that apps/models will be installed in the same order as your instance. In fact, this is frequently not the case, especially between different developers or different instances. The only solution to this I have come up with is to, as part of your tests (either in setUp or the top of your doctest), manually load up both the desired ContentType and your fixture objects that depend on them, reset their content_type to the correct id, then run the tests.

Finally, having to specify ForeignKeys by id (rather than by name like Rails¹) just kinda smells. I understand that it makes for an easier implementation but breaks unnecessarily in real life.

3. Testing Suite Speed

Here's a dirty little secret of testing Django apps. Doctests run way faster than django.test.TestCase (unittest) tests. The reason is that the TestCase-style tests truncate and reload all the fixtures in your database on every test method. So if you have a lot of fixtures and a lot of test methods, go brew some coffee/tea and sit for a spell.

4. Generic Views & Pagination

At the very least, date-based generic views seem to lack pagination. This is a borderline atrocity and leads to constantly wrapping these types of generic views. It's not hard but it's more work than it should be, especially if the list-detail generic views support it.

5. SQL Aggregation

This is more of a pony request than anything else, but I really can not wait for some sort of aggregation support to land in Django trunk. I know you can write manual queries (believe me, had to do a bunch of this already) and there are ways to do it now, but I'm waiting for an official API with some backward compatibility. I need GROUP BY and MAX/MIN/SUM/AVG in the worst way.

6. Documentation

Finally, I have a love/hate relationship with the new documentation. I love that there's search and deep links and, to an extent, that the pages are smaller. But finding what you want, especially if you're new to Django or even experienced with it, is ridiculously hard. It takes 4 clicks (minimum) to find the QuerySet reference, something that many of us use day in and day out. Additionally, using the search frequently sends you to the page you want with an "OLD DOCS OH NOES!" warning when it's actually the new documentation. Frustrating and probably kinda scary to a newbie.

The Bitter End

The best part about all of this, to me, is that everything on this list can be fixed/improved (and I intend to submit patches where I can). Also awesome is that these are my complaints, instead of lower-level details or more common cases. To me, Django is doing much better than other frameworks if this is the best I could come up with for a rant.

---

1 - HA! Snuck in the obligatory Rails reference!

This post is a reply to Eric Holscher's call for suggestions to extend and improve the documentation about testing in Django. By no means am I an expert in testing, but I thought I would share some thoughts I've had about this. It is more like a dump of ideas that come from my own experience, and hopefully that will contribute slightly to the debate.

The existing documentation is already pretty good but it is still a bit scarce. It is true that there is a gap in this area and that testing doesn't get the level of attention it deserves (I'm talking, broadly, in the Django community). Developers who regularly write tests already know how good testing is; but there should be more education provided to newcomers so that testing becomes a more common practice. And this would, in turn, benefit the community as a whole. By the way, Eric should be thanked for leading the way in this quest. If you haven't yet, you should definitely check out the great screencasts and tutorials he's posted on his blog about this topic.

Personally, I didn't know anything about testing until I discovered Django and started trying to write patches for it. I found that pretty much every patch had to include tests or it wouldn't stand a chance to be checked in by the core devs. I originally thought this was a bit of a stubborn ideology, but after diving into it I quickly realised how important and significant tests were. Put simply, Django would NOT be this good (in terms of features) and this reliable (in terms of robustness) if it didn't have that comprehensive test suite. I encourage anybody to have a look at it; this is one of the most pedagogic ways to understand Django's inner-workings and best practices.

Anyways, let's cut the crap. In the remaining of this post I'll talk about some misconceptions which I believe exist amongst the developers who are new to testing. Then I'll list some advantages I've noticed in writing tests for my Django apps. And finally I'll give some suggestions to improve the existing documentation, since that's what this is all about.

Misconceptions about testing

"Testing is hard, tedious and not much fun"

The word "test" sounds a bit like "spinach", doesn't it? When you were a kid, didn't your mom try to make you eat that disgusting green porridge saying that it was good for you? Well, when I first heard about testing that's exactly the feeling I had: everyone says it's good for you but no one really feels like giving the first bite. Maybe one of the reasons is that by writing tests you are very likely to find bugs in your code; and finding new bugs is never a good feeling. But, hey! Isn't it better to find bugs early on rather than when you least expect them?

So yes, testing requires discipline. But it is NOT hard! In fact, if you look at it closely, the code used in tests is usually damn straightforward. If you are scared by tests then you should realise that it is completely irrational. Simply jump into it! It won't take long before you recognise the benefits and it will give you the most rewarding feeling. There's even room for being creative when writing tests.

Believe it or not, but writing tests can get addictive. Oh, by the way, I now eat spinach and I like it :)

"Testing adds work load"

"What? I'm already swamped with development work and you want me write tests on top of that?!"

I think this would be a quite natural reaction from someone who's never tried testing. Well, let's put it this way: writing tests doesn't add work load, it just makes you reorganise the way you approach development.

You know, I'm going to tell you a secret: you already do "testing". Ok, this is not exactly a secret, but what I mean is that you must already be doing some kind of "human" testing: you open your browser, type in the URL, fill out some funky values in your forms (e.g. "test1", "dfdsfsdf", "blah"), click "Submit" and then check that it works of fails as expected. You're doing that, aren't you?

Writing "machine" tests will cut a lot of that work load. Even better, you can run those tests a zillion times each day for free, if you use a bot! Really, the machine should execute the tests, not you! I see the writing of tests more like a replacement (not complete, but at least partial) of the human tests that we already do on a daily basis.

"Testing takes time and slows development down"

I think there is a common belief that writing tests is time-consuming and that it slows you down in the development of your apps. In fact, if you look at the overall process, I contend that it actually makes you save time. And it does so at many different levels.

It's an investment: Save now for later

Here's a silly question: Do you want to find bugs and fix them now while you're building your app, or do you want to wait for your app to shamefully crash in front of the world? Obviously you don't want the world to know that your app contains bugs, and testing does help with that.

Like I said before, unless you're a total genius with a Python compiler implanted in your brain, it's very much likely that you will discover bugs in your code while writing tests. And that's in essence the whole beauty of testing. By killing bugs early on you can save a lot of the time which you would otherwise spend debugging when the bugs emerge days or weeks later. Also, for some reasons it seems to be much easier to think of all the edge cases when writing tests than when simply testing the app in the browser.

The other great thing about writing tests during development is that your mind is still very familiar with the code, so you can quickly identify the source of the bugs and fix them. Debugging weeks after means that you have to spend some extra time re-familiarising yourself with the code, before being able to do anything: W.A.S.T.E of time. Of course, writing tests as you go won't prevent some lurking bugs to appear once in a while, but if you are rigorous enough most bugs will be killed before they're even born.

Writing tests might seem a daunting task. But as development unfolds, it's just a matter of adding a few lines of code each time you add a feature or fix a bug. It is a capitalisation process and a solid investment of your time. It's like you keep putting money in the bank. During times of crisis like we're in, it's good to have at least something reliable! :)

Don't worry about the aesthetics

I don't know about you, but I often find myself spending 70% of the time fine-tuning small cosmetic details rather than writing actual code that does actual stuff. I can't help it, but I just hate developing something and testing it in the browser if it looks too wonky and is too naked visually. It just irritates me.

I'm pretty sure this happens to a lot of us. We can easily get distracted by the graphics and presentation of an app while we should be writing proper code instead. Testing really helps with that. When you're writing tests you naturally focus on getting the sh*t done. I believe this is because the purpose of testing is to make sure that your app WORKS well, not that it LOOKS good. You will do the graphic design work later, or even delegate it to someone else.

So, if you get in the habit of writing tests regularly, you will automatically cut a lot of time that you would otherwise spend in procrastination dealing with the aesthetical and superficial aspects of your app.

Don't worry about the browser

This follows directly on the previous point: if you don't get distracted by the aesthetics while writing tests it is simply because you don't have to use the browser. As I mentioned earlier, testing your app manually by clicking links, typing fake data, etc. can take a lot of time if you add all those tasks up.

Plus, as you gain experience and get better at writing tests you will also get faster at identifying and killing bugs. But could your browser be faster, or could you ever click and type faster? Probably not.

Obviously, you do need sometimes to check that your app works fine in the browser. But writing tests allows you to do that less often. As a challenge, I once tried to develop a simple, but big enough Django app (which processed some forms to add records in the database and also sent and processed confirmation emails), while writing tests in parallel and never using the browser at all. Then, once the core of the code was completed I tried it in the browser for the first time, and it just worked. All that was left to do was fine-tuning the templates and doing all the styling. If I can, I'll try to write another blog post explaining the process I've followed for that one.

Don't worry about database schema

Something that I think is just great is that the database is re-created each time you run the test suite. This is particularly useful when you start developing a new app, with new models. Indeed, during these early stages the models are likely to change a lot as you develop the app's specifications. If you do human/manual tests, it requires that you first create your database with syncdb, and then, if you want to make changes you have to do them manually in the database (unless you're using one of the emerging solutions for that problem, like django-evolution or South, but those are not quite fully functional yet).

If instead you write "machine" tests then you don't have to worry as much about experimenting with the database schema because the changes will be taken into account straight away the next time the suite is run. You don't need to do anything more than simply change your model declarations. Believe me, that alone saves a great deal of time and frustration.

Other advantages of testing

In the previous section I have already mentioned a few advantages of testing. That was to specifically respond to misconceptions some developers may have. Here I'll list a few other advantages that are important in my eyes.

First, as I already said before, testing allows you to anticipate bugs and kill them before they even exist. Another great benefit is that it prevents regression. Typically, regression means that something that used to work doesn't work anymore because of the insertion of some new code. In other words: you've broken your app while you were in fact trying to improve it. There is no worse feeling that breaking some good code on which you've spent time and sweat. If you have written tests that check your code is working properly, then those tests will fail and shout at you the next time you break your code, therefore preventing you from releasing a half-broken app. If you find a new bug, then fix it and write tests for it, so you're sure that you're done with it: that bloody bug won't come around again!

Second, writing tests forces you to have a critical look and to think effectively about the strengths and flaws of your app. After all, writing tests is about trying to break your app in any imaginable way. This forward-thinking process also greatly helps in the specification stage. If you're not enthusiast about drawing diagrams or drafting use cases, and you just want to dive into the code, then testing is for you. Testing makes you think about the big picture because it requires you to ask yourself questions like "What is this piece of code supposed to do and not do?" or "What would be appropriate and inappropriate input and output for this function?" Still, you're doing that thinking at the same time as you're programming, so you can satisfy you thirst for coding throughout the process :)

Finally, tests give you confidence. And confidence means both credibility towards your clients and peace of mind for yourself. For all the reasons enumerated above, you know that if you keep up with tests it is less and less likely your app will break. Priceless.

So, what should we do?

So, after all this wordiness, what is it we should do to entice more Django developers into writing tests and improving the (already awesome) Django's test framework? The existing documentation is already pretty good, but I believe there's a lack of practical examples with a good balance of best practices and purely technical considerations. Here are a few suggestions.

Strategies for testing

There should be some documentation explaining some good strategies in testing Django apps. It is out of question to give a full lecture on test-driven development, but at least some good hints and start points would be welcome. Here are some examples of strategies I think would be worth expanding on in the documentation:

• When you fix a bug, you want to fix the root cause, not the symptoms. Conversely, when you're testing you want to make sure the symptoms don't show up again. Functions, models, views, etc. should be treated as black boxes, and the tests should check that the inputs and outputs are correct, not how the job is actually done inside the black box.
• It would be good to give hints on the sequence in which things should be tested. Personally, I would advocate starting to test the small elements (e.g. custom form fields) before the bigger ones (e.g. the views). I would also recommend to tests all imaginable bad scenarios for a given element before moving to the next bigger element, so to ensure you build your app on solid grounds.
• There should be some pointers as to what is important to test in your app and what isn't. As I said, testing can be addictive and you can rapidly find yourself writing tons of tests that are useless. It's important to keep thinking: Is this test necessary? What should I really be testing? What are the priorities?
• There should be how-to guides explaining the best practices for testing a view, a middleware, a decorator, a template, a model (and a model field), a form (and a form field), a template tag, the sending of emails, etc. Obviously there is not a single way to test each of these, but providing a few alternatives would constitute a very good starting point.

Project-specific testing

The existing documentation explains quite well how to set up tests for individual apps, but there is very little information on how to set up project-specific tests. You might want, for example, to check that some specific data is systematically added to your database when the project is up and running. Again, I don't think there is one single way to do this, since the needs may vary from one project to another or from one environment to another. But it would be great if people could share their own tips about it, and maybe some best practices will emerge. I'm planning to write up another blog post soon to explain one particular setup I've made on a recent project. Stay tuned for more.

Tips and tricks

There should be a section in the documentation with some tips and tricks that make testing easier. Hey, I'll start with one! When I'm in development phase I need to run the test suite quite often, and that can take quite a while to process each time. So, what I usually do is use a SQLite database for testing, which is way faster than others I know. Then, once in a while I run the suite on the system that will be used in production (e.g. MySQL).

Conclusion

In conclusion I would say that, even if writing tests as you go might seem a bit time-consuming in the first place, it will save you a great deal of time and frustration in the long run. Django helps lever the foundations of your app quite quickly, and then testing should help consolidate them by closing the gaps and tightening the screws. You will then be waiting for the earthquake with a big smile on your face (sorry for the bad metaphor :) )

To finish, I recommend you to check out the slides from the Django master class that was given by Jeremy Dunck, Jacob Kaplan-Moss, and Simon Willison at OSCON 07. There are some excellent tips on unit testing and other areas of Django. Remember also that the best place to look for examples is Django's test suite itself. Dozens (hundreds!) of people have contributed to it over the years so you will find different styles of testing in it. You should also have a look at the tests included in the most popular third party apps, which should provide plenty of inspiration.

Phew... this post ended up being much longer than I anticipated. If you've read until this point, thank you :) I hope this contributes in some way to the debate, and I hope that at least this will help demystify testing for beginners. I'll post more on this topic, so stay tuned.

The Django team has once again announced a new release of their absolutely amazing framework. Right on schedule, version 1.0.1 has been made available to the world. This release packs over two hundred fixes over the original Django 1.0 codebase. I’m in the process of upgrading my…

Quan la gent dóna les seves primeres passes amb Django sovint queda enlluernada pel django-admin, una aplicació Django que ens permet definir un gestor de les nostres aplicacions, de manera que podem gestionar els usuaris, donar-hi permisos, gestionar les bases de dades, etc.

Hi ha que dir que l'aplicació està molt ben feta i es pot configurar moltíssim: indicar quins camps s'han de visualitzar, per quins camps es cercaran, fins i tot posar-hi javascript o modificar-ne l'aparença per a que la nostra aplicació faci el que nosaltres volem.

Aquesta facilitat però, té un perill, la gent té tendència a pensar que Django serveix per fer aplicacions web amb l'admin, que la seva aplicació ha de ser l'admin, així que recordem-ho:

El django-admin no és per fer aplicacions d'usuari final

Django admin fa molta màgia per dintre i aquesta màgia es basa en convencions que s'apliquen als models i a la definició del ModelAdmin, i aquestes convencions fan que per una part pugem tenir un gestor per a la nostra aplicació en quatre potades, però per altra, que si volem fer alguna cosa que surti del que està definit i establert ens durà molta feina.

La millor manera d'encarar-ho és tenir molt clar per a què serveix l'admin i per a què no:

• Manteniment de les nostres taules de configuració i mestres? sí. Normalment ens anirà fantàstic per això i a meś ho podem tenir molt ràpidament. És ideal en les primeres etapes del desenvolupament, ja que es pot donar a l'usuari per a que carregui dades de prova i ja veu com pot funcionar l'aplicació.

• Com a gestor web de la nostra base de dades? També anirà molt bé. Podem mapejar les taules i els camps que volem administrar, fer cerques, filtrar, editar i esborrar.

• Com a eina d'usuari final per a que pugui configurar l'aplicació? Sí, si aquesta sols implica operacions senzilles, per exemple l'edició d'una plana web, afegir registres a una taula, etc. Podem definir quines taules pot tocar cada usuari i definir-ne perfils.

• Com a eina d'usuari final on l'aplicació té molta lògica de negoci o bé un fluxe complex. NO. Millor començar un gestor pel nostre compte. Fer formularis CRUD amb Django és molt senzill i encara que pareix que dupliquem el que hi ha al django-admin, aquesta feina extra es veurà compensada a l'hora de definir els fluxs més complexos, ja que tindrem tota la potència de Python i Django al nostre abast i no estarem limitats a les convencions i funcionament del Django admin.

Les coses funcionen molt millor quan s'utilitzen per allò que estan pensades.

---

0 comentaris, 0 trackbacks (URL)

---

Following the previously-announced schedule, today the Django team has released Django 1.0.1. This is a bugfix-only release containing fixes and improvements to the Django 1.0 codebase, and is a recommended upgrade for anyone using or targeting Django 1.0.

For full details, check out the 1.0.1 release notes, and to grab a copy of Django 1.0.1, visit the downloads page. For the security-conscious, a file containing checksums of the 1.0.1 package, signed with the release manager's key, is available.

And with Thanksgiving coming up in the US, your friendly local release manager would like to pause for a moment and express thanks, on behalf of myself and the Django development team, for all the work put in by all the members of our community to help keep the releases coming, the tickets triaged and the bugs fixed. We wouldn't be able to do it without all of you, so give yourselves a big pat on the back and see if you can't sneak an extra slice of pie come Thanksgiving dinner.

We'll see you again in a few months, for either Django 1.0.2 or Django 1.1. Happy holidays!