We were talking about things that we wish we had known before while developing for Django the other day in IRC. I proclaimed that we should write them down somewhere. So I'm writing a post to get this effort started. Please feel free to leave comments with your own tips and tricks, and I'll compile them in some kind of good fashion. These are mostly just pointers, and not full-blown writeups, just more of a big list of stuff you should think about.

App level

Local library installation

When you don't have root access on a machine, and you want to use easy_install, you can install files into a designated directory. I use ~/lib to hold my python modules, so when I do easy_install I simply use the -d option like so:

easy_install -d ~/lib nose

Use iPython

IPython is a python distribution that gives you lots of handy features like tab completion, syntax highlighting, better debugging, and lots of other nice features. Poke around, I find more and more stuff I like every time.

Use django command extensions

This is one project I use every time I do some new code. I wrote up a whole post and screencast about how good they are. They have gotten even better since then, highly recommended.

Performance tips from the man himself

Jacob (co-BDFL) wrote up some performance tips back in '05 that are still relevant today. Some big architectural stuff, but they still make sense.

Use mod_wsgi

mod_wsgi is currently the best way that I know of to run Django. I did a simple write up a while back that has proved popular.

Running out of memory?

Web faction has a good blog entry about how to keep memory usage down. Might be useful even if you're not running on their hardware.

Use pdb

Pdb is a great debugger. Simon has a great post on it, and I took a lot of his ideas and expanded them to do my debugging django screencast series. Using pdb

Read b-list.org archives

My tips here are short and sweet, James has a wealth of amazingly informative Django information stowed away in his blog archives. Do yourself a favor and peek through it and be enlightened.

Don't be afraid of Reusable apps

Watch James' presentation on Reusable apps at Djangocon. Learn it, and use it. A lot of the functionality that you want to do has already been done for you. Check out Pinax which has a ton of nice reusable apps. django-basic-apps also has a ton of really nice reusable apps that use best practices. I use the blog here and it's a great way to learn how to use django well. Learn by other people's awesome examples!

Watch the Djangocon videos

The videos from Djangocon give you some great insights into Django.

Search and replace

Search and replace across an entire directory. This is useful for changing template vars or doing basic refactoring (good editors should do this for you too)

perl -pi -w -e 's/foo/bar/g' *.html

Check out virtualenv

virtualenv is an awesome python tool that allows you to create mini-sandboxes of python. You can contain an entire django install (and supposedly you can get mod_wsgi and some other stuff inside). I haven't played with it too much, but it sounds really nice to keep a contained python environment, and allows you to run different versions of libraries, django, and anything else you can think of.

Models

Use managers for commonly accessed queries

Writing managers is really simple, and they provide a better user interface to your code. This code snippet simply adds a latest() method to the default objects manager

class ForecastDayManager(Manager): def __init__(self, *args, **kwargs): super(ForecastDayManager, self).__init__(*args, **kwargs) def latest(self): return self.get_query_set().order_by('forecast_date')[0]

It can be called ForecastDay.objects.latest(). This is a trivial example, but there is a lot of power that lies in this functionality.

Meta is your friend

You can define the default ordering of your model, so when it returns things in a queryset you don't need to do an order_by() clause (like above). Possible settings. If you set get_latest_by, the above code is already written for you.

No really, Love meta.

Ever wonder where all of that lovely metadata that you have set goes? It all gets stored in your objects _meta variable. Note the underscore, this is private and might change at some future point. However, a lot of it is stable and it gives you some really nice things that you can get access to. defaultmanager is a another really nice one on query sets, this returns objects (or whatever the default manager is). It's really handy for writing re-usable code.

Settings

Relative imports

When you are using a setting file multiple times, it is nice to be able to define relative variables for your things. import os DIRNAME = os.path.dirname(__file__) DATABASE_NAME = "%s/dev.db" % DIRNAME MEDIA_URL = os.path.join(DIRNAME, 'media') TEMPLATE_DIRS = ( DIRNAME + "templates", )

more

Local settings

If you have local changes to your settings file, that you don't want to share, or that are specific to your box, there is an easy way to accomplish that. Put this at the bottom of your settings.py file:

try: from local_settings import * except ImportError: pass

This allows you to define a local_settings.py in that same directory (or on your pythonpath if you so feel). This can then override (or add on to) the settings previously defined in the file.

Use a settings debug file.

This kind of inverts the logic above, but runserver allows you to pass it a settings command. So you can run runserver with the command ./manage.py runserver --settings settings_debug and I keep a settings_debug.py file sitting around that looks like this:

DEBUG = True INTERNAL_IPS = ['24.xxx.xxx.xx'] MIDDLEWARE_CLASSES += ('debug_toolbar.middleware.DebugToolbarMiddleware',) INSTALLED_APPS += ('debug_toolbar',)

This allows me to keep my normal (production) settings file from ever having DEBUG set to True. That way there's no way to run with it in production. The other things are just good easy way to maintain some stuff that is useful for debugging/testing, but you don't want to include in your normal production server.

Views

Wrapping generic views

It's really easy to use generic views in Django. Sometimes you want to change a little functionality or what they return, so you think you have to write a whole new function. Malcolm goes into how to extend them, to save you some time.

Use RequestContext

By default, when you render a template, you aren't given the request object. It's nice to have and really simple to make django give it to you.

from django.template import RequestContext def index(request): return render_to_response('weather/index.html', {}, context_instance=RequestContext(request))

Templates

Use the {% url %} tag.

Using the url tag allows you to make your templates portable and is a good way to implement DRY. Whenever the links in your view changes, your templates automatically update, and they always have the correct links.

Use Template Utils

django-template-utils contain some really nice generic template tags and other goodies that make your life easier. From getting the latest X number of objects from a model, getting a random object from a module, or getting the last updated one; they provide you with a really nice generic way of extending template nodes and doing generic content tags really easy.

Use MEDIA_URL

Django now comes with a Context Processor that gives you MEDIA_URL in your templates. Use this so that you can apply DRY to all of your external media Urls, like you did with the {% url %} tag for internal things.

Use a 3-level template hierarchy

This is referenced in the Django docs(about a page down). But it works really well to do a base.html, app-base.html, and then templates on top of that. This gives you a really nice way to contain site-wide, app-wide, and view-wide functionality inside their own little spaces.

Testing

Watch Files

This tip is useful for doing TDD. You can go ahead and watch the output of your test file and see when something changes based on the edits you're making to your files.

watch "python tests.py"

Nose tests

Use nose tests. They have some neat auto-discovery tools and lots more. nose-django allows this to work with Django fixtures (note it may not work well yet). This would be nice if someone wrote a test runner in django for nose.

Mock objects

Using mock objects to test is really handy. There are a couple of good mock testing libraries for python, and i show a simple way to do it here This allows you to try your code when it's interacting with things that are somewhat random (like times of day, random numbers, etc.)

Use testmaker

I wrote an app that allows you to write view tests for you. A little self promotion, but go ahead and check it out.

Want to do something a little different?

You can define your own test runner and set it in the settings. Then you can tweak the way that django runs your tests for you. This is a lighter weight approach than using nose or something to run your tests, and is integrated with django, which makes it more portable.

Use testserver

Django comes with the testserver command that allows you to load a fixture into the development server and run against that. This is really useful. It also leaves the database around after it's done, so you can inspect it. This can be really handy in debugging fixtures and tests.

The lynchpin of Ruby’s pidgins and so-called DSLs (Douchebaggery as a Second Language) is the method known as instance_eval.

From an article titled Ruby DSL Blocks:

 def self.order(&block)
   order = Order.new
   order.instance_eval(&block)
   return order.drinks
 end

And another one Implementing an internal DSL in Ruby:

 def Expectations(&block)
   Expectations::Suite.instance.instance_eval(&block)
 end

From Creating DSLs with Ruby:

 class MyDSL
   def define_parameters
     yield self
   end

   def self.load(filename)
     dsl = new
     dsl.instance_eval(File.read(filename), filename)
     dsl
   end
 end#class MyDSL

So far, so good? Most often instance_eval is used, but you’ll see module_eval, too.

Now, the reason for this.

Jim Weirich: Within the builder code blocks, any method call with an implicit object target needs to be sent to our builder. To achieve this, the code blocks are evaluated with instance_eval which changes the value of self to be the builder.

He goes on to say why this could be troubling.

This is OK until you want to call a method in the calling object. Since self is no longer the calling object, you have to explicitly provide the caller.

In Weirich’s Builder, he prefers to use plain blocks and hand you the variable.

 builder { |xm|
   xm.em("emphasized")
   xm.em { xm.b("emp & bold") }
   xm.a("A Link", "href" => "http://onestepback.org")
   xm.div { xm.br }
   xm.target("name" => "compile", "option" => "fast")
   xm.instruct!
   xm.html {
     xm.head {
       xm.title("History")
     }
     xm.body {
       xm.comment! "HI"
       xm.h1("Header")
       xm.p("paragraph")
     }
   }
 }

With instance_eval, you’d end up with:

 builder {
   em("emphasized")
   em { b("emp & bold") }
   a("A Link", "href" => "http://onestepback.org")
   div { br }
   target("name" => "compile", "option" => "fast")
   instruct!
   html {
     head {
       title("History")
     }
     body {
       comment! "HI"
       h1("Header")
       p("paragraph")
     }
   }
 }

Builder once did use instance_eval, but now offers this explanation in the docs:

The instance_eval implementation which forces self to refer to the message receiver as self is now obsolete. We now use normal block calls to execute the markup block. This means that all markup methods must now be explicitly send to the xml builder.

Rails’ routing stuff also prefers to go without instance_eval:

 ActionController::Routing::Routes.draw do |map|
   map.with_options :controller => 'blog' do |blog|
     blog.show '',  :action => 'list'
   end
   map.connect ':controller/:action/:view'
 end

By now, it probably seems like instance_eval has been driven out of play by the mature libs and now only lingers in the dabbling blog posts.

I don’t care about the best way to do this. The fact is: even “normal blocks” are prone to bugs. I had to fix up the Builder example above because it’s wrong in the docs.

 xm.em("emphasized")             # => emphasized
 xm.em { xmm.b("emp & bold") }   # => emph & bold
 xm.a("A Link", "href"=>"http://onestepback.org")
                                 # => A Link
 xm.div { br }                    # => 

There are two errors in the above example. The date on the RDoc is Sun Feb 05 23:49:01 EST 2006.

Recently I discovered another option to all of this, thanks to the work of Guy Decoux.

While investigating a library Guy never released (called prop,) I realized that perhaps he was on to something while fooling with mixins and the inheritance chain.

Think about it: instance_eval changes self, intercepts method calls and alters instance variables. Really, all we really want to do is dispatch method calls. As Jim Weirich says, changing self is the troubling side effect here.

This is a pertinent topic. Even today, ruby-core discusses a with operator and Paul Brannan chimes in:

Instance_eval for initialization has surprising behavior for instance variables (e.g. as in Ruby/Tk).

A method that affects only method calls and not instance variables would make this idiom more viable.

I don’t know whether that is a good thing or a bad thing.

Again with the good and bad.

What if there was a way to temporarily add methods for the duration of a block?

 def to_html
    Builder.capture do
      html do
       head do
         title self.friendly_title
       end
       body do
         comment! "HI"
         h1("Header")
         p("paragraph")
       end
     end
   end
 end

The essence of Builder.capture is to mixin a bunch of Builder methods, inject them into the block’s binding. This adds the html, head, comment! methods into the calling self.

Using a very small extension called mixico:

 def Builder.capture &blk
   mix_eval(self, &blk)
 end

This extension enables and disables mixins atomically. It is a single, quick operation to add and remove a module from the inheritance chain. (See the mixico README for more on this technique.)

The mix_eval method code looks like this:

 class Module
   def mix_eval mod, &blk
     blk.mixin mod
     blk.call
     blk.mixout mod
   end
 end

The mixin and mixout methods enable and disable the Module in the block’s binding.

While you might be inclined to dismiss this on the grounds of it being mixin magic, I’m starting to believe that this is a notable omission from Ruby. It’s very quick and efficient to disable and enable mixins and could prove to be a very handy technique.

Like open classes, however, I’m afraid the timidity of the business community might label it as taboo, despite it offering great flexibility to you — all of my fine, able-minded friends out there.

Как вы догадались, я продолжаю тему денормализации и моей реализации композитных полей.

С момента прошлого поста я успел значительно улучшить базовый CompositionField и решить несколько концептуальных проблем.

Итак что же новое появилось:

• Сделал низкоуровневый CF полноценным классом, который теперь удобно сабклассить и добавлять новый функционал
• Появилось место для интроспекции. Присоединение к хост модели стало более интеллектуальным из-за возможного отложенного присоединения.
• Чуть-чуть изменился интерфейс - параметр commit теперь может быть задан для конкретного триггера. Это сугубо практическое изменение, которое помогло решить одну проблему с бесконечной рекурсией.
• По совету Вани Сагалаева добавил генерацию freeze_FOO метода, который включает/выключает обработку сигналов. Полезно когда надо обработать какие-то данные скопом, а потом так же скопом пересчитать денормализованные поля.

Я грозился написать высокоуровневые обертки над CF чтобы облегчить использование и упростить конечный интерфейс. Сегодня я расскажу о первом сабклассе CF: ForeignAttributeField - поле которое отслеживает изменения некого внешнего поля в связанном объекте. Причем, поле может находится на любом уровне вложенности связи, что иногда очень удобно. Как я уже писал в прошлый раз - поля объектов, на которые непосредственно ссылается модель, так денормализовывать смыла мало. Проще использовать select_related. Но если заветный атрибут лежит глубже и для доступа к нему надо приджоинить, допустим, пять таблиц, то тут уже другой расклад и это может оказаться вполне хорошим юзкейсом. Кстати, это поле очень похоже на оригинальный DenormField, предложенный Эндрю Годвином. Я конечно не буду тут показывать пять уровней связей, а покажу лишь две, но формально их может быть сколь угодно много. Так же и композитных полей в модели может быть не ограниченное количество.

Пример с фильмом и режиссером из предыдущего поста перепишу по новому и добавлю ещё одно поле:

class Country(models.Model):
   name = models.CharField(max_length=250)

class Person(models.Model):
   name = models.CharField(max_length=250)
   country = models.ForeignKey(Country)

class Movie(models.Model):
   title = models.CharField(max_length=250)
   director = models.ForeignKey(Person)

   director_name = ForeignAttributeField("director.name")
   director_country = ForeignAttributeField("director.country.name")

Теперь посмотрим как это работает:

>>> country = Country.objects.create(name="USA")
>>> person = Person.objects.create(name="Steven Spielberg", country=country)
>>> movie = Movie.objects.create(title="ET", director=person)

>>> movie.director_name
'Steven Spielberg'

>>> movie.director_country
'USA'

>>> person.name = "Steven Allan Spielberg"
>>> person.save()

>>> movie = Movie.objects.get(pk=movie.id)
>>> movie.director_name
u'Steven Allan Spielberg'

>>> country.name = "United States"
>>> country.save()

>>> movie = Movie.objects.get(pk=movie.id)
>>> movie.director_country
u'United States'

Как видите всё довольно прозрачно и естественно. За это я и боролся. Надо только отметить, что в данном примере я явно каждый раз после изменения связных моделей тяну из базы объект movie, чтобы в нем были уже обновленные данные.

Кстати о борьбе. Скажу я вам, это потрясающий фан - придумывать и реализовывать подобные вещи. Сколько же нового понимаешь о структуре фреймворка. Словно как какой-то путешественик, пробираешься сквозь джунгли, прорубаешь себе дорогу и находишь выходы в непростых ситуациях. Очень интересно, занимательно и позновательно.

Поразило также насколько универсальным и обобщенным оказался CF сам по себе. Поскольку его функциональную часть не пришлось менять практически совсем за исключением выноса параметра commit в объявление триггера. Это связано с тем, что далеко не после всех вызовов обработчиков сигналов можно звать save объекта. Например его нельзя вызывать после pre_save/post_save сигналов того же объекта что и был обновлен, а то получается - бесконечная рекурсия.

Ещё - вся функциональность этого высокоуровнего поля заключается в правильном составлении тригеров и обработчиков событий (на базе интроспекции моделей), которые передаются в "отложенный" конструктор CF. Ничего более. Супер!

Файл с исходниками доступен как и раньше тут. Если вы его посмотрите, то заметите заготовки для ещё двух типов полей: ChildsAggregationField, AttributesAggregationField - но они пока на уровне идеи. Как дойдут руки, то обязательно их реализую. Некие намеки на их использования можно посмотреть в тестах.

Ах да, все примеры использования которые я приводил и буду - либо уже являются частью тест набора для CF либо будут туда вписаны. Сами тесты доступны здесь. Они тоже пока ещё не устаканились, не покрывают максимум кода и возможно подвергнуться какой-то реорганизации. Но как источник примеров годятся уже сейчас.

А вам как? Нравится?

In this episode we update our events app to take advantage of the social networking features that we have built, using some more advanced QuerySet techniques (including one tip by reader janr). I also discover a bug on the live site, and give a preview of how deployment will work.

following = [f.to_user for f in UserLink.objects.filter(from_user=user).values('to_user')]
users = User.objects.filter(id__in=following)

following = UserLink.objects.filter(from_user=user).values_list('to_user', flat=True)
users = User.objects.filter(id__in=following)

following = UserLink.objects.filter(from_user=user).values('to_user')
users = User.objects.filter(id__in=following.query)