Optimizing Django: Some Quick Lessons
The Web Dev Zone is brought to you in partnership with Mendix. Discover how IT departments looking for ways to keep up with demand for business apps has caused a new breed of developers to surface - the Rapid Application Developer.
While working on a recent project, Luke Plant decided to try django-fiber, a lightweight CMS, because of its improved frontend editing and the ease of sharing content between pages over django-cms. But he noticed that it was requiring large numbers of queries, some pulling back tons of data, just to render fairly simple pages so he decided to create a "query count reduction branch" and was kind enough to share his results and some further thoughts.
If you're not familiar with Django-fiber, check out this video:
Now, let's look at Luke's results:
I used the example django-fiber project for testing my changes (as well as my own project), and tried them out on both the home page and a more deeply nested page.
-- Luke Plant
What he found was that the home page queries for an anonymous user dropped from 30 to 15, and from 103 to 28 for a staff user. For the deeply nested page, he found that queries for an anonymous user dropped from 64 to 16 and from 150 to 31 for a staff user. So by using his query count reduction branch, Luke was able to reduce queries for anonymous users by an average factor of 3 and by an average factor of 4 for staff users.
Luke is quick to point out that a lot of unnecessary work was done just to reach this point in his evaluation, and offers some advice for those working on similar projects:
- Understand how to optimize DB access for Django.
- Avoid queries that will duplicate information you already have.
- Ensure efficient DB filtering when searching for special values.
- Use the django-debug-toolbar from the get go and make sure to check it regularly.
- Consider big O scaling issues early in development to avoid future problems with schema design and third-party compatibility.
- Use tests created with assertNumQueries so you will know when performance regressions are affecting scaling.
So if you're working with Django-Fiber and find it's running too slow, try Luke's query count reduction branch method and be sure to follow his advice.