Categories: Portfolio, Software Development

Speaking at PyCon Canada 2017 on “Python as a Philosophy”

I’m going to be in Montreal this weekend for PyCon Canada to give a talk on “Python as a Philosophy”. It will explore some of the ideas that are part of Python, built into its core and affect its community and how those ideas have spread to other software and software developers:

A number of programming languages that have not been used quite as frequently as others contain “profound enlightenment” that a developer learns through their use. Languages such as Lisp and Smalltalk are in this category.

This presentation suggests that Python is also one of those languages. Python and PEP20 The Zen Of Python can affect how a developer writes code and then translates their practice into using other programming languages.

And more:

For example, my own hands-on programming in Python has allowed me to carry on a number of practices into JavaScript and Ruby. The talk will specifically cover “explicit rather than implicit” code and the value of documentation as a developer-user experience.

I will outline several examples and counter-examples of Python-inspired code practices within the context of JavaScript and Ruby.

Slides for the presentation:

Categories: Software Development

Django Unit Testing with Mocks

Here I’m going to talk about how to use mocks when writing unit tests for Django, the Python-based web framework. Using Django test mocks has really opened my eyes on how to write much better unit tests. Previously, and in some cases still do when using 3rd party services, I would use fake API servers to serve fake data for testing end to end. With the mock library, I can easily mock out server responses in Django tests.
Continue reading “Django Unit Testing with Mocks”

Disable CSRF Verification in Django Class-Based View

While working on a Django web app, I had to disable CSRF protection in a view. The goal was that our API was accepting data from another part of our website and we needed to skip the CSRF protection just for that one page.

There are two great answers on how to do this on StackOverflow:

  1. How to disable Django’s CSRF validation?
  2. How can I disable Django’s CSRF protection only in certain cases?

And nice documentation for Django Rest Framework on disabling CSRF for AJAX calls.

The answer is basically to use the csrf_exempt as a function that wraps the class-based view’s .as_view call and to do this in the urls.py routing file.

For example,

HelloView.as_view()

will become

csrf_exempt(HelloView.as_view())
Categories: Portfolio

Hotspots Tool

I created a tool on the Django admin for positioning hotspots on an image. Hotspots on an image are points that can be clicked which will then display a pop-up with more information. I created this because we had maybe 50+ hotspots to place. Without the tool it would be a manual job, figuring out exactly where to place them. With the tool, all we had to do was drag and drop the hotspots on the image.

I wasn’t involved with the front-end except for a few minor CSS fixes. I was only involved in working on the Django admin tool for placing the hotspots and assigning the extra information they needed to display.

Unfortunately, I didn’t take a screenshot of the tool (and I probably wouldn’t be allowed to share it) so I created a mockup instead. I also started work on a free/open source implementation, it’s called jquery-hotspot-placement.

A mockup of the implementation of a hotspots placement tool I created for the Django admin.
A mockup of the implementation of a hotspots placement tool I created for the Django admin.
Categories: Linux, Software Development

First code patch submission in a long time…

UPDATE: It got in, the patch was reviewed, updated and added to the main code base. The developer even added my name to the credits!

I submitted an issue to the Django Rest Framework project. There was a bit of a problem with the documentation’s example not reflecting the code. The oft-repeated “the docs are never updated with the code” line that other programmers like so much ran through my head.

I think the way I presented the issue was solid…providing enough detail about what I was trying to do, the code that failed, the error message that was trigged, the version of the framework I’m using, and pointing the area of the code base that potentially caused my issue: https://github.com/tomchristie/django-rest-framework/issues/955

To remedy this situation, I forked the repository, created a new branch, made the fix, and created a pull request: https://github.com/tomchristie/django-rest-framework/pull/956

Documentation is important, especially when a project has over 1000 users. Updating it should be seen the same way as fixing code or adding new features, it shouldn’t be seen as a chore. As I quoted in a previous post, “some text is better than no text”.

The author of Django Rest Framework, Tom Christie, was super quick to respond. I got a response to the issue within a few hours. The frequency of releases isn’t too bad either, it feels regular and I would definitely continue using the project over tastypie.

I haven’t submitted a patch to any project in a long time, I’ve only reported a few issues and it will be awesome if this patch is accepted.

Categories: Portfolio, Software Development

Presentation: Why PyCharm

Just created a short presentation for work about why I’m checking out the PyCharm IDE.

I wrote an article about trying out PyCharm for Python development. I learned to use it when working with Django (though I have used Emacs for web development usually). PyCharm has a lot of advantages and what I like about it, and RubyMine, is that you can use Ctrl+Click (on GNU/Linux and Windows) or Command+Click (on Mac OS X) to see where a method or class is defined or to find where it is being used in the code base.

Update: found a nice post by a VIM user who switched to PyCharm and found it awesome how integrated everything is.

Categories: Web Dev Training

Django: dumping and loading databases

At work , we have this situation where we want to dump the database from the production servers so that we can work with that data on our local development machines. As part of an internal app (which another developer is working on open-sourcing parts of, UPDATE: the app is on github, however I no longer use it myself, not sure if it’s still used internally at Trapeze), we have two management commands to help us with this: load_devdata and dump_devdata.

The reason we use a management command is because we don’t install Fabric on our production servers to run these commands, we don’t use shell scripts either.

The database commands are PostgreSQL-specific and we use rsync to copy the media files.
Continue reading “Django: dumping and loading databases”

Django filter versus get for single object?

Django get versus filter for single object?

While working on a project, we use a particular pattern in Django for getting a single object. We first filter the queryset by the id or slug and then check if the queryset exists. If so, then we return the object otherwise we return none.

Here is an example of how this looks in Python:

models = MyModel.objects.filter(id=24)
if len(models) > 0:
  return models[0]
else:
  return 'could not find model'

Which brings us to the question of get versus filter, which one should we use? Should we lean towards checking the number of objects from the queryset filtering or rely on the get and handle/catch any exception thrown when no object is found?

According to James Bennet, Django release manager and author of django-registration, this isn’t necessary,

get() is provided specifically for this case. Use it.

Option 2 is almost precisely how the get() method is actually implemented in Django, so there should be no “performance” difference (and the fact that you’re thinking about it indicates you’re violating one of the cardinal rules of programming, namely trying to optimize code before it’s even been written and profiled — until you have the code and can run it, you don’t know how it will perform, and trying to optimize before then is a path of pain).

The reason we avoid it is because the get() method requires that we use a try/except block and I rarely see exceptions being caught in our code. Jeff Knupp suggests that writing Python with exception handling leads to cleaner code.

Let’s take a look at that example. If we use the get() method to retrieve a model instance and use exceptions to catch the case where the object could not be found

try:
  model = MyModel.get(id=24)
  return model
except MyModel.DoesNotExist:
  raise

You’ll notice how we’re using the bare “raise” statement in the example in line 5. This will let you propagate the traceback and original exception up the chain. This means you do not have to invent new types of exceptions.

In his article, Jeff Knupp walks through the performance characteristics of using exceptions and finds that the trade-off in speed is acceptable, something on the order of thousandths of a millisecond. So there’s no way to use speed as an excuse to avoid exceptions!

Start using exceptions in Python and use the get() function of querysets whenever you want to find exactly one model instance.

UPDATE: Some good books on Django design patterns and coding style:

Categories: Software Development

Trying out PyCharm python IDE

I got a 30-day free trial copy of pycharm, and it’s pretty amazing so far. I’m usually an emacs user but I’ve struggled with making things easier in terms of testing and deployment.

PyCharm IDE for Python

At work I’m running emacs and a console with multiple tabs just to be able to develop django apps. When i was coding common lisp, scheme and even C/C++ i could stick to a single emacs session.

I’m also getting tired of having to write custom functions to do what I want. To be honest, I won’t ever get tired of customizing Emacs.

One neat feature is PyCharm markdown mode which displays a preview of Markdown. Another feature which I love is the refactoring.

How much does it cost?

I’m not sure if pycharm is free/open source (update 2017, Intellij IDEA has a community edition and the pricing model has changed) but the cost isn’t too bad: $100 for a personal license, free for free/open source software developers, $200 per developer at a company. If it’s good enough I’ll ask for a company license. Other developers at work are using Vim, Sublime 2 and some other text editors.

Deploying to Local & Remote Virtual Machines

The problematic part for me is that I had a remote virtual machine that ran Ubuntu Server to replicate the environments we deploy to. This requires mapping a local folder to the remote folder in PyCharm. Running commands remotely is also a pain with that setup (or maybe the commands I was running were broken?)

So I’m resigning myself to developing locally on my Ubuntu machine. I do not need a virtual machine because my machine is as close to the deployment environment as possible already. All other developers are using mac os x and there are just too many differences between gnu/Linux and os x; it makes sense for them to run things in a virtual machine.

Even if I don’t switch to pycharm, my time with emacs will be more fun. Constantly transferring files back and forth over ssh/sftp is slow and tedious.

Categories: Software Development

End to end tests posing as unit tests

I’ve been reading the excellent book The Art of Agile Development, I started to think more about why Test Driven Development is important and how it affects how you write code.

Using some of the 20%/practice group time we have at work, I started to review my own code to see if I’m living up to the test-driven development methodology. Are there enough unit tests? Yes, there were, code coverage was over 70% in most cases and with a little work I got it to 100% coverage for the most important code (the views and models). Then I started to noticed something about my tests…they’re not unit tests at all! They’re end to end tests! What Django calls unit testing is actually an end to end test. We’re testing from one end, the website, to the other end, the database. This means that we’re testing each view to ensure that it includes the right data, that it uses the right templates, etc. Then we’re testing interaction and checking the results in the database. Continue reading “End to end tests posing as unit tests”