Nov 21, 2011

Django: Instalation of Dev environment on a clear Lion Mac OS X

Like lots of Apple fans around the world I've finally decided to upgrade my OS to Lion release. I have bought a new laptop and it was the case. I've waited for a weekend to prepared myself for sex. And it was it. Now I want to share my experience with you guys. For you not to spend to much time on those issues, I've did.

Anyway what I'm usually using is a Fresh 2.7 Python, Django, PIL, virtualenv and many many minor tools which exceed scopes of this article. I also use Eclipse IDE for Python Dev. So all this I had to install and make spinning till monday.

If anybody tells me that Macs do not have good tools for this. He will not be right. Maybe there some itches and scratches of head about:
"Why is not those things working" or "Why do they work not as preferred". But there always is a good "Googled" decision to handle that.
Anyway lets go to installing stuff.

Short brief Plan of out work:
1. Install Apple Development SDK + Xcode (free from Apple App Store).
2. Install Python (from
3. Install Libraries needed for PIL (Libjpeg and Freetype support).
4. Install Django.

1. First of all! Mother of all Gods! We need Xcode and Mac Development tools provided from AppStore for Free. Let's download it and install it. If you have an apple store Account, of course. Actually here we do not need the whole bunch of stuff like Xcode or Dev tools except for GCC. And if you're advanced enough and have time to handle it... You should probably just install GCC, compiling it from sources for e.g. I'm not gonna spend time at those "low levels" and will simply install a set of packages that I may (or may not) need in future for dev.

After Xcode is Downloaded you should go to Applications of Lunchpad and hit Xcode icon. (They are actually only downloaded not installed yet. What a long time period,  waiting for 1,6 Gigs App download form App Store.) Hit 'next', 'yes' and so on... Agree to everything and probably close some apps like for e.g. iTunes somewhat weirdly but true. Anyway lets continue to our next and main part. Those actions should already give us required GCC compiler for all our needs.

2. Next step is installing Python. We should get it from here, for e.g.: I've chosen Python 2.7.2 Mac OS X 64-bit/32-bit x86-64/i386 Installer (for Mac OS X 10.6 and 10.7). It gave latest stable 2.7.2 Python with idle for about 17 Mb more downloading.
DMG file has a simple enough installer that will ask you for root password and do all the stuff you need for you...

3. Support of jpeg, freetype and littlecms is missing, by default. You need to compile PIL with support of at least jpeg and freetype. In order to make ImageField in your django project support jpeg formats and Have ability to handle Captcha fields or different text with picture manipulations.
You may need to switch to 'sh' interpreter from standart 'bash' typing sh in command line. It handles ./congure and make install commends more gracefully (IMHO). And you may need 'sudo' to gain permissions to some system stuff here.
Will try to be brief here:
  • Install JPEG support libraries:
    • Download JPEG library source curl -O
    • Extract tar -xvzf jpegsrc.v8c.tar.gz
    • Switch to jpeg directory cd jpeg-8c
    • Run ./confgure (I only made it work at 'sh' command prompt someway)
    • Run make
    • Run sudo make install
  •  Install Freetype support:
    • Download Freetype library source curl -O
    • Extract tar -xvzf freetype-2.4.5.tar.gz
    • Enter to freetype directory cd freetype-2.4.5
    • Run ./confgure (I only made it work at 'sh' command prompt someway)
    • Run make
    • Run sudo make install
  • Actually install PIL:
    • Type: sudo easy_install PIL
    • End!
Now PIL Should compile with support of FreeType Fonts and LibJpeg. I also used easy_install pip first. Because I simply like PIP more and it's much less to type pip install. But it's a matter of taste... Anyway we must see something like this after PIL installs:

This will indicate successful installation of PIL and components.

4. Installing Django is easy now. Jut do it whatever way you prefer. Like sudo pip install django or
sudo easy_install django or even sudo pip install django==1.1, if you prefer oldies. :)

I prefer to use virtualenv 'your virtual env name here' --no-site-packages for every app of mine... So I'm not even installing Django to main system path as well... Btw if you've used virtualenv with you projects, like I did, prepare yourself.  They need to be recreated. So be sure to save them with pip freeze. Maybe they will work if you pu them back to the same system path as they where... But there might be daemons here :).

I'll try to cover installation of MySQL environment on a "clean" Lion install in my next articles.

Anyway if you've found this article useful or think that I'm not right somewhere or even something less important... Feel free to drop me a comment here or contact me any available way.  Thanks for reading so far and hope this will shorten your time while installing some things.

Nov 16, 2011

Python: Debugging/Developing SMTP mail things

You often have 'situations' when you developing Contact forms, or any other forms which have to send emails with python. You have to install or have working SMTP server on your Dev machine like say 'monstrous' Sendmail or something like it. It can cause errors in Django/Python like [Errno 61] Connection refused:

[Errno 61] Connection refused

But python has it handled for you. Python's standard distribution has a test SMTP server that you can test sending mail with. It actually does not sends anything. But you can examine outgoing email message like you've actually send it.
Type something like:

python -m smtpd -n -c DebuggingServer localhost:1025

It will start a command line server on this console and it will print everything your server should send through SMTP.
Also make sure to modify mail sending code to use your non standart port number like so:

server = smtplib.SMTP(SERVER, 1025)
server.sendmail(FROM, TO, message)

Or with 'sudo' (if you have admin permissions) you can start a test SMTP server like you are actually using sendmail simply changing port number of your server.  Or just change 1025 to 25 (SMTP default port) if you are working under root permissions user. Like so:

sudo python -m smtpd -n -c DebuggingServer localhost:25

Found interesting? Have a better/simpler way or I am wrong?
Please drom a comment here. 

Nov 10, 2011

Django: Adding news archive. Querying models with date and time filtering.

Today I had some practice with filtering Django models by date.
Task was to archive old entries of the news. I've made 2 separate views for those. I was displaying actually news and another was ment to display archived items. Quite common task I guess. Also those 2 views are under wired buggy CMS, but we are not talking about that now.
Will save some examples for future of mine/someones usage.

Time. Subtracting a year from today with Python dateutil:

from datetime import date, timedelta - timedelta(days=days_to_subtract)

This example returns 'd' that has less days than specified in 'days_to_subtract'. timedelta function also can be fed with 'months', 'years' and so on. Becomes handy during time calculations. If you change this minus into plus you can calculate future time.

Filtering Querysets. Filtering by date ranges.

samples = Sample.objects.filter(, 1, 1),
                      , 1, 31)

samples = Sample.objects.filter(, 1, 1),
                      , 1, 31)

Django queryset filters is quite interesting things to discover. They also can cover almost any simple database query. In this example we are filtering model objects that are between dates range. This filters go through model's date fields and look for items in this date range. Only difference here that firs example includes 1-31 days and second includes 2-30 days range (excluding specified days).

  • '__gte' means '>='
  • '__lte' means '<='
  • '__gt' means '>'
  • '__lt' means '<'
Easy to remember right?
Found interesting, Helped,  Know a better way, I'm not right somewhere. Please drop me a comment below. 

Oct 31, 2011

Django: compressing CSS/JS files with django-compressor

There are 2 main usual tasks with web project's deployment and about .css and .js files.
First one - size minimization. But there are lot's of utilities helping you to compress CSS files. Delete unused spaces, comments and so on... Second one - Version control. For e. g. When you've updated the script on your deployment server, but user's browser uses old one until user manually hits 'Refresh'.
That's where Static files compressor comes in to mind.

Upon selecting among available one's found top "google" results:
- django-compress
- django-compressor
- webassets

Project uses 'django.contrib.staticfiles', so django-compress was not compatible... It does not support Django's static files gently. Webassets s a good lib. but project has a huge amount of different static. Maybe it's ok for e small project, but when you need to specify/change a 100's javascript in python module for certain templates... Nothing good comes in mind. And imho TEMPLATES... That's where thing's like those should live.

So Django-compressor was chosen using those criteria. And, to mention, project is actively developed, supported and documented. Anyway adding it to your django project is quite simple, as most good apps.

What it does, it turns this:

<script type="text/javascript" src="{{ MEDIA_URL }}public/js/jquery-1.4.1.min.js"></script>
<script type="text/javascript" src="{{ MEDIA_URL }}banners/js/jquery.cycle.all.js"></script>
<script type="text/javascript" src="{{ MEDIA_URL }}js/gb/greybox.js"></script>
<script type="text/javascript">
jQuery.browser.msie6 = jQuery.browser.msie && parseInt(jQuery.browser.version) == 6 && !window["XMLHttpRequest"];

if (!jQuery.browser.msie6) {
   $("div#body_wrapper").wrap('<div id="body_wrapper_shadow_right"><div id="body_wrapper_shadow_left">'+

Into this:

<script type="text/javascript" src="/static/CACHE/js/8dd1a2872443.js" charset="utf-8"></script>

The install is quite simple and you need to follow this manual.
Then configure it. Basically you hava all set for it to function. Except for me having a bit wired way of storing static files. I had to remap default COMPRESS_ROOT and COMPRESS_URL variables. It's quite easily done in And it has lot's of other settings, that I did not require.

Anyhow this is it. Now you can wrap all of your code with handy templatetags that will do all the work for you. it may look somehow like this:

{% load compress %}

{% compress css %}
<link rel="stylesheet" href="/static/css/one.css" type="text/css" charset="utf-8">
<style type="text/css">p { border:5px solid green;}</style>
<link rel="stylesheet" href="/static/css/two.css" type="text/css" charset="utf-8">
{% endcompress %}

or for .js files:

{% load compress %}

{% compress js %}
<script src="/static/js/one.js" type="text/javascript" charset="utf-8"></script>
<script type="text/javascript" charset="utf-8">obj.value = "value";</script>
{% endcompress %}

It will generate a custom .js/.css files with all of your compressed static at one file in your media/static dir that can be cashed. They will have links, like "/media/CACHE/css/105dsb963311.css"and put your newly generated script/style there. 

This is it. Use this in your projects. It economies your server's time/traffic and helps you to update scripts at all user's clients in their browsers when you deploy a new version.

Liked/used/hated/disliked/have a better way? Please leave a comment here about what and/or how you did this/suggest me to do this?

Oct 26, 2011

Django: 'ManyRelatedManager' object is not iterable

Upon development proces you often meet ManyToMany relations that throw errors in templates. I prefer to feed everything to context and play with it there.

This kind of error appears when you iterate through a model's ManyToMany field. To fix it just add a related 'all' manager through dot syntax. Like so:


class BannerImage(models.Model):
    image = models.ImageField(upload_to="somedir")

    def get_image_url(self):
        return 'Image url here'

class Banner(models.Model):
    name = models.CharField(max_length=250, null=True)
    images = models.ManyToManyField(BannerImage)

View/Tag or else, that creates context:

banner = get_object_or_404(Banner, pk=1)
return {'banner': banner,}

Template causing the error:

{% for image in banner.images %}
    <img src="{{ image.get_image_url }}" />
{% endfor %}

Template without the error:

{% for image in banner.images.all %}
    <img src="{{ image.get_image_url }}" />
{% endfor %}

I wrote this article for myself for remembering. I always forget this little thing. But if this method works for your case fill free to drop a "thankyou" comment or +1.

Aug 24, 2011

Django: Virtualenv with Eclipse befrending

Many people use Virtualenv. It has become common pattern now days. I like Virtualenv but it has no debugger. You can run 'python runserver' but it will newer help you see somthing like runtime variables and code flow. I like PyDev debugger. It helps me a lot.
Anyway I faced a problem recently. My employer wants me to run code at virtualenv. I needed to download and run project. I made it creating New virtualenv called 'venv' inside a project folder. Command is:

virtualenv --no-site-packages venv

It will create dir 'venv' inside current directory. So you should be at where you need to store your virtualenv (your project directory for e.g.). --no-site-packeges parameter says that your new virtual environment will not have packages installed in system. It's a must if you want to avoid collisions with newer versions of code.

Now we have a dir inside your project called venv containing our separated python environment. Lets activate it using command:

$ source venv/bin/activate

Your terminal will gain '(venv)' prefix at the beginning, indicating that we are now using alternative PYTHONPATH configuration. Now you may work almost like using normal system shell, except your PYTHONPATH is set tot  this newly created folder venv and all changes are stored inside it ONLY!

Lets make our dirty work. Usually it's something like in my case:

pip install -r requirements.txt

Or you can simply:

$ easy_install django

For the sake of test :). Anyway it's not the main point of an article.

I assume You already have eclipse project set up properly. I have an article about it: Installing Eclipse and PyDev for django. It also has some unralated info at first, about installing Eclipse from scrtch, but you can start reading from about in the middle of the text. It says how to set up a new project.

Anyway main point of befriending PyDev, Eclipse and virtualenv is IMHO the best way to do so:

1. Point your project's (in Eclipse) PYTHONPATH variable to your virtualenv dir's site-packeges.
To do so go to your project Properties, selecting it from the right mouse button context menu or any other way and select Python - PYTHONPATH option there. It might be like So:

2. Now add a directory inside your newly created virtualenv directory. (venv in my case) You mus not add the whole directory venv but a site-packages dir inside of it. It usually lies at this path:

$ venv/lib/python$$$/site-packages 
#where $$$ is your python version used for virtualenv created (2.7 in my case)

It may look like this:

3. Ta-da thats ALL!. Now hit Ok/Apply to save your settings and try to run/debug your project from eclipse using run/debug buttons. It Should be working right now. You need to create a proper run configuration BTW. But it's another whole story.

Comments and suggestions are welcome. Please write if you found it useful or not :)

Aug 12, 2011

Django: How and why to use migrations. Django-South.

Hi there guys and we're here to talk about migrations today. My app grown to complex app with profiles, social registration permits and so on. First time I've decided to make myself a simple app. Now it became complex enough and contains enough code to need code comments :). Anyway Migrations is a process anybody someday will need. I thought it's hard to learn or understand, but it's not.

Main point is that you:
- save your current database tables structure
- change your model tables
- scan for differences and create script
- then write changes to your database with automatically generated python script.

Sound's simple? I't not all so simple in fact. App that you need to learn is Django-south. Its main objectives are to provide a simple, stable and database-independent migration layer to prevent all the hassle schema changes over time bring to your Django applications. It has quite understandable tutorial here.

So if you're tired of ALTER'ing your tables and writing code to python Sqlite console or editing raw SQL, go get yourself a copy of south. It's an app that IMHO everybody should use/hear/understand.

It is meant to change old one and quite simple command ' syncdb' someday. For now go get yourself this tool and learn to use it. It took me about half an hour to understand basics.. So it's not hard (at first) :).

Brief look at usage:

1. Install app in some way. Use pip/easy_install script or simply put 'south' dir to your project root dir.
Add 'south', to your INSTALLED_APPS dictionary.
!IMPORTANT! you need to make syncdb after that. Another way wo will get an error with future commands.

2. Connect existing app's to migrations. Use command python convert_to_south myapp to convert your app from syncdb ready to south migratable. Or you can run python schemamigration newappname --initial to add south migrations ability to your new app. In general it will create directory 'migrations' inside your app directory.

3. Change your model. I dont think you need separate stop here. But... Just in case add some field to your django model like bollean or text and/or edit your existing field rather by adding 'null=True' for e.g.

4. Create your migration. It can be done by command python schemamigration newappname --auto to create your migration script. you can check it in your app's 'migrations' dir. It has numbers in the beginning with migration number.

5. Run your migration. It is done by command python migrate appname or even without an app name: python migrate to migrate all apps. Thats it.

Hope you'll never use 'syncdb' again for complex tasks like this. I'm adding it to my base developer tools. And you?

Aug 1, 2011

Django: adding code execution on your app syncdb, or how to use Django Signals.

Hi there and let's talk about app initialization in Django. There are some cases when you want to initialize a Django app with creating some default values in database. In my case it was necessity to create default album in database to post user photos to. Sometimes you could just use get_or_create for those purposes. But it will be a good example if we will need something more complex in our app initialization; for e.g. generating thumbnails for photos or cleaning unused temporary files etc. So let's get started:

Good place to put your initialization scripts is your app's file. You can examine Djangoproject wiki for more info. Anyway here is my code for making this:

  1. from django.db.models.signals import post_syncdb
  2. import models
  3. from models import Album
  5. def create_first_album(sender, **kwargs):
  6.     """
  7.    Create your album sequence to create default album to post photos to
  8.    checks for existence of this album and creates one if none exists.
  9.    """
  10.     obj, created = Album.objects.get_or_create(title="User's Posted"
  11.     if created:
  12.         print("Created album 'User's Posted'.")
  13.     else:
  14.         print("NOT CRATED album 'User's Posted'.")
  15.     pass
  17. post_syncdb.connect(create_first_album, sender=models)

It is made using Django Signals documentation example. This code creates album 'Users Photos' upon first running os 'syncdb' bu my apps user. It's quite simple but shows the idea.

Main idea here that you need to add code to file of your app. It will stick to signal you've chosen (post_syncdb in our case) and execute on it's call. 
Than you will write your function to execute (def create_first_album in our case) and say when to execute this function. We sticked it to post_syncdb signal.

This way cou can handle more handy signals like using pre_init signal to execute some code not only at syncdb but at every app's startup or even add some code to template upon rendering. Read Django docs Signals to know more.

Jul 18, 2011

Django: Beautiful multiple files Upload Plugin using jQuery UI.

After successful developing python back-end for Sebastian Tschan jQuey Plugin  I've decided to make a reusable app for that purpose. So:

This is a plugin, made using multiupload form from Sebastian Tschan.
It uses jQuey UI and jQuery instead of Flash uploader.
On Django side it uses sorl-thumbnails and PIL.
You can use it in your applications with simple inclusion tag

jQuery Features it has:
  • Multiple file upload:
    Allows to select multiple files at once and upload them simultaneously.
  • Drag & Drop support:
    Allows to upload files by dragging them from your desktop or filemanager and dropping them on your browser window.
  • Upload progress bar:
    Shows a progress bar indicating the upload progress for individual files and for all uploads combined.
  • Cancelable uploads:
    Individual file uploads can be canceled to stop the upload progress.
  • Resumable uploads:
    Aborted uploads can be resumed with browsers supporting the Blob API.
  • Chunked uploads:
    Large files can be uploaded in smaller chunks with browsers supporting the Blob API.
  • Preview images:
    A preview of image files can be displayed before uploading with browsers supporting the required HTML5 APIs.
  • No browser plugins (e.g. Adobe Flash) required:
    The implementation is based on open standards like HTML5 and JavaScript and requires no additional browser plugins.
  • Graceful fallback for legacy browsers:
    Uploads files via XMLHttpRequests if supported and uses iframes as fallback for legacy browsers.
  • Standard HTML file upload form:
    Shows a standard HTML file upload form if JavaScript is disabled.
  • Cross-site file uploads:
    Supports uploading files to a different domain with Cross-site XMLHttpRequests.
  • Multiple plugin instances:
    Allows to use multiple plugin instances on the same webpage.
  • Customizable and extensible:
    Provides an API to set individual options and define callBack methods for various upload events.
  • Multipart and file contents stream uploads:
    Files can be uploaded as standard "multipart/form-data" or file contents stream (HTTP PUT file upload).
  • Compatible with any server-side application platform:
    Works with Google App Engine (Python, Java), Ruby on Rails, PHP and any other platform that supports HTTP file uploads.
 Python Features:
  • Uses own model for temporary storing files:
    Uploaded files Model can be used anywhere in your code or transferred to another model of your choice.
  • Can be added using 2 simple templatetags:
    {% load multiuploader %} and {% multiupform %} in a form place.
  • Opensource!

Adding this AJAX form to your web site is as simple, as adding this 2 tags to your template:

    {% load multiuploader %}
    {% multiupform %}

It has it's own model for storing images there.
It has some bugs a bit but is quite usable. Feel free to use it in your projects.

Demo is available on Sebastian's website:
You can try it there but without python back-end.

I'm using it on my Photoblog:

Finally you can download/fork it and view installation instructions on my GitHub repository dedicated for it:

UPD: I've done a demo app with usage of this plugin. Maybe it will be usefull to figure out some undescribed usage moments or so.

Jul 12, 2011

Django: Creating multi upload form without using Flash

Hi there! Today I want to tell you about my experience of adding Multiple files upload form to my Django project Photoblog. I searched google for lots of plugins, but found only Flash usage examples. I dislike Flash technology, as for my personal usage, and want to build some more quickly working UI. Personally I have flash blocker installed on my browser. So that's main ideas pf my jQuery plugin selection. I found IMHO the best for my case plugin by Sebastian Tschan. It had only one problem - No available Django examples. Let's try to fill the gap here... So my experience on befriending Sebastian Tschan's jQuery File Upload Plugin with Django is the topic of this article. Let's finally get started. :)

Sebastian has built an important plugin IMHO. I want to thank him for his noble work of helping people with their tasks. It uses only jQuery UI and no flash, making it work more programmers way... So enough with the lyrics.

Let's try to adapt his example to a django Project. I used new one, built with Eclipse using Django 1.3. So fell free to create yours and Download Sebastian's plugin. After that we will get existing project using PHP as main gear. Let's try to adapt it.

Main problems I met creating it:
    - jQuery UI uses similar templates language with Django.
    Problem solved by changing Django code to generate '{{' scopes instead variables '{{ open_tv }}' and '}}' instead '{{ close_tv }}' accordingly. You can watch it in template example of this demo project.
    - write view to get files from AJAX form and return proper JSON in response. You can examine mine in app multiuploader.
    - write view to delete uploaded photo. Mine is multiuploader_delete.
    - generate thumbnail for showing uploaded image. I used sorl-thumbnails that uses PIL.
    - generate JSON response to form like this:


Let's get straight to business. So imagine this simple model. It has only image and title (used image's file name for title):
from django.db import models

class Image(models.Model):
    title = models.CharField(max_length=60, blank=True, null=True)
    image = models.FileField(upload_to="images/")
    def __unicode__(self):

Now let's write urls as we need for main images view, multiuploader form and file deletion like so:
from django.conf.urls.defaults import *

from multiuploader.views import multiuploader_delete

urlpatterns = patterns('',
    (r'^delete/(\d+)/$', multiuploader_delete),
    url(r'^$', 'django_multiuploader_demo.multiuploader.views.image_view', name='main'),
    url(r'^multi/$', 'django_multiuploader_demo.multiuploader.views.multiuploader', name='multi'),
Main view is quite simple. It's just for handy demo that photos exist after upload. It will look like so:
def image_view(request):
    items = Image.objects.all()
    return render_to_response('images.html', {'items':items})

It will pass our model directly to template. It will render our model like so:

{% load thumbnail %}
<h1><a href="{% url multi %}">Multiuploader</a></h1>
<br />
<div class="photos">
{% for item in items %}
{% thumbnail item "80x80" crop="10px 10px" as im %}
    <img src="{{ im.url }}" width="{{ im.width }}" height="{{ im.height }}">
{% endthumbnail %}
{% endfor %}

Final result is simple images model:
Now that we have application to upload photos to let's proceed.
Let's create a a main view that will return our multiuploader template in cese of GET and receive POST requests with photos. It might look like so:
def multiuploader(request):
    if request.method == 'POST':'received POST to main multiuploader view')
        if request.FILES == None:
            return HttpResponseBadRequest('Must have files attached!')

        #getting file data for farther manipulations
        file = request.FILES[u'files[]']
        wrapped_file = UploadedFile(file)
        filename =
        file_size = wrapped_file.file.size ('Got file: "'+str(filename)+'"')

        #writing file manually into model
        #because we don't need form of any type.
        image = Image()
        image.image=file'File saving done')

        #getting url for photo deletion
        file_delete_url = '/delete/'
        #getting file url here
        file_url = '/'

        #getting thumbnail url using sorl-thumbnail
        im = get_thumbnail(image, "80x80", quality=50)
        thumb_url = im.url

        #generating json response array
        result = []
        response_data = simplejson.dumps(result)
        return HttpResponse(response_data, mimetype='application/json')
    else: #GET
        return render_to_response('multiuploader_main.html', 

Now let's add the view that will delete photos for us. It might look like so:
def multiuploader_delete(request, pk):
    if request.method == 'POST':
        image = get_object_or_404(Image, pk=pk)
        return HttpResponse(str(pk))
        return HttpResponseBadRequest('Only POST accepted')

Finally main Form would look similar to this:

I will not provide main template of uploader, because it's quite big. You can fork/download it from my django_multiuploader_demo at github.

Feel free to use this demo anyhow in your project, but please drop me a comment here if you find it useful. 

Ideas, Suggestions, Critics? Welcome to comments...