Dec 20, 2014

Error copying files in Finder of OS X

I have an SD flash card (8 GB Kingston) and a MacBook Pro 13" (Late 2011 model). I also use external card reader time to time.
This problem persisted on all the conditions. The error message was stating:

The Finder can’t complete the operation because some data in “” can’t be read or written.
(Error code -36)

This was happening while copying Photos from my camera (cr2 files). It worked, however, in case of copying files up to 200 MB in total size of batch. It dropped this error message and did stop to copy files upon selecting of lots of RAW files. E.g. all of them and attempting to copy them from a flash drive.

Problem occurred with building miniatures of the CR2 files. Those files are quite heavy photos (25+ MB) and building miniature did take some time. While building those miniatures on both MAC and SD card finder windows it did die.

For me it was enough to change the view from icons to list. E.g.:
This did solve it for me. Other solution (Suspect in case your MAC would be lower processor model and/or less productive hardware version) is to disable building miniatures for files in Finder.

Jul 31, 2014

5 most common Rsync command usage examples

rsync is a command for remote sync.

It is used to synchronise one location to another in a simple way. Location is meant to be local directory, server or remote web server or whatever accessible by ssh.

Advantages of using rsync over other tools is speed and bandwidth requirements. First time rsync copies entire contents of the directory provided and increments changes over next sync times.

Command syntax:

$ rsync options source destination

1. Synchronising local directories:

$ rsync -zvr /tmp/logs/ /tmp/logs1/
building file list ... done
created directory /tmp/logs1

sent 98 bytes  received 48 bytes  292.00 bytes/sec
total size is 0  speedup is 0.00
Used rsync command options here:

  • -z is for compression
  • -v is for verbose output
  • -r is for recursive directory scanning
By default rsync does not preserve timestamps. 

2. Preserve timestamps and permissions during sync:

$ rsync -azvr /tmp/logs/ /tmp/logs1/
building file list ... done

sent 122 bytes  received 48 bytes  340.00 bytes/sec
total size is 0  speedup is 0.00
Option -a preserves symbolic links, timestamps, user permissions and ownership.

3. Synchronize from Local to Remote:

$ rsync -avz /tmp/logs/ root@
root@'s password: 
building file list ... done
created directory /home/user/logs

sent 122 bytes  received 48 bytes  68.00 bytes/sec
total size is 0  speedup is 0.00
It is required to specify username and ip-address of the remote server, while doing synchronization. It is also required to specify the destination directory on the remote server. The format is username@machinename:path. Sometimes depending on your credentials and ssh authentication method you may need to enter a password. Like in this example.

4. Synchronize from Remote to Local:

$ rsync -avz root@ ~/temp/
root@'s password: 
receiving file list ... done

sent 290 bytes  received 11002 bytes  3226.29 bytes/sec
total size is 16956702  speedup is 1501.66
This example is opposite to previous. we have synchronized a list of only changed files here. So speedup is relatively high.
5. View the rsync Progress during Transfer:
$ rsync -avz --progress root@ ~/temp/
root@'s password: 
receiving file list ... 
284 files to consider
        1372 100%    1.31MB/s    0:00:00 (xfer#1, to-check=252/284)
        1024 100% 1000.00kB/s    0:00:00 (xfer#2, to-check=251/284)
        8658 100%    8.26MB/s    0:00:00 (xfer#3, to-check=250/284)
           0 100%    0.00kB/s    0:00:00 (xfer#4, to-check=249/284)
        1024 100%  500.00kB/s    0:00:00 (xfer#5, to-check=248/284)
        8632 100%    2.74MB/s    0:00:00 (xfer#6, to-check=247/284)

sent 158 bytes  received 9666 bytes  3929.60 bytes/sec
total size is 16956814  speedup is 1726.06
Running with --progress option showcase detailed progress of server interaction during load operations.

Thats it. Hope you find this article helpful to you. Comments? Suggestions?

Jul 29, 2014

Django adding custom widget to Django Admin

Sometimes you need to get out of standard behaviour of your Django admin. It means expanding its functionality with custom things. In our case we have a model that we want to add a custom html element. Button that must load something via AJAX from another place to be exact.
We have a typical Django polls application for simplicity. Here is its structure:

First of all we need to create a structure. So here comes our model:
from django.db import models

class Poll(models.Model):
    question = models.CharField(max_length=200)
    pub_date = models.DateTimeField('date published')
We have to have our django application admin configuration to display that model:
from django.contrib import admin
from polls.models import Poll
It will look in Django admin like so:
You can see polls available here in admin. You can do all the typical things with Polls model. Add/Delete/Edit... Using standard Django features. Imagine we have something on that polls model that we need to expand. Say, we need a button to visit external site here, made like an additional edit field. Imagine we need to take something on external website. We can post something to that website from our filled django admin form. To do that we need to create a custom form field. I have not found any standard widgets to do that. It is required to write our own field and attach it to the Django admin form.
We do not need own field here. Just a widget. Widgets are standard Django types. It is possible to read wide description of them in the official documentation: Django Widgets. I wont discuss here best ways to implement this task and there might be better ones. However I have chosen a form widget for simplicity.

It is required to have a model expansion form and a custom widget in order to do so. Registration fo that for m in django admin setup is also required. Lets implement this.
Starting with a form and a widget:
from django import forms
from django.utils.safestring import mark_safe
from django.template.loader import render_to_string

from polls.models import Poll

class ButtonWidget(forms.Widget):
    template_name = 'auth_button_widget.html'

    def render(self, name, value, attrs=None):
        context = {
            'url': '/'
        return mark_safe(render_to_string(self.template_name, context))

class PollsForm(forms.ModelForm):
    button = forms.CharField(widget=ButtonWidget)

    class Meta:
        model = Poll
It is located in a newly created file - at the django app - polls directory. Here we have a custom written widget that inherits a typical default Django widget (forms.Widget). we have provided it a template - auth_button_widget.html and a custom render() method that provides extra context for that template.
Next comes the PollsForm class with a custom button field that uses that widget. Note here Meta for that model is specified, indicating we are adding that field to the Polls model add/edit form.
Template, residing in all django templates directory will look like so:
{# auth_button_widget.html #}
<a href="{{ url }}">Go Button</a>
We have our form and widget and a template to render that form element. Time to use that form.

Using this form means modifying our file and registering custom model admin there. It will use our custom form and a widget. Resulting will look like this:
from django.contrib import admin
from polls.models import Poll
from polls.forms import PollsForm

class PollsAdmin(admin.ModelAdmin):
    form = PollsForm, PollsAdmin)
Resulting change will add a custom form to edit or add the Polls model, containing our custom button field. It will look something like so:

Notice the button we have add here. Its just a simple HTML element that needs to be styled and animated by some CSS and JS. It can be done adding Media to our form widget. Resulting new widget will look like this:
# (partially)
class ButtonWidget(forms.Widget):
    template_name = 'auth_button_widget.html'

    class Media:
        js = (
        css = {
            'all': (

    def render(self, name, value, attrs=None):
        context = {
            'url': '/'
        return mark_safe(render_to_string(self.template_name, context))
Its a registration of static files to use in your widget. They will represent a scripting and styling for that field and will be included to the page of django admin form automatically.
For this to function you have to have django static files configured

Content of scripts and styles are out of the scope of this article. Main idea here is to highlight a proper way to add and write a custom widget from the backend perspective.

Comments, suggestions?

Jul 14, 2014

Virtualenv for Django

One of main features and troubles for Django projects is the Django version you are using. Example for this would be the occasion of starting project for Django 1.4.x version. Trying to run it under Django 1.7.x would lead to various troubles, counting template changes, misconfiguration errors and so on.

Solution is to use virtual environment. The tool for this is called Virtualenv. It is a wrapper for a python interpreter. It enables you to have multiple versions of python packages that run independently.

The Virtualenv creates you a standalone environment. Basically it copies and keeps a system Python of yours in a specified directory. All the python packages will be installed there. (In case you have not forgot to activate it first, of course)

Lets get started then. We need Virtualenv itself for the first. There are many ways to install it depending on your system. Most obvious to use pip. Your installation command will look something like that:
pip install virtualenv
This installs the tool itself to your system. this only must be done once.

Assuming we have it now and going forward to usage basics. We need to create a virtual environment for our particular project. Usual rule for most of developers is to have a single virtual environment for a standalone project.
To create one for ours we need to be in the directory where we want it to reside. (Virtual environment is not movable by default). So choose wisely.
I suggest it be somewhere in the project directory. The directory that is up one step from the project one is ok too. Assuming we have a project that is called "example". I will enter the directory of that project:
$ cd example
garmoncheg:~/Projects/example$ ls -la
total 8
drwxr-xr-x  4 garmoncheg  staff   136B Jul 14 14:26 ./
drwxr-xr-x  3 garmoncheg  staff   102B Jul 14 14:26 ../
drwxr-xr-x  6 garmoncheg  staff   204B Jul 14 14:26 example/
-rw-r--r--  1 garmoncheg  staff   250B Jul 14 14:26
We have here a file to run our project with and a project files directory. Seems like a nice place to create a virtual environment at. Time to do it with executing a command:
$ virtualenv venv
New python executable in venv/bin/python
Installing setuptools............done.
Installing pip...............done.
We have commanded our virtualenv to create a directory for our new project environment and call it venv. You could use any name here. it will indicate your projects name or whatever your fantasy is up to. Mine project directory now looks like this:
$ ls -l
total 9
drwxr-xr-x  6 garmoncheg  staff   204B Jul 14 14:26 example/
-rw-r--r--  1 garmoncheg staff   250B Jul 14 14:26
drwxr-xr-x  6 garmoncheg  staff   204B Jul 14 14:32 venv/
I have a script here, example directory with project files and a venv directory containig my environment files.
Now we have created a directory for environment and put our environment in it. However system is not aware we are using this environment. We must activate it now. Doing that is easy enough.
Assuming you are in the same project directory and called the virtual environment we are creating "venv". Execute a command:
$ source venv/bin/activate
This will tell the system we are using the interpreter set we have just created. It is indicated with (venv) at the start of your terminal prompt. this means we are "inside" the environment now.
Note we are in the same directory we where before.

Lets familiarise ourselves with main virtualenv functionality now. Whats the main point of it?
Simple. All the "pip install something" made now in this terminal will install a Python package inside this venv folder, leaving the main system python files intact. You can have as many environment as you wish. You can install whatever Python package you want inside of that virtual environment.

Now lets go through basic commands. Lets try to run our project now, assuming you have followed my previous instructions. To do that just type:
(venv)$ ls
total 9
drwxr-xr-x  6 garmoncheg  staff   204B Jul 14 14:26 example/
-rw-r--r--  1 garmoncheg  staff   250B Jul 14 14:26
drwxr-xr-x  6 garmoncheg  staff   204B Jul 14 14:32 venv/
(venv)garmoncheg:~/Projects/example$ python runserver
Traceback (most recent call last):
  File "", line 8, in <module>
    from import execute_from_command_line
ImportError: No module named
Whoops. We can not do that because there is no Django installed. Even though you have had it in system it does not seem to be present now.
All the credits to virtualenv activation command. (source path/to/activate/script). We have to have it installed in the environment now. However it is more convenient to look for ourselves there is no Dango in environment. Lets execute:
(venv)garmoncheg:~/Projects/example$ pip freeze
Tadaam. We only have wsgiref in our console environment now. Lets install django:
(venv)garmoncheg:~/Projects/example$ pip install django
Downloading/unpacking django
  Downloading Django-1.6.5.tar.gz (6.6MB): 6.6MB downloaded
  Running egg_info for package django
    warning: no previously-included files matching '__pycache__' found under directory '*'
    warning: no previously-included files matching '*.py[co]' found under directory '*'
Installing collected packages: django
  Running install for django
    changing mode of build/scripts-2.7/ from 644 to 755
    warning: no previously-included files matching '__pycache__' found under directory '*'
    warning: no previously-included files matching '*.py[co]' found under directory '*'
    changing mode of /Users/leopard/Developer/Python/Me/Tests/Projects/example/venv/bin/ to 755
Successfully installed django
Cleaning up...
Tadaa. We have our django for the project now. Lets make sure with pip freeze:
(venv)garmoncheg:~/Projects/example$ pip freeze
And our project now can run inside a wrapper. Lets test with:
(venv)leopard@garmoncheg:~/Projects/example$ python runserver

Validating models...

0 errors found
July 14, 2014 - 12:00:03
Django version 1.6.5, using settings 'example.settings'
Starting development server at
Quit the server with CONTROL-C.
Yes it works.

Cons here that we must activate a virtual environment each time we want to seriously interact with our project. So even to run it we need to activate it first. but it is really the only option to work with multiple project in a system in an easy way.

To return the terminal to a proper state (back to system Python interpreter) you could just close it or deactivate the environment:
(venv)garmoncheg:~/Projects/example$ deactivate
Tadaa. (venv) preceding the terminal prompt is gone and we are back to the system interpreter.

Hope this helps someone. Suggestions? Helped? Please comment! I will really appreciate all the info.

Jun 12, 2014

Django CMS custom Plugin ManyToMany fields problems.

Developing a website for Django CMS I have done a plugin. This plugin however have had a problem. It had an m2m field. While selecting images there (It was a gallery plugin). It was not saving it to production. This way main problem with this field was that plugin have displayed m2m choices on a draft page and while storing it to live it fails.
Django CMS Pages is built like so it has multiple versions of pages. Each Page model that is changed is stored under new revision. And so plugin is copied and relinked to that page too. So plugin revisions multiply to.
For e.g. in case you have a Draft page with two plugins that have model primary key number (pk in future) 12 and 14 accordingly. Say you hit publish changes. Not only PK of Page model rises and data are copypasted into new empty Page model instance. Those plugins are copied to. So mentioned plugins PK would change to 15 and 16 accordingly, assuming 14 is the latest plugin pk. New copied plugins will be linked to new page. Thats how things work in Django CMS.

There is also a line in the CMS code stating in its comment that you need to override m2m storing by yourself.  Somehow its omitted from basic Django CMS tutorials and I can only find it in the old versions documentation for now. SO THE SOLUTION TO ALL THIS:

Say you have CMSPlugin a model for your custom plugin:
class SelectImagePlugin(CMSPlugin):
    """Extension model for plugin class"""
    display_text = models.BooleanField('Project text overlay display')
    images = models.ManyToManyField(Image)

    def copy_relations(self, old_instance):
        self.images = old_instance.images.all()
Here method copy_relations overrides default empty method.
For foreign keys it is described another behaviour and copy_relations mappings. However Foreign keys were working like intended by themselves in my plugins. SO I believe info about them is outdated a bit.
Otherwise it may be useful to read about it in official docs.
Thats it. Hope it will help someone like myself to solve a problem like this in a timely manner.
Comments? Suggestions?

May 9, 2014

Implementing a Multiple (Radio) Select + “Other” widget in Django

I have had a task to implement Radio Select field with "Other" choice field in a Django Model Form. Here is my implementation in case someone would benefit from it. The idea is to have it all stored to Django's CharField type of data and have a preselected set of fields. We have a model:
class Entry(models.Model):

        ('choice1', 'choice1'),
        ('choice2', 'choice2'),
        ('choice3', 'choice3'),
        ('Other', 'Other Please Specify'),

    choice = models.CharField(_("Selected your choice"), max_length=250)
That makes it ready for the set of fields for for our form. We are overriding Model form default field for this type of data (CharField) by intentionally specifying field with the same name.
from django.forms import ModelForm, ChoiceField, RadioSelect
from models import Entry
from fields import ChoiceWithOtherField

class EntryForm(ModelForm):

    choice = ChoiceWithOtherField(label=_("Selected your choice"), choices=Entry.SET_OF_CHOICES)
Note the import here. We are using custom form field here. It is a modified version of Django's
Here is the fields snippet:
from django import newforms as forms
from django.utils.encoding import force_unicode

class ChoiceWithOtherRenderer(forms.RadioSelect.renderer):
    """RadioFieldRenderer that renders its last choice with a placeholder."""
    def __init__(self, *args, **kwargs):
        super(ChoiceWithOtherRenderer, self).__init__(*args, **kwargs)
        self.choices, self.other = self.choices[:-1], self.choices[-1]

    def __iter__(self):
        for input in super(ChoiceWithOtherRenderer, self).__iter__():
            yield input
        id = '%s_%s' % (self.attrs['id'], self.other[0]) if 'id' in self.attrs else ''
        label_for = ' for="%s"' % id if id else ''
        checked = '' if not force_unicode(self.other[0]) == self.value else 'checked="true" '
        yield '<label%s><input type="radio" id="%s" value="%s" name="%s" %s/> %s</label> %%s' % (
            label_for, id, self.other[0],, checked, self.other[1])

class ChoiceWithOtherWidget(forms.MultiWidget):
    """MultiWidget for use with ChoiceWithOtherField."""
    def __init__(self, choices):
        widgets = [
            forms.RadioSelect(choices=choices, renderer=ChoiceWithOtherRenderer),
        super(ChoiceWithOtherWidget, self).__init__(widgets)

    def decompress(self, value):
        if not value:
            return [None, None]
        return value

    def format_output(self, rendered_widgets):
        """Format the output by substituting the "other" choice into the first widget."""
        return rendered_widgets[0] % rendered_widgets[1]

class ChoiceWithOtherField(forms.MultiValueField):
    ChoiceField with an option for a user-submitted "other" value.

    The last item in the choices array passed to __init__ is expected to be a choice for "other". This field's
    cleaned data is a tuple consisting of the choice the user made, and the "other" field typed in if the choice
    made was the last one.

    >>> class AgeForm(forms.Form):
    ...     age = ChoiceWithOtherField(choices=[
    ...         (0, '15-29'),
    ...         (1, '30-44'),
    ...         (2, '45-60'),
    ...         (3, 'Other, please specify:')
    ...     ])
    >>> # rendered as a RadioSelect choice field whose last choice has a text input
    ... print AgeForm()['age']
    <li><label for="id_age_0_0"><input type="radio" id="id_age_0_0" value="0" name="age_0" /> 15-29</label></li>
    <li><label for="id_age_0_1"><input type="radio" id="id_age_0_1" value="1" name="age_0" /> 30-44</label></li>
    <li><label for="id_age_0_2"><input type="radio" id="id_age_0_2" value="2" name="age_0" /> 45-60</label></li>
    <li><label for="id_age_0_3"><input type="radio" id="id_age_0_3" value="3" name="age_0" /> Other, please \\
specify:</label> <input type="text" name="age_1" id="id_age_1" /></li>
    >>> form = AgeForm({'age_0': 2})
    >>> form.is_valid()
    >>> form.cleaned_data
    {'age': (u'2', u'')}
    >>> form = AgeForm({'age_0': 3, 'age_1': 'I am 10 years old'})
    >>> form.is_valid()
    >>> form.cleaned_data
    {'age': (u'3', u'I am 10 years old')}
    >>> form = AgeForm({'age_0': 1, 'age_1': 'This is bogus text which is ignored since I didn\\\\'t pick "other"'})
    >>> form.is_valid()
    >>> form.cleaned_data
    {'age': (u'1', u'')}
    def __init__(self, *args, **kwargs):
        fields = [
            forms.ChoiceField(widget=forms.RadioSelect(renderer=ChoiceWithOtherRenderer), *args, **kwargs),
        widget = ChoiceWithOtherWidget(choices=kwargs['choices'])
        self._was_required = kwargs.pop('required', True)
        kwargs['required'] = False
        super(ChoiceWithOtherField, self).__init__(widget=widget, fields=fields, *args, **kwargs)

    def compress(self, value):
        if self._was_required and not value or value[0] in (None, ''):
            raise forms.ValidationError(self.error_messages['required'])
        if not value:
            return [None, u'']
        # Patch to override model specific other choice and return CharField value instead of choice tuple
        if value[0] == 'Other':
            return value[1]
            return value[0]
        # Use this for field to return tuple
        #return value[0], value[1] if force_unicode(value[0]) == force_unicode(self.fields[0].choices[-1][0]) else u''
It is based on the code from the snippet by sciyoshi. It is put here because there are some modifications to in inline with idea that things could easily disappear without a trace through the internet.
Anyway it will show something like this in the end:
It's up to your imagination to style it now. Hope this howto helps someone with similar problem to save some time.

May 1, 2014

How to delete a remote git tag

I'm always forgetting this. So decided to put it here for someone's benefit.

I have the git tag added by command:
git tag -a 1.2.0 -m "1.2.0"
this means my tags tree will be updated with the tag 1.2.0 and message 1.2.0
Usually it marks the version of the build/release.

I now push a tag to origin by command:
git push origin --tags
It is possible to see it at my repository github tagged.

Time now to delete a tag.
It can be done by command:
git tag -d 1.2.0
This will remove a local tag 1.2.0. Leaving origin intact. Pushing tags to origin does not give anything.
To remove a remote tag now it is required to execute command:
git push origin :refs/tags/1.2.0
It will remove tag 1.2.0 at origin. This is the only way how to move tags to another commits.

Apr 28, 2014

Pillow compile error clang: -Wno-error=unused-command-line-argument-hard-error-in-future fix

I have had a problem with compiling Pillow recently. the error was with Pillow/Pil compilation.
pillow install error clang: error: unknown argument:
Apple updated command line tools. So the cc command was updated too.
$ gcc --version
Configured with: --prefix=/Applications/ --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 5.1 (clang-503.0.38) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin13.1.0
Thread model: posix
The bug is in GCC. It is quite serious because those flags support will be dropped in future. Fortunately there is a worakround for now:
export ARCHFLAGS="-Wno-error=unused-command-line-argument-hard-error-in-future"
It works for Pillow 2.4.0 and I can compile it with Apple LLVM 5.1 (That is current now), as of Xcode 5.1.
Hope my findings will help someone.

Feb 3, 2014

Django: adding additional Admin static and writing JS snippets for Django Admin

It often gets that you need it because you invent something that is meant to be displayed for admin. E.g. some nasty form field or something. Here is my recipe to add some additional Django static to your admin.

Rather than writing a widget, that is more proper way here... I was using a widget already (from another package). So I could not use it properly, at least without a ton of overrides and complexity it leads to. SO I've decided to add the JS functionality to override a lack of backend.

First of all we should add the javascript/css like so:
class MyModelAdmin(admin.ModelAdmin):
    # admin additions

    class Media:
        css = {
            "all": ("css/my_style.css",)
        js = ("js/my_script.js",)
This add should list your files, loaded in the admin. So you can access them and see in the Django admin frontend itself. Like so:
Now that your script and style are loaded you could just hack anything you want dynamically. Django admin already has a jQuery attached, so you can do whatever you want with it. But there is a problem. You can not just do $(....... your code. This simply does not work and will get error like this:
(Uncaught TypeError: Property '$' of object [object Object] is not a function)
There is no usual jQuery namespace there. but there is a simple hack.
var $ = django.jQuery;

// your usual jQuery code goes here
// e.g.:
// $(document).ready(function() {
    // $('#admin_big_image').wrap('<div class="mrmen-b">');
// });
Thats all. Yu may improvise. E.g. dynamically create or alter DOM elements and manipulate them. Im my case it was a simple Admin banner overlay into image displayed by 3-d party django plugin.
Imagination is the limit.

External links: About what may be wrong with your scripts About whee is this idea taken from in Django docs.

Suggestions? Wrong? Comment here!

Jan 2, 2014

Lightning unofficial cable trick for iOS 7

I have found a hack of how to firs time sync your iphone via unofficial (cheap Chinese) cable.
It may be a hint to you if you are buying an iPhone from a second hand without accessories.

You will need TADAA! Official Apple cable but not for a lightning port. And another Apple device.

The thing is if you connect both your new iPhone or iPad via lightning port and using unofficial cable and ANY of your Apple devices using OFFICIAL Apple cable. It will trust your lightning connected device.

Some magic happens and one device connected with official apple cord makes another device connected via official cord too.

TADAA. Now you can sync at least once... and use Wi-Fi sync later on...

Hopoe it helps you. Comments?