Dec 30, 2012

Raspbery Pi as a home file server

I've wanted more from my PI, besides Time Machine functions, established in my earlier articles. So I've decided to make it a SAMBA server. I have a Mac and a Windows PC. And several iPads/iPhones. So the decision was obvious. To access my storage content I'd need a samba share. Because it is recognized by all this tech. iPad has Oplayer, to watch movies without conversion... GoodReader for docs. One word - SAMBA share. SO this article will be about adding a samba share to your Raspberry Pi.
Note I have a Time Machine already set up. But it won't matter much. I only have another drive index letters and so on. I'll try to cover this as much as possible. And macs somewhat easily read NTFS filesystems. So we will be mounting NTFS filesystem volume to a Raspberry Pi with SAMBA network sharing. Let's get on to it.

To set up a Raspbery Pi home network Samba server you will need to:
- Connect an external drive (USB HDD in my case).
- Update fstab for auto-mounting of it on system boot
- Install and configure samba
- Enjoy

1. Connecting an external USB drive.

Assuming you hava an external hard drive connected type the command:
pi@raspberrypi /mnt $ sudo blkid
/dev/mmcblk0p1: SEC_TYPE="msdos" UUID="XXXX_XXXX" TYPE="vfat" 
/dev/mmcblk0p2: UUID="XXXXXX_XXXXXX_XXXXX_XXXX" TYPE="ext4" 
/dev/sda1: UUID="XXXXXXX-XXXXXXX-XXXXXXX-XXXXX" LABEL="Time Machine" TYPE="hfsplus" 
/dev/sdb1: LABEL="Data1" UUID="5XXXXXXXXXX1" TYPE="ntfs"
Note I have sda1 drive with hfsplus filesystem for AFP shares so my NTFS drive is named sdb1. Yours will probably be sda1 out of the box. Note that. We now see the NTFS hdd. Lets make it mounted. Typing mount command will show it's not yet there.
pi@raspberrypi /mnt $ mount
/dev/root on / type ext4 (rw,noatime,user_xattr,barrier=1,data=ordered)
# ...
/dev/sda1 on /mnt/TimeMachine type hfsplus (rw,nosuid,nodev,noexec,relatime,umask=0,uid=0,gid=0,nls=utf8)

2. Update fstab for auto-mounting of it on system boot

Lets make a directory for mounting a drive:
pi@raspberrypi /mnt $ sudo mkdir /mnt/data
Now lets update fstab to add a setting for proper auto-mount of our drive on boot.
pi@raspberrypi /mnt $ sudo nano /etc/fstab
Lets add a line. Something like.
UUID="XXXXXXXXXXXXXX" /mnt/data ntfs rw,auto 0 0
Where UUID is your device UUID, that you can find out using sudo blkid command.
Now we can either reboot or mount a hard drive. I choose to mount. Lets type:
pi@raspberrypi /mnt $ sudo mount /dev/sdb1
pi@raspberrypi /mnt $ mount
/dev/root on / type ext4 (rw,noatime,user_xattr,barrier=1,data=ordered)
# .....
/dev/sdb1 on /mnt/data type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other,blksize=4096)
And our drive is mounted and will be auto-mounted on each reboot.

3. Install samba

Now lets move on to installing samba server for our network sharing. Lets do it with apt-get:
pi@raspberrypi /mnt $ sudo apt-get install samba
Confirm additional package downloads by typing y to the prompt of this command. And wait while it installs some demons and defaults.
Now lets edit the default samba config:
pi@raspberrypi /mnt $ sudo nano /etc/samba/smb.conf
And add the config of our drive to the end of this config file. (After numerous other parameters):
[Data]
comment = DataDrive
read only = no
locking = no
path = /mnt/data     
guest ok = yes
force user = pi
After this is done simple restart of samba server will be sufficient:

pi@raspberrypi~$ sudo service samba restart

4. Enjoy

Now I have a file dump server I ever dreamed of. I can connect as guest with all my devices and have a cross platform decision built on a PI platform.
And this all stuff looks like this now. With a second ntfs external HDD and my Pi in Punnet MK1 paper case.

Nov 27, 2012

Raspbery Pi Time Capsule HDD managing power saving mode

I want to sugest power saving changes after installing and handling creation of Time Capsule for Time Machine for my home network (form my previous article). I like when it's quiet. And like when disks does not spin, when it's not necessary to do so. So if you usually backup once in some hours and main time (Night and a part of the day) you do not touch a Time Machine... You would probably want to slow down your disks for that time. Fortunately Linux does it very well. And support of it is encoded into Linux core (kernel). You just would need a UI for this possibilities and a bit of knowledge and luck ;)

Warning: Operations described farther may be potentially hazardous to your hardware.

So now after you are warned ;) let's continue. I guess I will not touch those hazardous functions, but you may, typing wrong letters to this command in Linux shell... So be careful. Better read manuals for farther commands and so on...

Anyway what we need is a UI. And we have it in form of nice utility called hdparm. It is not included in current Raspbian distribution (and it should not be perhaps) so you would probably need to install it by yourself. In fact you can find a package with it. But I'd recommend installing from latest version. It is http://hdparm.sourceforge.net/ situated. You can download it there. Using wget, for e.g. So move on to downloading and unpacking it. If you have a default Raspbian kernel, or did not screw up something useful while configuring and compiling your on one... You would probably simply enter a directory and type something like this:
pi@raspberrypi ~ $ wget http://switch.dl.sourceforge.net/project/hdparm/hdparm/hdparm-9.43.tar.gz
pi@raspberrypi ~ $ tar -zxvf hdparm-9.43.tar.gz
pi@raspberrypi ~ $ cd hdparm-9.43
pi@raspberrypi ~/hdparm-9.43 $ ./configure
pi@raspberrypi ~/hdparm-9.43 $ make
pi@raspberrypi ~/hdparm-9.43 $ sudo make install
pi@raspberrypi ~/hdparm-9.43 $ hdparm

hdparm - get/set hard disk parameters - version v9.43, by Mark Lord.

Usage:  hdparm  [options] [device ...]

Options:
...
Hurray! we have our desired utility. Reading through it's help you would probably find a useful for us paramether:  -S   Set standby (spindown) timeout
Here is our goal. I've set mine to 120 (10 minutes) by executing:
pi@raspberrypi ~ $ sudo hdparm -S 120 /dev/sda1

/dev/sda1:
 setting standby to 120 (10 minutes)
pi@raspberrypi ~ $
I have a WD "green" drive and will benefit from this. But you may harm your drive with often standbys and spinups. So choose wisely and read man hdparm for more ;).
Hope you would benefit from my finding. Please comment in either way.

UPD: to persist this upon boots see article:
http://garmoncheg.blogspot.com/2013/01/raspberry-pi-boot-applications-autorun.html

Time Capsule for $25

The real article name might be something like: Configuring Raspbery Pi to serve like a Time Capsule with Netatalk 3.0 for Mountain Lion.  But it's too long ;)

Here I will describe the process of using Raspberry Pi like a Time Machine in my network. To be able to backup your MAC's remotely (Like it would be NAS of some kind). It assumes you have a Raspberry Pi and have installed a Raspbian there and have a ssh connection, or somehow having access to it's console. Refer to my previous article for details.

Now that we have a Pi that is ready for action let's animate it. So to make it suit you as a Time Capsule (NAS) for your MAC's you need to do those basic steps:
- connect and configure USB hard drive(s)
- install support of HFS+ filesystem to be able to use MAC's native filesystem
- make mount (auto-mount on boot) of your hard drive
- install Avahi and Netatalk demons
- configure Netatalk daemon to make it all serve as a Time Machine
- configure avahi demon
- put avahi and netatalk demons into autolaunch on system boot
- ENJOY.

1. Setup a Hard drive.

I assume you know what you are doing and understand you have to have a hard drive formatted under your mac's HFS+ filesystem. That you, like me, used as an external hard drive for backups before this. Just for example. Or bought it... Whatever. One word you will need (or assume you have) an external USB drive to follow me farther.

Start your drive (in case it has external power). I think Pi would handle only drives with external power. But who knows ;). And connect it to your Pi. Let it spin for some seconds to be recognized.
Now connect to console of your Pi and let's roll.
leopard$ ssh pi@192.168.1.140
pi@192.168.1.140's password: 
Linux raspberrypi 3.2.27+ #250 PREEMPT Thu Oct 18 19:03:02 BST 2012 armv6l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Mon Oct 29 01:04:39 2012
pi@raspberrypi ~ $
As you can see I have a clean system installed from latest (as for 24.11.2012) downloaded image.

Let's do some magic. Because Raspbian does not support Apple's filesystem out of the box. At least we need to mount it write enabled. Which is not set up by default too. SO let's start from typing command blkid under root and looking at output.
pi@raspberrypi ~ $ sudo blkid
/dev/mmcblk0p1: SEC_TYPE="msdos" UUID="8B12-9112" TYPE="vfat" 
/dev/mmcblk0p2: UUID="AAAAAAA-1111-1111-......." TYPE="ext4" 
/dev/sda1: UUID="AAAAAAA-1111-1111-......." LABEL="Time Machine" TYPE="hfsplus" 
/dev/sda2: LABEL="Data" UUID="1234567890......." TYPE="ntfs" 
pi@raspberrypi ~ $
Note UUID of your device. Mine is called sda1 and has a partition type hfsplus.  Hurray! Drive is connected and is recognized by Pi.

2. Install support of AFP filesystem (MAC's native).

Now let's install tools we need for HFS+ filesystem mounting. We need  to mount it RW to be able to backup.
pi@raspberrypi ~ $ sudo apt-get install hfsplus hfsutils hfsprogs
Press "y" in the process of install. You will get HFS+ support libraries installed. So let's move on to mounting.
After this you should be able to mount your HDD. You may try it or skip to making permanent mounting config via /etc/fstab (Step 3).
pi@raspberrypi ~ $ sudo mount -o force /dev/sda1 /mnt/TimeMachine
pi@raspberrypi ~ $ cd /mnt/TimeMachine
pi@raspberrypi /mnt/TimeMachine $ ls -l
total 116
drwxr-xr-x 1 root      99      6 Jan 11  2012 Backups.backupdb
drwxrwxrwx 1  501 dialout     31 Feb  1  2012 !DVD-s
-rwxr-xr-x 1 root root    115716 Sep  1 07:23 tmbootpicker.efi
pi@raspberrypi /mnt/TimeMachine $
You may see your files from a HDD. Hurray! Again ;). Btw you can look at how is your device was mounted by typing mount command into prompt. It will show your mounted volume and mode it was mounted. So you will see something like this:
pi@raspberrypi /mnt/TimeMachine $ mount
/dev/root on / type ext4 (rw,noatime,user_xattr,barrier=1,data=ordered)
devtmpfs on /dev type devtmpfs (rw,relatime,size=118872k,nr_inodes=29718,mode=755)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=23788k,mode=755)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=47560k)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620)
/dev/mmcblk0p1 on /boot type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=cp437,iocharset=ascii,shortname=mixed,errors=remount-ro)
/dev/sda1 on /mnt/TimeMachine type hfsplus (rw,relatime,umask=22,uid=0,gid=0,nls=utf8)
Note last line with /dev/sda1 mounted in mode "rw". Yes everything works well. Let's move on.

3. Make mount (auto-mount on boot) of your hard drive

Let's create a directory for our new tome. We will need this to mount filesystem.
pi@raspberrypi ~ $ sudo mkdir /mnt/TimeMachine
Lets unmount our tome (in case you did tested it exists with me).
pi@raspberrypi ~ $ sudo umount /dev/sda1
And add a proper fstab entry to auto-mount this hard drive on boot. Open fstab:
pi@raspberrypi ~ $ sudo nano /etc/fstab
And add a line to the end of the file, indicating our Time Machine tome mounting preferences. Mine was:
UUID="AAAA-BBBB..." /mnt/TimeMachine hfsplus rw,force,exec,auto,users 0 3
Where AAAA-BBBB...  is your devise UUID from that blkid command remember?
I used UUID to be sure it won't matter if I'll switch hard drives places in USB ports. But you may use /dev/sdXX, where XX is your device counting numbers in /dev. Mine in this case was /dev/sda1.

Note: You can skip reboot.

Let's now do a reboot to make sure it all spins in automatic mode ;).
pi@raspberrypi ~ $ sudo reboot

Broadcast message from root@raspberrypi (pts/0) (Wed Nov 21 19:13:49 2012):
The system is going down for reboot NOW!
After system boots up run mount command to make sure we are ok in rw mode. Btw you may play with commands:
# to unmount volume:
pi@raspberrypi ~ $ sudo umount /dev/sda1
# to mount again:
pi@raspberrypi ~ $ sudo mount UUID="AAAA-BBBB..."
While you will find your string. BTW you can read about how and why in /etc/fstab manual here... Or google about it.

Note: if your HFS+ tome does not mount and/or mounted read only. You should try to run fsck for hfsplus partition type manually.
pi@raspberrypi ~ $ sudo fsck.hfsplus -f /dev/sda1
Anyway we must now have an rw mode HFS+ volume mounted and ready for our Time Machine. So Lets move on...

Checking your user permissions. As we will use user "pi" farther along the article we must add him ability to manage Time Machine data. (In case you are not mounting empty drive). You should consider changing all the data owner to user "pi" to make sure we will be abler to backup in future:
pi@raspberrypi ~ $ sudo chown -R pi /mnt/TimeMachine

4. Install and configure Avahi and Netatalk demons

First make sure you have the latest packages lists, running:
pi@raspberrypi ~ $ sudo apt-get update
And updating what necessary.

Now install all the required packages with db and encryption support in order Avahi and Netatalk demons to support for HFS+ filesystem:
pi@raspberrypi ~ $ sudo apt-get install avahi-daemon libavahi-client-dev libdb5.3-dev db-util db5.3-util libgcrypt11 libgcrypt11-dev
After this is complete we will have almost everything required for our little demon.

Download and unpack 3.0.0 (latest, as for now is 3.0.1 but I had problems with it spinnig) Netatalk demon sources. For e.g. using command: $ wget http://somesite.com/photos.zip You may get Netatalk at SourceForge. And unpack using e.g. tar: tar -xvf netatalk-3.0.0.tar.bz2

After you are ready with unpacked netatalk distribution go ahead and make custom configuration of it. Enter directory of unarchived netatalk and execute command:
pi@raspberrypi ~/netatalk-3.0 $ ./configure --with-init-style=debian --with-zeroconf
It will do the proper configuration for our needs. After this is done run:
pi@raspberrypi ~/netatalk-3.0 $ make
pi@raspberrypi ~/netatalk-3.0 $ sudo make install

5. Configure Netatalk daemon to make it all serve as a Time Machine

Now when you will have all this done go edit a configuration file of Netatalk. From version 3.0 it is located in /usr/local/etc and has Samba-like look. You can symlink this file to /etc/afp.conf if you wish... 
pi@raspberrypi ~ $ sudo nano /usr/local/etc/afp.conf
And insert this sample configuration. This config file works with latest edition ( now it is 10.8.2 ) of OS X Mountain Lion. So the config:
;
; Netatalk 3.x configuration file
;

[Global]
; Global server settings
; Name of your computer in apple devices network
hostname = TimeMachine Pi
; IP of your Pi
afp listen = 192.168.1.140
; logging config
log file = /var/log/netatalk.log
log level = default:info

[Homes]
basedir regex = /home
cnid scheme = dbd
; Display each user home directory in this format
home name = Home: $u
[Time Machine]
; Our Time Machine volume
path = /mnt/TimeMachine
cnid scheme = dbd
file perm = 0660
directory perm = 0770
time machine = yes

; Example to add a new static share:
; [My AFP Volume]
; path = /path/to/volume
It is quite well commented so you will be able to copy-paste and edit ;).

6. Configuring AVAHI services 

Now you would probably want to have a Time Capsule as a separate share in your network shares. Note this step is optional. But if you plan to use your pi for more than 1 thing, like I do... I'd recommend to do it. Create a special afpd services configuration file by executing:
pi@raspberrypi ~ $ sudo nano /etc/avahi/services/timecapsule_afpd.service
Then add this contents to file:
<?xml version="1.0" standalone='no'?>
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
      <name replace-wildcards="yes">TimeCapsule %h</name>
      <service>
          <type>_afpovertcp._tcp</type>
          <port>548</port>
      </service>
      <service>
          <type>_device-info._tcp</type>
          <port>0</port>
          <txt-record>model=TimeCapsule</txt-record>
      </service>
</service-group>
After this is configured for your desired settings. Go run your demons:
pi@raspberrypi ~ $ sudo /etc/init.d/netatalk start
pi@raspberrypi ~ $ sudo /etc/init.d/avahi-daemon start
Avahi demon was installed from package. So you might need to actually run restart instead of start command. Now go check your Mac's Network Shares.

7. Put avahi and netatalk demons into autolaunch on system boot

We have installed here an avahi demon from package. So it must be written to all system run-levels during installation. Only thing here is compiled Netatalk 3.0 so you can make a proper trick here by putting it to run with default init.d command:
pi@raspberrypi ~ $ cd /etc/init.d/
pi@raspberrypi ~ $ sudo update-rc.d netatalk defaults
update-rc.d: using dependency based boot sequencing
update-rc.d: warning: default stop runlevel arguments (0 1 6) do not match netatalk Default-Stop values (1)
pi@raspberrypi /etc/init.d $
You can read about system runlevels and why this command and not another in this article:
http://www.debuntu.org/how-to-manage-services-with-update-rc.d I find it nice simple and easy.
You can also read about Time Capsule theory and find answers to many "why?" questions here: http://buffalo.nas-central.org/wiki/Time_Machine_&_Time_Capsule_support_on_your_LinkStation

Result:

TADAM. The moment of truth. You can look into your Network Shares on a MAC. Select a virtual Time Capsule and hit "Connect as..." button. Login to your raspberry with user pi and saving password. Enter Time Machine directory... And Time Machine share will appear on a desktop (If it is configured to appear for newly mounted volumes...). You will experience something like this:
After all of this is done. Go enter your Time Machine settings and add a new drive. Bckuping on a time capsule is some kind of different to backing up into external (e.g. USB) hard disk drives. Using external HDD as a Time Machine backup disk will store folder Backups.backupdb into the root of your drive. But backup into time capsule (network backup) creates a volume like leopard.sparsebundle, where this leopard is your MAC's name. And connects it as a file system image, storing this folder inside. So to make initial backup connecting through USB and then continue making backups is possible but irrelevant. Because you would have to mount this image and change permission of all files. Then move them into mounted virtual image under Pi or over network. So the advantages of this idea are controversial. 

It was looking like so for me:
And my pi, serving like a Time Capsule and external HDD:

Please comment your thoughts about all of this. Every comment is appreciated. Thanks. And hope someone would benefit from those my findings.

Nov 19, 2012

Raspberry Pi first steps and basic network configuration on a Mac

Here are my first steps. And I hope you will find something useful here, while configuring your Pi...
First of all. Mine have been bought on ebay, from resellers. And were delivered a while ago. Main purpose of this purchase war to attach a headless server to my router. I have 2 external HDD's and would like to have torrents, Time Mashine fro my macs and so on. So buying a handheld computer like this would be a bargain for me.

As for built in decisions like some kinds of NAT devices and different routers with external HDD features... They are either cost a lot or lack some kinds of desired functionality. So the goal is to make some kind http/api manageable server in my local network with Time Mashine and file storage/backup. Just for fun. And to have only laptop on my work table.

SO back to the Pi. If you are buying "device only" configuration, like I did. First of all you'll need some different kind of things many computer fans usually have for backup purposes, or just in case... Those are device with HDMI port or either a TV set with AV input, e.g. SCART or Video in port. A keyboard (USB, of course) and SD card usually bigger then 2 Gigabytes.

I had this all. So I've connected my PI and tried to flash a card, having problems doing it. I'll try to cover most of them here.

1. Install a system. Usually it is a simple "burning an SD card" with Raspbian Linux distribution. To do this you need to download it from: HERE and do actions from official manual here with looking further note first.

Problem: "dd" does not work like intended, because volume is bisy.
Solution: We need to unmount volume, using diskutil command with "force" flag for e.g.:
sudo diskutil umount force /Volumes/my_flash_drive

Problem: Doing "dd" command under mac environment did write the card, but took ages on my mac's config. So the...
Solution: Is modified dd command call with manually specifying block size. This will speed up sd card write tremendously.
dd bs=1m if=~/Downloads/debian6-19-04-2012/debian6-19-04-2012.img of=/dev/rdisk1
Note "bs=1m" here. It helped my SD card burning not to hang out to infinite time.

After SD card was written and you are good to go...

2. Pi initial config. Appears after first boot and looks like so:
You can run it afterwards using command "sudo raspi-config". Just in case you would missed something hitting Esc or similar. It will be available via ssh in future also.

Tools you will use here are similar to bios config on a PC. So to get started:

Here you need to expand rootfs (to use entire SD card space) I doubt you have original Pi Raspbian card. But you may skip this step in case you do. This will be executed on next start. So feel free to do it first thing.

Next important option is change pass and you need it no-mater what. It will change your default user's with username pi password. It is similar to user with root permissions... So be sure to do it. Note password symbols are not displayed during entry.

change timezone goes next. If you do not leave in other timezone, like I do, change it. ;) Note pi does not have an internal "bios" clock so you would need to do it. Because time is used from internet atomic clocks...

memory split is also important option that help you to manage precious pi's RAM. I've used 16 because I will not need GUI and will run "headless", using it via ssh/network services. To use GUI consider 32+ Mb or RAM for video memory. But note RAM is really in lack for most modern services. and you may need every Kb of it in future.

ssh is our main goal. To access it/configure it in future. Hit it to enable ssh connections.

and you can run update if you have already connected network somehow :).

That's it with basic config for our needs. Here goes...

3. Static network configuration.
I think this is the most important part. Because I have googled a lot to find a proper solution for my needs.

Pi is connected via dhcp IP address, usually given randomly by your router. And you need to make it static to know you will always have pi under certain network address.

You may know your Pi's ip address by entring route command into prompt. Yo will see something like this.

pi@raspberrypi ~ $ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.1.1     0.0.0.0         UG    0      0        0 eth0
192.168.1.0     *               255.255.255.0   U     0      0        0 eth0


Here you can see your network is in range 192.168.1.x. Biut if you are configuring your Pi without actual network config, like I did, you will need to do something like this:

You need to know it from your router config. And you can simply connect it via network, if you did it properly in past. Or either connect, like you did for initial router config. You need something like this, saying about your LAN network address. I have a wired mix of different network decisions in my router. So note your typical network router address here:
Here you can know your router's DHCP address. And usually you can do it by examining your router's user manual.
Anyway this tells me that I have a wire (LAN) IP network with adress range 192.168.1.x where x is between 1 and 254. Your DHCP in router may work in another way and have different interface. So you may have other IP range set. E.g. Mine had a 100-199 range by default:
I've chosen a 192.168.1.140 IP  for my PI, because I would not have 40 devices in my network and would not have any possible IP conflicts this way.

This are data required for our Pi network config. We need to manually change this in our Pi's config files.

Try to do this:

cd /etc/network
sudo nano interfaces

This will start default Pi's text editor, called "nano", pointing to networks configuration file. Here is how it might look like:
You can notice nice hotkeys here in the bottom. So I think you will be pleased with this user interface. Let's replace the line iface eth0 inet dhcp with lines.

iface eth0 inet static
address 192.168.1.140
netmask 255.255.255.0
gateway 192.168.1.1

This will create a static address for our Pi. Now to make sure we have a proper DNS. Go to editing /etc/resolv.conf. E.g. type:

sudo nano /etc/resolv.conf

Mine had a a line:

nameserver 8.8.8.8

Pointing to google's DNS. And I've left it, adding my router as a first DNS. So my resolv.conf became looking like:

nameserver 192.168.1.1
nameserver 8.8.8.8

And that's it. After doing this you would have to restart network interfaces. Or better do "sudo reboot" to make sure everything is configured. At this point you are OK to go running headless already. You can now do ssh connections to static ip at your pi.

So to use it you would need only to type in your terminal:

ssh pi@192.168.1.140

This will connect you to your pi. And you are ready to go ahead.
you may see something like this:

And to finish here is how my Pi looks like in a "printable case" The Punnet from user "E". Thanks to him.

Nov 3, 2012

Migration from HDD to SSD with Mac OS X Lion

Today I'll try to cover my migration experience. I have bought myself an OCZ Technology 128GB Vertex 4 SSD from Amazon. And I also have an external 1 Tb HDD. I'm using it for migration purposes as a backup Time Mashine. So to migrate yourself without reinstalling operating system and so on you need:

1. Compact your data. Usually HDD's, installed in Mac's, are much bigger then SSD's you buying. And to tell you the truth, you probably do not need a 512 Gigabytes SSD at all. Most of data are accessed rarely and do not require speedy storage. So COMPACT them. Move to an external HDD and/or delete. Time to cleanup one word. There are also several places that can be easily deleted without much troubles. They are iTunes backup caches for your iOS devices, like iPhones and iPads.
they are usually sotred on your user library path.
You can delete them without sorry from either:
~/Library/Application Support/MobileSync/Backup 
folder or in your iTunes => Preferences => Devices section, hitting delete on each backup.
They will be created from scratch on next sync. So no worries about them. Your data are safe on your mobile devices. If you do not screw things up on next sync (:
Besides this you have ability to link files. For e.g. I have an iTunes library of around 100 Gigs of songs, books and etc. So storing it by default path is bad idea. My SSD only has 128 Gigabytes. Anyway I'll tell what to do with this later on. For now I'll just say you can move iTunes Libraries and relink them. And you can move iPhoto library. I have around 100 Gigs of photos too... So those 2 are moved to external HDD. Leaving me 200 gigabytes of data. After deleting iTunes backups I've fried around 60+ gigs. I have had several iPhones and their backups also were stored there. So the list was big.
Anyway. The file system started to look like this:

But I've fried this space, and made a backup. Why the hell I have no free space. And my Mac says I have only 70+ gigs of space used. The catch is in .MobileBackups folder. Mac stores your Time Mashine backups to local hard drive. Backing up data locally. What a mess :). You can disable it by hitting terminal command:
$ sudo tmutil disablelocal 
this will disable local Time Mashine backups. TADAAM. We really have a hard drive with around 78 gigs of space used for system and files.
Now make a Time Mashine backup to external HDD, or second hdd, that will not be formatted and can be accessed during setup. Math is simple. Last backup must be smaller then your SSD capacity. 78 is quite ok for me.
In case you need to fit something on your HDD and have all the space used by Time Mashine backups, like I did, you can delete old Time Mashine backups in manual mode. Simply go to folder on your Time Mashine drive, called Backups.backupd => YourUserNameHere. You will see a set of date stamped folders. Those are Time Mashine backup dates.
You can free them up by deleting each folder and cleaning the trash. Note it will come to cleaning Trash with - amount of objects, e.g.: "deleting file -123". Don't bother with that and simply wait for cleanup.

Now that we have a most recent backup in Time Mashine of all our system HDD. We can proceed. Power of your Mac/Macbook and insert a SSD drive, if you have not done so before.
I have used an Second SSD/HDD drive Caddy that is around 10$ at eBay.
Looks something like this:
I've plugged my HDD there. And SSD came in place of HDD. But it's a metter of taste and you can do as you wish here. 
Now time to boot. Boot, holding "Option" key. You will see the menu with OS boot choice, Looking something like that:
Choose Recovery HD. This will be mush simpler/quicker then making any external boots. But you may have to do it in case you do not have this option here. Anyway booting this option will lead you to a screen with recover options.
Choose Restore From Time Mashine Backup. Will lead you to the drive with your Time Mashine selection, in case you have several. Afterwards select your LATEST! backup. And choose where to restore to. Be careful to choose your SSD. And hit Fire. :)
You will wait several minutes, depending on your Time Mashine drive speed with this screen:
After load you will have fully restored system with all your data on a SSD.
Now be sure you have boot from an SSD ( It is usually hard to make a mush here because of boot speed :D ). But make sure you are on SSD now and go ahead to diskutil and format your HDD. You will not need it anymore. TADAM. You're done.

Thanks for reading this far and comments with your success are welcome, as long as critics apriciated ;).

Sep 25, 2012

Django Log Files Viewer documents

Django Log File Viewer.

This is a PYPI package django-log-file-viewer documents.
Github repo: django-log-file-viewer@garmoncheg.github.com

Usage:

Useful to add log files view functionality to your Django admin web site.
Instead of using database log files storage, it gives you ability to store/view log files through GUI.
It requires a directory with Django log files to function. E.g. directory structure:


$ project_dir/logs/:
       applog.log
       applog.log.2012-09-22
       ...
       errors.log
       applog.log.2012-09-22
       ...


Screenshots:


To parse/display these log files you need:

1. Install an app and add it to your settings.py INSTALLED_APPS section:


# settings.py
INSTALLED_APPS = (
    # ...
    'django-log-file-viewer',
    # ...
)

2. Set UP 2 django variables in settings.py:


# settings.py:
LOG_FILES_DIR = '/path/to/your/log/directory'
    # Relative or static path string of your log files directory.
    # I recommend using more pythonic way of defining tis with os module. E.g. :
    # settings.py:
    #
    # LOG_FILES_DIR = os.path.join(APP_PATH, 'testdata', 'log')
    # 
    # where APP_PATH is your app's/project's path.



LOG_FILES_RE = '(?P<date>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3})\s\[(?P<type>[A-Z]+)\]\s(?P<message>.+)'
    # Is a regex to parse your log file against. It completely depends of your Django logging settings.
    # And table column names (in a parsed logfile) depend from group names you provide in the regexp.
    # E.g. for Django logging server to parse with this regexp you need to have log, as in example
    # django_log_file_viewer/testdata/testing.log file.

    # to produce this log I've added this formatter to my website.
    
    #'formatters': {
    #    'verbose': {
    #        'format': '%(asctime)s [%(levelname)s] %(message)s'
    #    },
    #},

3. And add urls to your main urls section:


# urls.py
urlpatterns = patterns('',

    # Include this before admin to enable app admin url overrides
    # Note url must be the same as admin
    # This is required step
    url(r'^admin/', include('django-log-file-viewer.admin_urls')),
    url(r'^admin/', include(admin.site.urls)),

    # To view with custom views:
    # Optional step
    # Will ad urls like www.example.com/logfiles/
    url(r'', include('django-log-file-viewer.urls')),
)

TODO's:

  • Add pagination to both log files list and log file content

Reference:


Sep 19, 2012

SSH Unix/Lunux Recipes for daily usage.

I've decided to create a post with several recipes for commands I need to use daily. I use them from a Mac OS X Lion default console, except for some mac ports installed... Remote is special linux distribution.

How do I Compress a Whole Linux or UNIX Directory? 

You need to use tar command as follows (syntax of tar command):
tar -zcvf archive-name.tar.gz directory-name
Where,
  • -z: Compress archive using gzip program
  • -c: Create archive
  • -v: Verbose i.e display progress while creating archive
  • -f: Archive File name
For example, you have directory called /home/garmoncheg/data and you would like to compress this directory then you can type tar command as follows:
$ tar -zcvf data.tar.gz /home/garmoncheg/data/
Above command will create an archive file called data.tar.gz in current directory. If you wish to restore your archive then you need to use following command (it will extract all files in current directory):
$ tar -zxvf data.tar.gz
Where,
  • -x: Extract files

Put or Retrieve a file from/to remote directory via SSH

For all those operations you need to use scp command. It is like a cp command under local unix/linux console. Except it does remote transfers.

So to put one file:
scp file.dat garmoncheg.google.com:NewData.txt
copy file "file.dat'' from your current computer/directory to your home directory on garmoncheg.google.com, naming the copy on remote server "NewData.txt''.  
scp file.dat garmoncheg.google.com:
is the same thing, except file will be named like original at local computer HD.
scp file.dat garmoncheg.google.com:/home/garmoncheg/data.txt
with specifying original path to store files at.

To get files you need to reverse paths given via arguments to scp. E.g.:
scp garmoncheg.google.com:data.txt file.txt
This will copy file from your default ssh directory named "data.txt" to local (current directory) file called "file.txt"

To work with directory (get/put files) recursively there is an -r switch:
scp -r project-stage:logfiles /home/garmoncheg/logfiles
This will download directory "logfiles" from server with ssh shortcut "project-stage" into current directory naming it "logfiles"

That's it. There are many more decisions and recipes for those typical enough operations. Dont hesitate to comment with yours ;)

Aug 6, 2012

Python: simple recipe to measure your function's execution time

We always write something unusual while doing basic things. Our own bicycles and crutches to work out some unusual situation. Here is another recipe to do a thing like so. I have tried several libraries and readymade decisions. But it assumes you have them installed. And you often try things in console, don't you?

Anyway the recipe is simple and quite straightforward.

import datetime

# Getting first timestamp
t1 = datetime.datetime.now()
# Your function e.g.:
data = [g.name for g in request.user.groups.all()]
# Second timestamp
t2 = datetime.datetime.now()

print "Execution time: %s" % (t1-t2)

It is rude and quite simple but may often suit you well to measure execution time in a simple and straightforward manner.
Also nice idea to write down this function into logs. It may be handy on refactoring of your core app for e.g.;

import datetime
import logging
log = logging.getLogger('mylogger')

# Getting first timestamp
t1 = datetime.datetime.now()
# Your function e.g.:
data = [g.name for g in request.user.groups.all()]
# Second timestamp
t2 = datetime.datetime.now()
# Using Logging to write down calls
log.debug("my_function Execution time: %s" % (t1-t2))

This also can grow up to Django decorator that can log time of function execution into console/log file/whatever. But it's quite another story.
Those snippets are quite Django based. But, I think, the core is pure python, if you've got the core concept of this.

Used, have thoughts, know how to do it in a better/simpler manner? I'm wrong? Please comment!