Made by Nathan

programming and projects

Scheduling One-off Tasks With ‘At’

| Comments

You might be familiar with the cron job scheduler, which is great for repeating tasks. But when you want to schedule a command to only run once in the future, the at command is what you are looking for.

In my case, I was updating a plugin for our Thoughtworks Mingle instance, but the update wasn’t hugely important. Many of our staff rely on Mingle for their work, and restarting it takes it offline for a few minutes.

So I used the at command to schedule the restart to happen at midnight, after everyone had gone home:

$ echo "/etc/init.d/mingle restart" | at -m 00:00
job 6 at 2012-02-26 00:00

Use atq or at -l to see the list of pending jobs:

$ atq
6   2012-02-26 00:00 a root

Use at -c <job id> to view the script that will be run:

$ at -c 6

#!/bin/sh
# atrun uid=0 gid=0
# mail     root 1
umask 22
HOSTNAME=...
<lots of environment variables set here>

/etc/init.d/mingle restart

To delete a scheduled task, run at -d <job id>:

$ at -d 6
$ atq
(no output)

Travis CI Status in Shell Prompt

| Comments

Update:

I’ve updated the build status checking script, because updating all branches was taking too long (over 20 minutes.) Now it performs frequent updates for the current branch, and only periodic updates for all branches.


Travis CI is a distributed continuous integration service for the open source community, and it can be used with any of your public projects on GitHub. You’ve probably seen some ‘build status’ badges like this:

Travis CI build status

Wouldn’t it be cool if you could see that build status in your shell prompt whenever you are working on a project?

Here’s what my shell prompt looks like now:

Travis CI status in prompt

This shows the build status for the current branch.


You will need to cache the build status, since looking it up takes a few seconds.

You should use my fork of mislav’s travis-ci script, which can check the build status of a project. Make sure ~/bin is in your PATH, and if you are using RVM, make sure you are using your default ruby.

Run the following to install it:

mkdir -p ~/bin/
curl -sL https://raw.github.com/gist/1708408/travis.rb > ~/bin/travis-ci \
  && chmod +x ~/bin/travis-ci

gem install hub | tail -2
ruby -e 'require "json"' 2>/dev/null || gem install json

Next, we need to update the cached status. The following code is included as part of my SCM Breeze project, but feel free to save the update_travis_ci_status script at the bottom of this post [1] to /usr/bin/update_travis_ci_status.

We will also need a way to run this update task every few minutes, across all of our local git repos. We only want to frequently update the status for the currently checked out branch, and periodically update the status for all branches.

The SCM Breeze project also maintains an index of your git repositories, which gives you the ability to run batch commands via the git_index function. So the build status update can be easily set up as a cron task:

*/5 * * * * /bin/bash -c '. $HOME/.bashrc && git_index --rebuild && git_index --batch-cmd update_travis_ci_status'
*/45 * * * * /bin/bash -c '. $HOME/.bashrc && git_index --rebuild && export UPDATE_ALL_BRANCHES=true && git_index --batch-cmd update_travis_ci_status'

Alternatively, you could save the following script to /usr/bin/update_all_travis_ci_statuses.

#!/bin/bash
# (Replace `$HOME/code` with the location of your projects)
for f in find "$HOME/code" -maxdepth 4 -name .travis.yml; do
  (builtin cd "$(dirname $f)" && update_travis_ci_status)
done

… and use the following cron task:

*/5 * * * * /bin/bash -c '. $HOME/.bashrc && /usr/bin/update_all_travis_ci_statuses'
*/45 * * * * /bin/bash -c '. $HOME/.bashrc && export UPDATE_ALL_BRANCHES=true && /usr/bin/update_all_travis_ci_statuses'

(you need to source your .bashrc if your default ruby comes from RVM)

Finally, you need a way to display the cached status in your prompt.

Here are the functions I use to return a colored pass / fail / running symbol:

# Returns the Travis CI status for a github project
parse_travis_status() {
  local branch="$1"
  if [ -z "$branch" ]; then branch="master"; fi

  local stat_file=$(find_in_cwd_or_parent ".travis_status~")
  if [ -e "$stat_file" ]; then
    case "$(grep -m 1 "^$branch " "$stat_file")" in
    *passed)  echo "\[\e[01;32m\]✔ ";; # green
    *failed)  echo "\[\e[01;31m\]✘ ";; # red
    *running) echo "\[\e[01;33m\]⁇ ";; # yellow
    esac
  fi
}

# Test whether file exists in current or parent directories
find_in_cwd_or_parent() {
  local slashes=${PWD//[^\/]/}; local directory=$PWD;
  for (( n=${#slashes}; n>0; --n )); do
    test -e $directory/$1 && echo "$directory/$1" && return 0
    directory=$directory/..
  done
  return 1
}

(it also works if you are in a project’s sub-directory.)

Finally, add $(parse_travis_status "$current_branch") somewhere in your $PS1. You should set the $current_branch variable to the current git branch, but it defaults to the master branch if you leave it blank.

You may like to have a look at the prompt section of my dotfiles, to see how I do it.

Enjoy! Please let me know if you have any questions, or need some help.


[1] update_travis_ci_status script:

#!/bin/bash
if [ -e ".travis.yml" ]; then
  if type ruby > /dev/null 2>&1 && type travis-ci > /dev/null 2>&1; then
    local stat_file=".travis_status~"
    local tmp_stat_file="$stat_file"".tmp"

    # Either update all branches, or only current branch
    if [ "$UPDATE_ALL_BRANCHES" = "true" ]; then
      local all_branches=$(\git branch -a)
      # All branches on origin remote that have local copies
      local branches=$(comm -12 <(echo "$all_branches" | \
                                  sed "s/ *remotes\/origin\///;tm;d;:m;/^HEAD/d;" | sort) \
                                <(echo "$all_branches" | \
                                  sed "/ *remotes\//d;s/^[\* ]*//" | sort))
      # Create a new, blank temp file
      echo -n > "$tmp_stat_file"
    else
      # Only current branch
      local branches="$(\git branch 2> /dev/null | sed "s/^\* \([^ ]*\)/\1/;tm;d;:m")"
      # Copy current file to temp file
      touch "$stat_file"
      cp -f "$stat_file" "$tmp_stat_file"
    fi

    for branch in $branches; do
      local travis_output=$(travis-ci "$branch" 2>&1)
      local status=""
      case "$travis_output" in
      *built\ OK*)    status="passed";;
      *failed*)       status="failed";;
      *in\ progress*) status="running";;
      esac

      # If branch has a build status
      if [ -n "$status" ]; then
        if grep -q "^$branch" "$tmp_stat_file"; then
          # Replace branch's build status
          sed -e "s/^$branch .*/$branch $status/" -i "$tmp_stat_file"
        else
          # Append new line for branch
          echo "$branch $status" >> "$tmp_stat_file"
        fi
      fi
    done

    # Replace current stat file with finished update
    mv -f "$tmp_stat_file" "$stat_file"
    # Ignore status file from git repo
    if ! ([ -e .git/info/exclude ] && grep -q "$stat_file" .git/info/exclude); then
      echo "$stat_file" >> .git/info/exclude
    fi
  fi
fi

Testing Multiple Databases for a Rails App on Travis CI

| Comments

I’m currently doing a lot of work on an open source Ruby on Rails project called Fat Free CRM. The code is hosted on github, and we are using the amazing continuous integration service provided by Travis CI.

Find out more about Travis CI here.

We’ve been working on some powerful features for Fat Free CRM, such as dynamic custom fields, and we wanted to make sure that they work across all of our supported databases. So here’s how I set up our Travis CI build matrix to test multiple databases, with some help from the Travis docs:

.travis.yml

We add the databases to our build matrix by setting ENV variables. Add the following lines to your .travis.yml:

env:
  - DB=mysql
  - DB=postgres
  - DB=sqlite  

Database configuration

We package multiple example database configurations for each of our supported databases, like this:

  • config/database.mysql.yml
  • config/database.postgres.yml
  • config/database.sqlite.yml

We also have a rake task that is a prequisite for the spec task, and this sets up the example configuration files for Travis.

It copies the database.yml template specified by our DB variable, using postgres as the default.

FileUtils.cp "config/database.#{ENV['DB'] || 'postgres'}.yml", 'config/database.yml'

Gemfile.ci

I created a new Gemfile for CI. It simply tells bundler to use the gem specified by our DB variable, prevents any other database gems from being loaded, and then loads the ‘real’ Gemfile.

Here’s the contents of Gemfile.ci:

case ENV['DB']
when "mysql"; gem "mysql2", "0.3.10"
when "sqlite"; gem "sqlite3" 
when "postgres"; gem "pg", ">= 0.9.0"
end

def gem(*args)
  # Override 'gem' method to block any other database gems in the 'real' Gemfile
  super unless %w(pg sqlite3 mysql2).include?(args.first)
end

# Eval Gemfile
eval(IO.read(File.join(File.dirname(__FILE__), 'Gemfile')), binding) 

That’s all there is to it.


This is a slightly unrelated topic, but I had a lot of trouble getting our new Gemfile.ci to work properly. After a serious headache, I figured out that we hadn’t updated config/boot.rb to the latest version after our upgrade. This new boot.rb had a very subtle difference, and contained the following line:

ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../../Gemfile', __FILE__)

instead of:

ENV['BUNDLE_GEMFILE'] = gemfile

Notice the ||=, which meant that the BUNDLE_GEMFILE variable could actually have an effect when it was set by Travis.

You might have found this post if you are googling for Could not find multi_json-1.0.3 in any of the sources, which is the symptom that I was experiencing (due to an updated gem and an outdated Gemfile.lock). In that case, you may need to update your config/boot.rb to the latest version from Rails.

Managing Project Design Assets for Git Repositories: A Simple Solution

| Comments

When you’re creating logos or icons for a project that uses git, have you ever wondered where you should store those .psd or .xcf files? Do you commit all of your raw design files, or does it put you off that any changes to those files will bloat your repository?

The following post describes a part of my SCM Breeze project. If you haven’t seen it already, take a look at my blog post that describes what it can do.

Here were my goals when I set out to find a solution:

  • I wanted a design directory for each of my projects
  • I didn’t want the design directory to be checked in to the git repository
  • The design directory needed to be synchronized across all of my machines

I decided that I would need to store all of my design files in one place so that they could be easily synchronized, and I would create symlinks from each project to the root design directory.

The simplest way for me to synchronize files was via my Dropbox account. However, if you work with a larger team, you could set up a shared design directory on one of your servers and synchronize it with rsync.

Either way, here’s how you can effectively manage your design assets for git projects:

1) Install SCM Breeze

This gives you the design() function, as well as a bunch of other features that you can read about here.

To install, run:

git clone git://github.com/ndbroadbent/scm_breeze.git ~/.scm_breeze
~/.scm_breeze/install.sh
source ~/.bashrc   # or source ~/.zshrc

2) Create and configure a root design directory

I created my root design directory at ~/Dropbox/Design.

After you’ve created your root design directory, edit ~/.scmbrc and set root_design_dir to the directory you just created. You can also configure the design directory that’s created in each of your projects (default: design_assets), as well as the subdirectories you would like to use. The default base subdirectories are: Images, Backgrounds, Logos, Icons, Mockups, and Screenshots.

After you have changed these settings, remember to run source ~/.bashrc or source ~/.zshrc.

3) Initialize design directories for your projects

To set up the design directories and symlinks, go to a project’s directory and run:

design init

If your root directory is ~/Dropbox/Design, directories will be created at ~/Dropbox/Design/projects/my_project/Backgrounds, ~/Dropbox/Design/projects/my_project/Icons, etc.

It will then symlink the project from your root design directory into your project’s design directory, so you end up with:

  • my_project/design_assets –> ~/Dropbox/Design/projects/my_project

It also adds this directory to .git/info/exclude so that git ignores it.

Here’s the awesome part: If you use the SCM Breeze git repository index, you can run the following batch command to set up these directories for all of your git repos at once:

git_index --batch-cmd design init

If you want to remove any empty design directories, run:

design trim

And if you want to remove all of a project’s design directories, even if they contain files:

design rm

4) Link existing design directories into your projects

If you’ve set up your design directories on one machine, you’ll want them to be synchronized across all of your other development machines.

Just run the following command on your other machines after following steps 1 and 2:

design link

This uses your git index (from SCM Breeze) to figure out where to create the symlinks. If you don’t use the git index, the same outcome could be achieved by running ‘design init’ for each of the projects.

Enjoy!

Please leave a comment on this blog post if you have any questions. If you find a bug, or it doesn’t work quite right on your operating system, please raise an issue on Github.

Shameless Dropbox Referral

If you don’t already use Dropbox, click here to sign up! It’s an awesome service! And if you sign up via that referral link, you’ll be giving me a little extra free space :)

10 Reasons Why Hackers Should Jailbreak Their iPhones

| Comments

There was a post on the front page of Hacker News recently, titled “Making Chrome better on iOS”. Google’s Chrome browser is now available on iOS, and the author of the article was talking about how app developers could be persuaded to add some code to their apps which would allow Chrome to open links.

Restrictions like this (not being able to change your default browser) are one of the many reasons why I wouldn’t buy another iPhone if it couldn’t be jailbroken. It also seems strange that ‘Hacker News’ readers don’t appear to be very excited about ‘hacking’ the incredible device in their pockets.

We don’t need to ask developers to support Chrome when we can just change the default browser ourselves:

Changing Default Browser

You can find the “Browser Changer” package on Cydia.

Here’s 9 other reasons why a jailbroken iPhone is better than any other mobile device:


Grooveshark

The Grooveshark iPhone app was removed from the App Store, so you can only install it on a jailbroken iPhone through Cydia.

Grooveshark

The ‘Grooveshark Anywhere’ plan is required if you want to download unlimited albums and songs to play offline, but it’s well worth the $9 per month. It’s definitely a replacement for iTunes, so it’s no wonder that Apple feels threatened.

Note: I also tried Spotify for a week when it was released to New Zealand, but I found that Grooveshark has a much bigger music collection.


SSH

Having SSH access to your phone opens up a lot of possibilities. For example, I use capistrano to deploy iPhone apps and Cydia packages while I’m developing them. I can also SFTP into the iPhone’s filesystem, and use Ubuntu’s file browser to copy and paste files.


VLC media player

Plays anything you can transfer to your iPhone. Another great app rejected from the App Store.


Mobile Terminal

It’s pretty neat to have a terminal on your phone, even if I don’t use it very often. You never know when you might need to restart or rollback a remote server:

Terminal


SBSettings

This is an incredibly useful shortcuts menu, displayed by swiping the status bar:

SBSettings

The AutoLock control is really handy when using the Maps application. I use Maps as the GPS for my car, and I don’t want the screen to turn off while I’m driving.


MyWi – 3G tethering

I bought the MyWi package while using a 3GS. It’s pretty expensive at $19.99, but it was a lifesaver when I was living without DSL for a few weeks. I’m still using it after getting a 4S, since iOS 5’s ‘personal hotspot’ feature was a bit flaky. MyWi is rock-solid, and can display a small icon instead of taking over the entire status bar like the iOS hotspot.


5 columns for icons

This is just my personal preference, but I like having 5 icons in my dock and home screen.

Five Icons


Compiling and installing any open source iPhone apps

No $99 developer license required. One example is the Last.FM app, which is open source, but not available in NZ or HK iTunes stores. Here’s a decent list of some other open source iPhone apps.


Script anything

Finally, and perhaps most importantly, it’s just awesome to be able to script the phone to do whatever I want. For example, I can make the iPhone change the wallpaper every 30 minutes, show my prepaid balance in the status bar, or scroll up and down a page when it tilts.

Conclusion

Jailbroken iPhones FTW.

Please leave a comment if you have any tips to share!