Matt Fawcett

Playing with Node.js

Fri, 19 Feb 2010 22:47:00 +0000

Today I decided to have a look at Node.js. I have heard quite a lot about it but have avoided looking at it for a while. Node.js provides a way of building server side network programs, written in Javascript. I took a look at a tutorial over at http://simonwillison.net/2009/Nov/23/node/ for installing Node and building a simple Hello World web service. I ran an apache benchmark test and was suprised with the results. I managed to get 6075 requests per second on my Macbook Pro!

To compare it with something that I'm used to, I wrote the same Hello World script service Sinatra and served it up using the thin webserver, which gave me 1606 requests per second which is quite a difference.

The 2 scripts and apache benchmark outputs are below:

Installing Prey on snow leapord

Thu, 05 Nov 2009 21:48:00 +0000

Today I decided to install Prey on my mac book. Prey is basically a bash script that runs from a cron job every 10 minutes or so. It contacts a remote server to ask if the computer is stolen or not and if so can send a screenshot of the desktop and also a photo from the webcam as well as providing the IP address of the thief.

Prey does have the ability to have the data sent to their website but I decided to handle this myself and have the script email the data if a webpage I have set up on my domain gets deleted. I ran into a few problems getting the script to email me, it uses a perl script called sendEmail. I banged my head against a wall getting errors from Google like “Username and Password not accepted.” even though everything was correct. The solution was do download the latest version of SendEmail from http://caspian.dotconf.net/menu/Software/SendEmail/sendEmail-v1.56.tar.gz, extract and replace the file called sendEmail in /usr/share/prey/lib/ with the new one.

Use a PHPBB forum to handle user authentication in your rails app

Mon, 31 Aug 2009 12:12:00 +0000

Im currently in the process of rebuilding www.realalehunter.co.uk in rails. Its currently uses PHPBB to handle logging in and authentication so to make the rebuild simpler I thought keep it working in the same way instead of trying to migrate all the users/passwords to a separate system.

So I wrote a rails gem / plugin to make this a bit easier. It currently works on Rails > 2.3.3. I decided that I still wanted to have a users table in my applications database to make relationships easier but I would just store the stuff that’s important to the app itself (not passwords etc). So if you visit the rails site and are logged in to the forum a user will also be created in your rails app database if one does not exist.

So here is how to get started:

# install the gem
gem install mattfawcett-phpbb-auth

Tell rails that you want to use the gem it by adding the following to your environment.rb

config.gem "mattfawcett-phpbb-auth", :lib => "phpbb_auth", :source => "http://gems.github.com"

Add an entries to database.yml for phpbb_database_development, phpbb_database_production and phpbb_database_development.

In your config directory create a file called phpbb_auth_settings.rb, copy whats below into the file and change to match your setup.

PHPBB_AUTH_FORUM_DATABASE_TABLE_PREFIX = 'phpbb_'
PHPBB_AUTH_COOKIE_NAME = 'phpbb3_7uah4'; 
PHPBB_AUTH_LOCAL_USER_MODEL_NAME ='User' #this is the name of the model that you will use to store information about  users in your rails app 
PHPBB_AUTH_REMOTE_REMOTE_IDENTIFIER = 'user_email' #I use email to identify somebody, you may want to use the username  instead 
PHPBB_AUTH_REMOTE_LOCAL_IDENTIFIER = 'email' #this is the name of a field on your local user model that will be used to  lookup the value of remote identifier.
PHPBB_AUTH_COLUMNS_TO_DUPLICATE = {:user_website => :website} #specify any additional fields that you would like copying  to your local user model

In application_controller.rb, include the module

include PhpbbAuth

How you handle the rest is up to you but here’s what I did. Create a method called set_current_user.

def set_current_user   
  @current_user = current_user 
end

Add a before filter to the controller to run the method.

before_filter :set_current_user

You will then have access to the @current_user variable in your controllers and views. This will be either nil if the user is not logged in, or an instance of your local User.

The source is up on Github.

Monit Aggregator - A Sinatra interface for Monit

Sun, 05 Jul 2009 16:41:00 +0000

I use Monit to keep an eye on server processes and restart them if they go down. It comes with a useful web interface that gives you an overview of your servers health, and what processes are up/down. If you have a lot of servers then it can be a bit of a pain because you have to access each web Interface with a different URL.

This weekened I wrote a Sinatra app that aggregates the data from each web interface and displays it all on one page. The source is on Github.

It currently does not support HTTPS, and it only currently displays the overall system and individual processes.

To get going, install the dependant gems

gem install sinatra nokogiri haml

Now clone the repo

git clone git://github.com/mattfawcett/monit-aggregator.git

Then edit the monit_installations.yml file to include the details of each of your Monit web servers. To start the app running just type

ruby app.rb

Open your browser to http://yourdomain.com:4567

Alternatively you could run with Passenger.

There is also a free hosted version available on my server monitoring site Runs Like Clockwork

s3ify gem - put the assets for a HTML email online

Sat, 27 Jun 2009 12:35:00 +0000

I often have to deal with setting up email sends, which usually involves getting a zip file from a designer containing a html file and some images. I usually unzip, convert all the image references to point to an S3 path, upload the whole folder to S3 using S3fox, and change the permissions to be publicly readable.

This is a bit repetitive and manual for my liking so I wrote a small ruby gem to make it a bit simpler. It uses Nokogiri to parse the HTML (so you will need a recent version of libxml) and the aws-s3 gem to upload everything to S3. I used the commander gem to create a command line interface to use it.

So first, get it installed

sudo gem install mattfawcett-s3ify

You will need to have your Amazon web services access information set in your path, if you don’t have this already, your best bet is to create a hidden file in your home directory called .amazon_keys with the contents

export AMAZON_ACCESS_KEY_ID='abcdefghijklmnop'
export AMAZON_SECRET_ACCESS_KEY='1234567891012345'

then add the following to your shell’s rc file (usually .bashrc)

if [[ -f "$HOME/.amazon_keys" ]]; then
    source "$HOME/.amazon_keys";
fi

Then to use it simply open up a shell and type

s3ify -b my_bucket -p my_folder_in_bucket ~/Desktop/my-email/

The -b flag is the bucket you wish to upload to, the -p flag is the path in the bucket that you want to upload to and finally you declare the path to the folder where your HTML email is on the file system. For each HTML file, the gem will create an additional file with the prefix of “_s3” which will have the image references pointing to S3.

The code is on Github

Server Monitoring with SMS alerts

Sun, 07 Jun 2009 17:02:00 +0000

For the last few weeks Iv been working on a new website in my spare time. Its now live and is at runslikeclockwork.com. Its a server monitoring tool where you add your sites and you will get an SMS alert when a site goes down / comes back up again.

Its slightly different from most other services because it checks the rendered HTML of a page for a keyword or phrase specific to that site. This covers you when you misconfigure an Apache/Nginx Virtualhost which can sometimes leave you with the wrong site being served on a domain, but of course the server is still giving out 200 status headers.

At the moment its very simple but I am planning on adding some more features over the next few weeks.

spree-phpbb-auth - Use an existing phpbb forum for users/authentication with a Spree shopping cart

Sun, 22 Mar 2009 13:58:00 +0000

Im in the process of setting up a Spree shopping cart on an existing site which uses a phpbb forum for storing all users and logging in etc. I wanted the existing users of the site to be able to purchase from the shopping cart area without having to go through another sign up process and have 2 sets of usernames and passwords so iv created a Spree extension called spree-phpbb-auth.

The extension looks for the phpbb session cookie and looks it up in the forum database, if the user is unknkown to the spree application, a user will be created within Spree in the background. The standard Spree authentication system is then used to set that users session up to declare them as being logged in. It overwrites the existing login /logout/register/edit my account actions so that they redirect to the phpbb equivalents.

Its not being used in Production yet but it all seems to work ok locally. There are a few issues that I can think of

Won’t work accross different domains (should work on subdomains if you set the cookie domain to be ".site.com" instead of "forum.site.com")
All users that get automatically added to the Spree users table get assigned to the group "users" so it doesn’t look at the phpbb permissions at all.
If you allow your phpbb user to change email addressess then when they hit the cart after a change, they will be registed as a new user.

Installing

script/extension install git://github.com/mattfawcett/spree-phpbb-auth.git
Change the contents of config/phpbb_auth_settings.rb
Add an entry to your database.yml file for "phpbb_database_production"

Sync your rails development environment data with production or staging

Sun, 01 Mar 2009 14:51:00 +0000

When your developing a web app, you usually end up with lots of duff data both in your database and file system from when you have been testing inserting data and uploading files when things have been half implemented. Usually its a good idea to take a fresh database dump from your production server and download the assets (or whatever you call the directory where user uploads go) so that you actually have some real data.

This is usually a bit of a pain to do as you need to ssh in to dump the database, SCP the assets. It also forces you to develop using a MySQL database if thats what your production server uses.

Adam Wiggins has released a plugin called yaml_db that allows you to dump and import your database into/from YAML format. Today I wrote a quick rake task that connects to your production or staging over SSH, runs the task to dump the YAML file (rake db:data:dump) and then downloads it locally with SFTP. It then runs the rake task to import the data (rake db:data:load). It also uses Rsync to sync any asset or other folders on your remote server that are installation specific to your local development environment.

At the moment the only downside to the script is that it calls rsync from the shell which asks you to type your password. It would be good to have it all inclusive within the script, but that doesn’t seem possible with rsync. The script is a gist on Github

Hoptoad notifier for PHP applications

Tue, 03 Feb 2009 08:07:00 +0000

I recently started using Hoptoad to in my ruby/rails projects that notifies you of any exceptions that your app throws. Unlike other solutions such as the exception notifier plugin, if an error occurs hundreds of times you just get 1 email to let you know. You can then login to Hoptoad and set it to resolved.

I thought this would be also good to use in PHP applications as well and they do provide an API for you to use. I found a project on Github called php-hoptoad-notifier by Rich Cavanaugh. I gave it a go but it relied on a couple of PEAR packages being installed and as I don’t fancy installing them on lots of servers. Iv forked and modified it to be its own standalone code base. It now just relies on the PHP curl module.

Here it is: http://github.com/mattfawcett/php-hoptoad-notifier/tree/master

My first Ruby gem - ruby-reads-php

Sat, 24 Jan 2009 23:52:00 +0000

Iv being meaning to experiment with building my own Ruby Gems for a while now but have been too busy to actually give it ago. This weekend I took the plunge and wrote a gem called ruby-reads-php.

Iv got quite a lot of applications and websites online that are written in PHP and most of them have some kind of configuration file to define things like database settings to keep everything in one place. I occasionally want to bolt an extra temporary feature onto an app (eg for the next month send a CSV file of people registered to somebody by email).

As Ruby is now my language of choice for most things and due to the fact that I could do something like this faster in Ruby than PHP it makes sense for me to do so. The problem I have is that I need to replicate my database configurations into a Ruby file which isn’t very DRY. I could convert into a more standardised format such as YAML but this would mean changing the PHP code that I know currently works.

So this gives me an ideal oppertunity to write a Ruby Gem. I wrote some Rspec tests for what I wanted to do and then wrote the code. I used TechnicalPickles Jeweler gem to manage my Gem Spec and publish to Github. Within 15 minutes my Gem is available to download

To install it

 
#First add Github to your gem sources 
gem sources -a http://gems.github.com 
#Now install the Gem 
gem install mattfawcett-ruby-reads-php

To use it

   
#require rubygems and the ruby_reads_php gem   
require 'rubygems'   
require 'ruby_reads_php'      

#load and parse your PHP configuration file   
config = RubyReadsPHP.read(File.dirname(__FILE__) + "/config.php")  
my_database_hostname = config.constants['DB_HOST']

Example PHP configuration file (config.php)

 
<?php
define('DB_HOST' ,'localhost');
define('DB_USER' ,'web');
define('DB_USER' ,'my_password');
?>

At the moment it only supports constants but could be built on to handle arrays etc. Please feel free to fork, hack, send pull request.