Converting between Latitude/Longitude & OS National Grid Reference points in PHP (another fun postcode map)

I decided to have a play around with the Google Maps Javascript API. Seeing as I recently acquired the UK postcode database I thought I'd plot postcodes onto Google Maps.

The tricky part turned out to be converting between the grid references used in the PAF and latitude and longitude. Fortunately I found a page explaining how to do the conversion in Javascript. I needed the conversion to be done server side so I have ported the code to PHP [source].

To play with my map go to this page. To stop people screen scraping my postcode database the script will stop working after you have moved the map more than 200 times.

PHP Error Handling for Dynamic Images

A long time ago I wrote a script to handle PHP errors in a useful way. I still use it when writing in PHP today but it is not much use if you are writing a script that is producing images so I recently wrote a version that outputs to an image. Here is a sample error and here is the source code. As long as you insist on writing totally error free code (i.e. no E_NOTICE's) it can be included directly into any page that generates images. Remember to remove it before going into production though (it spits out source code and variable dumps when there is an error).

All source code and related services that I release are "dontation-ware". If you use the above please make a dontation (pay whatever you think the it is worth) or (where applicable) leave the link to my site attached.

RSS Ticker Screensaver

I've been playing about doing some C# programming. I started out by making myself a screensaver. In the configuration window you enter a list of your favourite RSS feeds, then when your computer is idle it shrinks your primary display and surrounds it by a ticker interface not dissimilar to those seen on 24 hour news channels. If you want to have a look download this file, then right click on it and click 'Install'.

Blogger Beta Label Cloud

I have just upgraded to Blogger Beta. I love that they have finally added support for labels, however I cannot get a list of my labels to appear in my template. In fact I am not even getting the new WYSIWYG template editor that they are bragging about. I think it may be because I publish to my own server via FTP.

Anyway, I created a quick solution for the time being and am posting it hear in case anyone else has the same problem and wants to steal it:

This PHP code creates a label cloud (put it anywhere on the server you publish to via FTP and change the obvious constants):

Update 30 April 2007 to remove duplicate entries added by Blogger.

<?php
 define('PREFIX', '/labels/');
 define('SEARCH_DIR', 
  '/home/httpd/vhosts/da.vidnicholson.com/httpdocs/labels/');
 define('THIS_FILE', 'cloud.php');
 
 if(file_exists(SEARCH_DIR.'_cloud_include_cache.html') && 
  filemtime(SEARCH_DIR.'_cloud_include_cache.html')>(time()-(60*60)))
 {
  echo file_get_contents(SEARCH_DIR.'_cloud_include_cache.html');
 }
 else
 {
  $output = '';
  $files = array();
  $dir = opendir(SEARCH_DIR);
  $max_size = -1; $min_size = -1;
  while($file = readdir($dir))
  {
   if($file != '.' && $file != '..' && $file != THIS_FILE && $file != 
    '_cloud_include_cache.html')
   {
    $files[$file] = filesize(SEARCH_DIR.$file);
    if($max_size==-1 || $max_size<$files[$file]) $max_size = $files[$file];
    if($min_size==-1 || $min_size>$files[$file]) $min_size = $files[$file];   
   }
  }
  closedir($dir);
  $displayed = array();
  foreach($files as $name=>$size)
  {
   if(!isset($displayed[urldecode(str_replace('.html','',$name))]))
   {
    $displayed[urldecode(str_replace('.html','',$name))] = true;
    $output .= "<span style='margin: 3px; font-size: ".
     (10+round((1+(($size-((($max_size-$min_size)/2)+$min_size))/($max_size-$min_size)))*100)).
     "%;'><a href='".PREFIX.htmlentities($name)."'>".
     htmlentities(urldecode(str_replace('.html','',$name)))."</a></span> ";
   }
  }
  echo $output;
  $fp = fopen(SEARCH_DIR.'_cloud_include_cache.html','w');
  fwrite($fp, $output);
  fclose($fp);
 }
?>

Now setup some mechanism whereby this file is included in your sidebar. A naive way would be to use an include() statement in your template and make your server parse .html files as PHP (e.g. by adding

AddType application/x-httpd-php .html

to a .htaccess file) but you are relying on Blogger correctly escaping any comments so malicious users cannot run arbitrary PHP code on your server. A better way is to introduce some keyword to your template and have code on your server scan for this keyword and overwrite the correct content. That is all beyond the scope of this post though.

You will notice that my code caches the cloud so that the labels directly does not get scanned on every single page view (just once per hour). For this to work your SEARCH_DIR must be writable by your web server. Also I am hoping that Blogger do not paginate label pages, if they do this will break my code, none of mine seem to be paginated though.

All source code and related services that I release are "dontation-ware". If you use the above please make a dontation (pay whatever you think the it is worth) or (where applicable) leave the link to my site attached.

File Upload Status Bar for PHP

When you are using PHP and Apache and you upload a file to a PHP script using a multipart/form-data POST your PHP script typically does not start executing until all of the POST data has reached the server. This makes it impossible to provide server-side feedback to your clients about the status of their uploads.

One way around this problem is to have another process running on the server capable of reading the clients headers, passes all data received onto the actual web server whilst keeping count of how much data it has passed on and telling clients the current status of a given request. This however sounds easier than it actually it is when it comes around to implementation. Your server program has to understand quite a bit about HTTP (not a simple protocol in its full detail) and there are logging and security implications (suddenly your webserver thinks a lot of its requests are coming from 127.0.0.1 when they are not).

Anyway, I have created a "first draft" of such a program. It cheats a bit by forcing the Connection header of every HTTP request and response to Close so that it only has to deal with one request per connection (this should be fine if you only send file uploads through the proxy program). I attempt to handle the logging program by writing to standard output my own log in the same format as my Apache server does so that this output can be piped into the Apache log file (requests will then appear duplicated but at least you have a record of the actual end users who have used the server).

The solution is by no means complete, it works, and it is secure in the sense that it wont give a remote user any more that it "says on the tin" but I have not tested it under load and it is probably quite susceptible to denial of service attacks in its current state. I am releasing the Java source code hoping that people might make improvements and give feedback, use it at your own risk!

Usage instructions from the source code:

 /*
  * Command Line Usage:
  *  java UploadServer 81 that-server >> 
  *    /location/of/your/apache/access_log
  *  Where 81 is a spare TCP port to listen on, and 
  *  that-server is a server listening for web 
  *  connections on port 80.
  *
  * Remote usage:
  *  http://this-server:81/foo
  *  Will return http://that-server:80/foo (works with GET,
  *  POST, etc.)
  *  http://this-server:81/foo___STATS___ will return a string
  *  describing how many bytes have been transmitted in the 
  *  current users (based on IP address and Cookie contents) 
  *  request to http://this-server:81/foo and (if available) 
  *  how many bytes the server is expecting.
  */

Some JavaScript to display a status bar might work something like this (using the forms onsubmit attribute to call statusBar() and <div id="status"><h2>Upload status</h2><div id="statusinner">Starting upload...</div></div> somewhere in your HTML):

 function requestObject()
 {
  return ((navigator.appName == "Microsoft Internet Explorer") ? 
   new ActiveXObject("Microsoft.XMLHTTP") :  new XMLHttpRequest());
 }
 var statusconn = requestObject();
 var statusBarUri = '';
 function statusBar()
 {
  document.getElementById('status').style.visibility = "visible";
  startStatusBar();
 }
 function startStatusBar()
 {
  statusconn.open('get', 'http://this-server:81/foo___STATS___',
   true);
  statusconn.onreadystatechange = updateStatusBar;
  statusconn.send('');
 }
 function updateStatusBar()
 {
  if(statusconn.readyState == 4)
  {
   if(parseInt(statusconn.responseText)==-1)
   {
    document.getElementById('statusinner').innerHTML = 
     "(Status bar loading...)";
   }
   else
   {
    if(statusconn.responseText.indexOf('/')==-1)
    {
     document.getElementById('statusinner').innerHTML = "(" + 
      (Math.round(parseInt(statusconn.responseText)/1024)) + 
      "KB uploaded so far.)";
    }
    else
    {
     var bits = statusconn.responseText.split("/");
     document.getElementById('statusinner').innerHTML = "(" + 
      (Math.round(parseInt(bits[0])/1024)) + 
      "KB/" + (Math.round(parseInt(bits[1])/1024)) + 
      "KB uploaded so far.)";     
    }
   }
   setTimeout("startStatusBar()", 100);
  }
 }

All source code and related services that I release are "dontation-ware". If you use the above please make a dontation (pay whatever you think the it is worth) or (where applicable) leave the link to my site attached.

PHP Image Uploader and Manipulator

Recently I created some code to allow users to upload a JPEG image to a website and resize and crop the image afterwards.

Seeing as this sort of feature could be useful on a lot of sites I made the code as general as possible and am releasing the source code here.

Here is a live demo of the utility.

The source code is in a zip file. You provide a new version of the cropper.php file for any place on the site that you wish to use the tool. Here is an example to show the customisation options available:

<?php

 // This script stores temporary image data in the user's session
 session_start(); 
 
 // The dimensions the uploaded image is resized to.
 // (The aspect ratio is maintained and no quality is lost during 
 // the final crop no matter how small you set these values)
 $default_max_width = 640;
 $default_max_height = 480;

 // The size of the image the user will create:
 $cropped_width = 440;
 $cropped_height = 280;

 // A name for the session variable the script will use (must be
 // unique for your domain)
 $unique_session_key = 'imgcropper';

 // A directory that the webserver can create files in, read from
 // and delete files from (the trailing slash is required)
 define('TEMP_DIR','temp/');

 // Before the user uploads their own image they get to play with
 // this one:
 $default_image = 'default.jpg';

 // A semi transparent gif used to create the darkened effect
 // around the edge of the page.
 $border_background = 'screen.gif';
 
 // After the user has chosen the area to crop, the cropped 
 // image is passed to the following function that you must 
 // write your own code for:
 function storeImage($img_id)
 {
  // $img_id is a GD image reference, if you want the actual 
  // JPEG data pass it to imgdata (as below)
  $_SESSION['imgcropper']['cropped'] = imgdata($img_id); 
 }
 // ...then the user is redirected to this page: 
 $redirect_after_crop = 'done.php';

 // All of the code:
 include("cropper-include.php");
 
?>

The reusable file cropper-include.php contains all of the PHP image uploading and resizing logic, along with the HTML, JavaScript and CSS for the page that allows users to upload their image, drag it around, resize it and save it.

My code assumes that users will not upload images that are too large and I use the memory limit setting in my php.ini file to prevent denial of service attacks. You may wish to alter the code to check the size of uploaded images.

Update (7 Feb 2007): I have previously answered peoples questions in comments on the blog post However due to new work commitments I will probably be too busy to continue doing this so I have set up a Google Group at for people to post their questions to and hopefully other users of the script will be able to provide answers.

All source code and related services that I release are "dontation-ware". If you use the above please make a dontation (pay whatever you think the it is worth) or (where applicable) leave the link to my site attached.

Very Easy Blogger Categories [Making a Backup]

Donny Bahama posted this comment about my Very Easy Blogger Categories:

This is the best looking categories solution I've seen. I haven't liked any of the blogger search or del.icio.us methods. Only one problem with it... If you ever stop hosting the database and php file (or even if your server goes down), everyone's blog categories will stop working! I'd be much more comfortable with this knowing that my categories will work as long as MY server is working. I don't suppose you could be persuaded to post your php code and database schema?

First off I should mention that I don't intend to stop hosting the service, note that it is running on the same server as another free service that I provide: QuizSender.com and this has been up and running since 2002. I should also mention that the same problem exists with del.icio.us or any other similar method.

Maybe this has not put you at ease though? Well I am afraid I don't want to publish the source code but how about a compromise?

I have modified the script so that you can easily backup your section of my categories database. If I ever do disappear of the face of the earth you will have your own copy of your categories database that you can import into any alternative categories system.

Just go to http://da.vidnicholson.com/blogtags.php?backup=yourdomain.com as often as you like and save the XML file that you are given. So, for example, to see my backup file go to http://da.vidnicholson.com/blogtags.php?backup=da.vidnicholson.com

Bug in Solitaire

I just found a bug in Solitaire! Arguably one of the most tried and tested components of Windows.

I was playing in Vegas mode and had gone through my stack, in an effort to do some last minute rearranging I was placing cards from the Ace piles back onto the table. I pulled the 9 of diamonds down and accidentally dropped it on the 9 of spades and it stuck, bringing the 8 of diamonds with it. There must have been something else special about what I did though because I have tried to engineer similar situations and am not seeing the same behavior.

Yes, I know, I play too much Solitaire!

Automated Word Speller [Update]

My Automated Word Speller is now capable of displaying codes/e-mail addresses as images as well as converting them to MP3 dictations.

Codes produced using the script look like this:

Try it out for yourself; use the form below to create a one-time link, or read my article on Public-Key Cryptography in PHP to create your own links on the fly.

Try it out; enter a word: (supports a-z, 0-9, @ and .)

All source code and related services that I release are "dontation-ware". If you use the above please make a dontation (pay whatever you think the it is worth) or (where applicable) leave the link to my site attached.

Very Easy Blogger Categories [Internationalisation]

As can be seen in the comments to my previous posts, I have been under quite some pressure to support multiple character sets in my Very Easy Blogger Categories system.

I now try and automatically detect the character set that you are using when I crawl your page. This allows me to output correctly escaped characters in the category lists that I place on your page.

Aside: The PHP htmlentities function does not support many character sets at all. As a work around I use the following code to correctly escape text in (hopefully) any character set: htmlentities(mb_convert_encoding($text, 'UTF-8', $charset), $quotestyle, 'UTF-8').

This is an entirely server-side change and does not require any changes to your templates or blog posts. Please let me know how it works by adding comments to this post.

Public-Key Cryptography in PHP

I have just been implementing a Public-Key cryptography system in PHP. This will allow users of my Automated Word Speller to link to sound files without them having their address/code in plain text and without me having to run a server-side database storing sensitive data.

I chose to make a system based on Diffie-Hellman key exchange. I share with you the parameters g and p that we will be using, along with my 'public key' g^y mod p; I keep my 'private key', y secret. You will generate your own private key, x and keep it secret (probably hidden in your code) but you will send me your public key, g^x mod p with every request. We can now both generate a key, k = g^xy mod p = (g^x mod p)^y mod p = (g^y mod p)^x mod p. This key will be used as a key to a function that does the base 10 equivalent of Vernam Encryption (addition/subtraction modulo 10). If the plaintext is longer than the generated key then the plaintext is split into blocks and ECB chaining is used.

Links to sound files look something like this: http://da.vidnicholson.com/reader/test.php?enc=eNoNysENAEEMwsCWIOQg9N%2FY7cvSyFSv61T8aKDkhtLyjCfDGoJNANPXRLyOZ6ay81YgacBbf1CFfsaeMyfIPwP%2FFF8%3D&pk=eNoNy8ERADAIAsGWEAW0%2F8aS583s9XRoZm6lxuh3XS2ILXncR%2FVBNdJZDrdBLnCi6KrwLx03bqykAmR9H3nR6wcIwRR5
Where:

• enc is a base64 encoding of a gzipped comma-separated list of encrypted blocks, and
• pk is a base64 encoding of your gzipped public key.

This URL cannot be converted to the original plain text without my private key (I invite you to try).

To create some encrypted links for yourself you can use the form below, or you may wish to see some sample code.

Try it out; enter a word: (supports a-z, 0-9, @ and .)

All source code and related services that I release are "dontation-ware". If you use the above please make a dontation (pay whatever you think the it is worth) or (where applicable) leave the link to my site attached.

Very Easy Blogger Categories [Update]

As pointed out in the comments by Jenny my Very Easy Blogger Categories didn't seem to work.

The problem was an AJAX 'security' feature that prevents accessing URLs on a different server using an XMLHttpRequest object. In my opinion the restriction is not implemented very well; in this case the XMLHttpRequest object was forbidden from accessing URLs on a server that the creating script was actually retrieved from.

I have made a work around (the categories menu doesn't actually use AJAX any more but appears the same). Follow the instructions as before, but use this script tag instead: <script language="JavaScript" src="http://da.vidnicholson.com/blogtags.php?dom=1"></script>.

Automated Word Speller

I have been working on a program that will spell out words. My aim is to turn it into an obfuscation service for websites that usually display e-mail addresses or bot prevention codes as images -- this program would allow them to also provide a sound file version of the text for visually impaired users.

Try it out; enter a word: (supports a-z, 0-9, @ and .)

Todo:

• Output MP3 instead of WAV [ Done ]
• Integrate with an image producing script [ Done ]
• Make a flash player so that sounds can be easily placed on websites
• Make use of public-key cryptography or a server side database so the text is not visible to bots [ Done ]
• Publish an API for webmasters [ Done ]

Very Easy Blogger Categories

I am a bit surprised that Blogger do not have a system in place for categorising blog posts. A quick Google search on the matter reveals that it is something a lot of people want to do but there are not really any elegant solutions available. The methods that I saw before I decided to make my own system involved creating a new blog for each category, or used the del.icio.us bookmark site, which does not allow for a very attractive user interface.

I have made a system that can be used by any website to categorise pages. Here are the installation instructions:

1. Add this code to any page that you want to assign a category: <img src="http://da.vidnicholson.com/blogtags.php?tag=Category Name" />, replacing Category Name with the name of the category the page belongs to. (If you are using Blogger insert this in the Edit Html tab whenever you create a new post.)
2. On any page that you want a list of categories to appear add the code: <div id="tagbox"></div> where you would like the list to appear and put the code <script language="JavaScript" src="http://da.vidnicholson.com/blogtags.php?dom=1"> </script> just before the closing </body>. (If you are using Blogger enter these in the Template tab.)

Update The bold text above reflects changes made to fix a bug.

Update Support has been added for other character sets.

...And you're done. No need to setup an account or password with me, no need to install any scripts on your server; as people start to view your articles through your 'permanent links' a category list like the one to the right will be generated for you.

How it works Including the img tag in each of your posts causes a transparent pixel to be shown. Whenever that image is loaded the referrer data is captured by my PHP script and your page along with the tag specified is added to my database (root URLs and archive pages are ignored so only individual posts should appear in your category lists). There is a delay of a few minutes before the page will actually appear in your category lists while the page is added to a queue for the server to verify it actually should have the tag given (this stops people spamming your category lists). If you later decide to remove a page from a category, just remove the img tag; it may take up to 24 hours for the change to be reflected in your category lists though. If you want a post to belong to more than one category, just use more than one img tag. For this to all work correctly you must have all your posts appearing on the same domain (e.g. myname.blogger.com) and be the only person using that domain. Including the JavaScript file and div tag in your Blogger template causes a nice AJAX category menu to be drawn into each of your blog pages.

Update If you want the names of the assigned categories to be displayed on each post instead of a transparent pixel, then use the following img tag: <img src="http://da.vidnicholson.com/blogtags.php?tag=Category Name&r=127&g=34&b=255&font=3" /> where r, g and b are decimal numbers between 0 and 255 specifying the amount of red, green and blue to use in the text colour and font is a number between 1 and 8 to specify the typeface that should be used.

Update I have added some true type fonts; font numbers 1, 2 and 3 are Times New Roman, Verdana and Arial respectively, 4 through to 8 are the older fixed width fonts.

Showing the number of posts in a category Change the Javascript shown above to <script language="JavaScript" src="http://da.vidnicholson.com/blogtags.php?dom=1&showcounts=1"> </script> if you would like the number of posts in each category to be shown on your blog.

Making a Backup If you would like your own copy of your category database as a backup just go to http://da.vidnicholson.com/blogtags.php?backup=yourdomain.com as often as you like and save the XML file that you are given. So, for example, to see my backup file go to http://da.vidnicholson.com/blogtags.php?backup=da.vidnicholson.com

All source code and related services that I release are "dontation-ware". If you use the above please make a dontation (pay whatever you think the it is worth) or (where applicable) leave the link to my site attached.

A Picture is Worth a Thousand Words

I have created a new blog that I will be posting photos to directly from my mobile phone. I simply have to take a photo using the digital camera built into my phone and send it by e-mail to a 'secret' address (hopefully at 3G speeds if I can find coverage!). I have a PHP script on my server that is periodically run by cron to check for new mails and post them to the blog using the Atom API.