Dynamic Tile Server with Google Maps, Mapnik, and Drupal

Lately I have been working on a project for projecting map data from shapefiles onto a google map.   It needed to be easy to manage and the client had to be able to upload shape files to drupal and have the map automatically update itself upon posting.  What we came up with was a Drupal back end that dynamically generates a Cascadenik map file.  From there we use a python subclass for tilecache to render that file into a Mapnik ready xml file.   Then of course that is projected onto a google map.

Lets start with the python sub-class.  Install TileCache via easy_install.  In the directory that python installs TileCache you are going to want to add the following file

RemoteMapnik.py:

import os
import sys
import tempfile

from TileCache.Layer import MetaLayer
from TileCache.Layers.Mapnik import Mapnik

from cascadenik.compile import compile

class RemoteMapnik(Mapnik):

    def __init__ (self, name, mapfile = None, projection = None, fonts = None, **kwargs):

        (handle, compiled) = tempfile.mkstemp('.xml', 'cascadenik-compiled-')
        os.close(handle)

        open(compiled, 'w').write(compile(mapfile))
        print(compiled)
        mapfile = compiled

        Mapnik.__init__(self, name, mapfile, projection, fonts, **kwargs)

This is basically a copy of the Mapnik.py file that already comes pre-packaged with TileCache.  The difference here is our new subclass takes a Cascadenik formatted xml file from a remote url and compiles it into a temporary file and then passes it on to the Mapnik class.

The next step is to configure tilecache.cfg to use our new class instead of Mapnik.  Here is a sample config.

tilecache.cfg:

[drupal]
type=RemoteMapnik
mapfile=http://example.com/mapnik
spherical_mercator=true
tms_type=google

And now onto the drupal side of things.

Drupal is doing alot of the heavy lifting for us.  First create a content type with a single cck filefield.  This will be the field used to store your zipped up shapefiles.  In my examples the content type is called “battle” and the field is field_shapefiles.

Lets start with the basics.

ahc.module:

/**
 * Implementation of hook_menu
 */
function ahc_menu() {
    $items = array();

    $items['battle'] = array(
        'page callback' => 'ahc_battle_map',
        'access arguments' => array('access content'),
    );

    $items['admin/settings/battle'] = array(
        'page callback'     => 'drupal_get_form',
        'page arguments'    => array('ahc_admin'),
        'access arguments'  => array('access administration pages'),
    );

    $items['mapnik'] = array(
        'page callback'     => 'ahc_mapnik_export',
        'access arguments'  => array('access content'),
        'type'              => MENU_CALLBACK,
        'file'              => 'ahc.mapnik.inc',
    );

    return $items;
}

/*
 * The battle map
 */
function ahc_battle_map() {
    drupal_set_html_head('<script src="'. check_url(url('http://maps.google.com/maps/api/js?sensor=false')) .'" type="text/javascript"></script>');
    drupal_add_js(drupal_get_path('module', 'ahc').'/js/ahc.js');

    // Setup JS settings
    $settings = array(
        'tmsServer' => variable_get('tms_server', ''),
        'lat'       => variable_get('location_center_lat', 0),
        'lng'       => variable_get('location_center_long', 0),
    );
    drupal_add_js(array('ahc' => $settings), 'setting');

    return theme('battle_map');
}

/*
 * Implementation of hook_theme
 */
function ahc_theme() {
    return array(
        'battle_map' => array(
            'template'  => 'templates/battle-map',
        ),
    );
}

/*
 * Administration Page
 */
function ahc_admin() {
    $form = array();

    $form['flush_cache'] = array(
        '#type'     => 'submit',
        '#value'    => t('Flush Tilecache'),
    );

    $form['tms_tilecache_location'] = array(
        '#type'             => 'textfield',
        '#title'            => t('Path to tilecache'),
        '#description'      => t('NOTE: The tilecache path must reside in your /tmp directory'),
        '#default_value'    => variable_get('tms_tilecache_location', '/tmp/tilecache'),
    );

    $form['mss'] = array(
        '#type'             => 'textfield',
        '#title'            => t('MSS File'),
        '#description'      => t('The full path to the mapnik stylesheet you wish to use'),
        '#default_value'    => variable_get('mss', ''),
    );

    $form['tms_server'] = array(
        '#type'             => 'textfield',
        '#title'            => t('TMS Server'),
        '#default_value'    => variable_get('tms_server', ''),
    );

    $form['location_center_lat'] = array(
        '#type'             => 'textfield',
        '#title'            => t('Map Latitude'),
        '#default_value'    => variable_get('location_center_lat', 0),
    );

    $form['location_center_long'] = array(
        '#type'             => 'textfield',
        '#title'            => t('Map Longitude'),
        '#default_value'    => variable_get('location_center_long', 0),
    );

    return system_settings_form($form);
}

/*
 * Implementation of hook_nodeapi
 */
function ahc_nodeapi(&$node, $op) {
    switch($op) {
        case 'insert':
        case 'update':
            //unzip shapefiles
            foreach($node->field_shapefile as $shapefile) {
                $path = _shp_path($shapefile['filepath'], true);
                $dst = _shp_path($shapefile['filepath']);
                _shp_unzip($path, $dst);

            }
            _ahc_flush_cache();
            break;
        case 'delete':
            foreach($node->field_shapefile as $shapefile) {
                $path = _shp_path($shapefile['filepath']);
                rrmdir($path);
            }
            _ahc_flush_cache();
            break;
    }
}

/*
 * @param $path
 * The path relative to the default files directory
 * @param $ext
 * If true will return a file extension to the zip file
 * @return
 * returns the full path to a zip file
 */
function _shp_path($path, $ext=FALSE) {
    $file_path = $_SERVER['DOCUMENT_ROOT'].base_path().$path;
    if (!$ext) {
        $file_path = str_replace('.zip', '', $file_path);
    }

    return $file_path;
}

/*
 * @param $path
 * source path of zip file
 * @param
 * destination directory for unzipped files
 */
function _shp_unzip($path, $dst) {
    $zip = zip_open($path);
    if (!$zip) return FALSE;

    if (!file_exists($dst)) {
        mkdir($dst);
    }

    while($file = zip_read($zip)) {
        $fp = fopen($dst.'/'.zip_entry_name($file), 'w');
        if (zip_entry_open($zip, $file, 'r')) {
            $buf = zip_entry_read($file, zip_entry_filesize($file));
            fwrite($fp, "$buf");
            zip_entry_close($file);
            fclose($fp);
        } else {
            drupal_set_message('Shapefile was not unzipped', 'error');
        }
    }
    zip_close($zip);
}

/*
 * Recursive rmdir
 */
function rrmdir($directory, $empty = false) {
    if(substr($directory, -1) == "/") {
        $directory = substr($directory,0, -1);
    }

    if(!file_exists($directory) || !is_dir($directory)) {
        return false;
    } elseif(!is_readable($directory)) {
        return false;
    } else {
        $directoryHandle = opendir($directory);

        while ($contents = readdir($directoryHandle)) {
            if($contents != '.' && $contents != '..') {
                $path = $directory . "/" . $contents;

                if(is_dir($path)) {
                    rrmdir($path);
                } else {
                    unlink($path);
                }
            }
        }

        closedir($directoryHandle);

        if($empty == false) {
            if(!rmdir($directory)) {
                return false;
            }
        }

        return true;
    }
}

/*
 * Clear tilecache files
 */
function _ahc_flush_cache() {
    $tilecache_location = variable_get('tms_tilecache_location', '/tmp/tilecache');
    // safeguards to only allow clearing files in /tmp
    $dir = explode('/', $tilecache_location);
    if ($dir[1] != 'tmp') return false;

    // Recursively delete files
    if (rrmdir($tilecache_location)) {
        drupal_set_message('Successfully flushed tilecache');
    } else {
        drupal_set_message('Unable to flush tilecache', 'error');
    }
}

In a nutshell this is what is going on in the above code. Shapefiles must be zipped up when they are uploaded to your content type. In hook_nopeapi, upon insertion or an update the script unzips the file into a directory named after the shape file and then flushes the tilecache directory.

ahc.mapnik.inc

/**
 * @return
 * Cascadenik ready xml file for tilecache
 */
function ahc_mapnik_export() {
        header('Content-type: text/xml');
        $xml = _ahc_export_xml();
        $xml->formatOutput = true;
        print $xml->saveXML();

        exit();
}

/**
 * @return
 * Cascadenik ready xml object
 */
function _ahc_export_xml() {

        $mss = variable_get('mss', '');
        $xml = new DOMDocument('1.0', 'UTF-8');

        // Create Map Element
        $map = $xml->createElement('Map');
        $map->setAttribute('srs', "+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null");

        $style = $xml->createElement('Stylesheet');
        $style->setAttribute('src', $mss);
        $map->appendChild($style);

        $result = db_query('SELECT nid FROM {node} WHERE type = "battle"');
        while ($row = db_fetch_array($result)) {
                $node = node_load($row['nid']);
                if ($node) {
                        _ahc_layer_export($xml, $map, $node);
                }
        }

        $xml->appendChild($map);

        return $xml;
}

/*
 * Build each layer from the shapefiles attached to a node
 */
function _ahc_layer_export($xml, &$map, $node) {
        foreach($node->field_shapefile as $shapefile) {
                if ($shapefile) {
                        $path = _shp_path($shapefile['filepath']);
                        $dir = opendir($path);
                        while ($file = readdir($dir)) {
                                if (strpos($file, '.shp')) {
                                        $file_path = $path.'/'.str_replace('.shp', '', $file);
                                }
                        }

                        $layer = $xml->createElement('Layer');
                        $layer->setAttribute('id', $shapefile['data']['description']);
                        $layer->setAttribute('srs', '+proj=latlong +ellps=WGS84 +datum=WGS84 +no_defs');

                        $datasource = $xml->createElement('Datasource');

                        $type = $xml->createElement('Parameter', 'shape');
                        $type->setAttribute('name', 'type');
                        $datasource->appendChild($type);

                        $file = $xml->createElement('Parameter', $file_path);
                        $file->setAttribute('name', 'file');
                        $datasource->appendChild($file);

                        $layer->appendChild($datasource);

                        $map->appendChild($layer);

                }
        }
}

In this file we are leveraging php’s DOMDocument class to generate the XML we need to send to tilecache. In my example here the projection is hard coded, and a simple query to pull the nid of all the battle nodes was used. In my use case this worked adequatly but you could replace the select query with a view if you needed more flexibility and if you are using shapefiles that are not WGS84 you’ll need to expand on that section of code.

Now onto our javascript. If you have worked with Google Maps API version 3 this should be pretty straight forward.

js/ahc.js

$(document).ready(function () {

    /************************************
     * MAPS
     */
    var mapCenter = new google.maps.LatLng(Drupal.settings.ahc.lat, Drupal.settings.ahc.lng);
    var zoom    = 13
    var myOptions = {
        zoom: zoom,
        center: mapCenter,
        mapTypeId: google.maps.MapTypeId.TERRAIN,
        mapTypeControl: false
    }

    // Mapnik Layer
    var battleMapOptions = {
        getTileUrl: function(coord, zoom) {
            return Drupal.settings.ahc.tmsServer + zoom + "/" + coord.x + "/" + coord.y + ".png";
        },
        tileSize: new google.maps.Size(256, 256),
        isPng: true
    };
    var battleMapType = new google.maps.ImageMapType(battleMapOptions);

    // Create map
    var map = new google.maps.Map(document.getElementById("map"), myOptions);
    map.overlayMapTypes.insertAt(0, battleMapType);

});

I’m not even finished with the module yet, so there may be improvements to my initial design. But hopefully this is helpful to anyone needing to do something similiar.

Now we have a fully functional TileCache server with a user friendly management interface via Drupal. And it all gets nicely displayed on a google map