Recently I’ve been working on a congressional tweet aggregator to get a handle on what our legislators are saying. To make that easier to see, I figured I’d start adding some charts and lists and other snazzy dataviz gizmos that are so hot these days.

I like D3.js as a graphing library. It makes clean, interactive, data-driven charts a snap to make in just a few lines of Javascript. Then it does its magic to render the data in crisp SVG, which I am quite fond of. On my site, I wanted to turn the crank on the back-end for charts that don’t update all that frequently, inject them into my templates, and spare the viewers of my site the heavy-lifting required for multiple charts—not to mention my poor server that has to execute several complicated queries to do the appropriate counting to generate the data to back the charts.

After a little poking around on the internet, I stumbled on to PhantomJS, which bills itself as a full-stack headless WebKit. Perfect. It can ping my website periodically, load the chart pages, and extract the SVG, I thought.

Not so fast. The Phantom is excellent at reading SVG, and it’s even good at rendering it to PDF or PNG. But that’s not what I wanted! I just wanted it to spit out the SVG for me after D3 was finished making it, untouched. And since SVG elements don’t have an innerHTML property, I needed to think harder to find a solution; i.e., ask Google. But Google didn’t seem to know, either. So I wrote a tiny script to extract page elements by ID. Maybe one of you will find it useful, too.

var system = require('system');

if (system.args.length != 3) {
    console.log("Usage: extract.js  ");
    phantom.exit(1);
}

var address = system.args[1];
var elementID = system.args[2];
var page = require('webpage').create();

function serialize(elementID) {
    var serializer = new XMLSerializer();
    var element = document.getElementById(elementID);
    return serializer.serializeToString(element);
}

function extract(elementID) {
  return function(status) {
    if (status != 'success') {
      console.log("Failed to open the page.");
    } else {
      var output = page.evaluate(serialize, elementID);
      console.log(output);
    }
  phantom.exit();
  };
}

page.open(address, extract(elementID));

All this Olympics has inspired me. I’ve decided to train for the summer games in Rio and to pursue my hobbies in full force. The former requires me to learn to kayak; the latter, to program, program, program.

I’m building a fun web-app game on top of Django. And I want to make it dance with a little Javascript. I really like that I can look up the URLs of my pages by an internal name in templates using the {% url %} tag, but unfortunately that functionality isn’t available to me inside my static media—particularly, I’d like to be able to reverse look-up URLs in my site’s javascript, but I don’t want to have to render the file each time it gets served.

After hunting around for a while, I figured, “Why not just render to Javascript once, fill in all the links, and be done with it?” So that’s what I did. And it doesn’t take much.

I wrote a custom command called populate_urls that consumes a template (in this case, my Javascript) and spits out the finished product with all the holes filled in by Django and its handy URL resolvers. Then I stash its output in my static directory and pretend the whole thing never happened.

from __future__ import print_function

import sys

from django.core.management.base import BaseCommand
from django.template import Template, Context

class Command(BaseCommand):
from __future__ import print_function

import sys

from django.core.management.base import BaseCommand, CommandError
from django.template import Template, Context

class Command(BaseCommand):
args = ‘ ‘
help = ‘Populates the URLs in the template. If no output filename is supplied, renders the template to standard out.’
traceback = True

def handle(self, *args, **options):
if len(args) < 1: raise CommandError("You must supply a template.") with open(args[0], 'r') as templatefile: template = Template(templatefile.read()) context = Context() with sys.stdout if len(args) < 2 else open(args[1], 'w') as output: print(template.render(context), file=output) [/sourcecode]

A pet project of mine has flung me into the exciting though less-than-firm territory of web-backed geographical information systems. Since I don’t have the thousands of dollars it costs to get a commercial server like those provided by ESRI, I’ve had to check out the open-source alternatives. And there are some out there. I’m using GeoServer, and it works great! I can send all the web-feature service transactions (WFS-T) in XML I want and it works every time. Not bad if you want to make a GoogleMap of your house on your own—so long as you’re content to hard-code everything by hand. Should you want to jazz things up a bit (i.e., make minimally useful dynamic maps), like me, then you have to do a little more work. Actually, you need to do a lot more work.

GeoServer is built on top of a gargantuan set of Java libraries, collectively packaged under the name GeoTools. Now I appreciate that this thing exists, all two hundred and fifty megs of source code and all the functionality that comes with it. However, navigating the mountain of documentation for this thing is, at least for me, a little daunting. It took me a few days (and some serious help from my friend Matt) to figure out how to write a simple update transaction using their API. (Compare that to the forty-two seconds it takes me to type up the XML.)

Since other people might want to know what they have to do update an attribute field using WFS with GeoTools, and since I couldn’t easily find out how to do it elsewhere, I’ve decided to post a short snippet of code right here on my blog. That’s right: my charity knows no bounds.

In this example I’m going to update the value of all the features (polygons, lines, points, whatever) that match a simple filter. Here I’m going to change the value of propertyToUpdate to updatedValue using a filter to get all the features with the attribute called constraintProperty with a value of constraintValue. I’ve marked them in red, so that it’s as easy as possible to customize this example to fit your needs. Let’s start with the XML that the Open Geospatial Consortium standards expects to see.

<wfs:Transaction service=”WFS” version=”1.0.0″
     xmlns:myns=”http://www.domain.com/ns”
     xmlns:ogc=”http://www.opengis.net/ogc”
     xmlns:wfs=”http://www.opengis.net/wfs”>
  <wfs:Update typeName=”myns:LayerToUpdate“>
    <wfs:Property>
        <wfs:Name>propertyToUpdate</wfs:Name>
        <wfs:Value>updatedValue</wfs:Value>
    </wfs:Property>
    <ogc:Filter>
        <ogc:PropertyIsEqualTo>
            <ogc:PropertyName>constraintProperty</ogc:PropertyName>
            <ogc:Literal>constraintValue</ogc:Literal>
        </ogc:PropertyIsEqualTo>
    </ogc:Filter>
  </wfs:Update>
</wfs:Transaction>

Now let’s rock out the Java.

Like I said, GeoTools is mammoth. To make life easy, we’re going to import a whole bunch of classes for this example. So many, in fact, that their number really warrants my displaying them here in their own list. What’s more, the names of some of classes (like Filter) show up in more than one package, and you need to keep track of which is used where. So keep an eye out for things from org.opengis.

import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.HashMap;
import java.util.Map;
import java.util.logging.Level;

import org.geotools.data.DataStore;
import org.geotools.data.DefaultTransaction;
import org.geotools.data.FeatureStore;
import org.geotools.data.Transaction;
import org.geotools.data.wfs.WFSDataStoreFactory;
import org.geotools.feature.AttributeType;
import org.geotools.feature.FeatureType;
import org.geotools.filter.FilterFactoryFinder;
import org.geotools.xml.XMLSAXHandler;

import org.opengis.filter.Filter;
import org.opengis.filter.FilterFactory;
import org.opengis.filter.expression.Expression;

In our constructor we’ll set up a connection to the WFS server using a URL. If you’re tinkering with GeoServer, then that URL you’re looking for probably looks something like http://localhost:8080/geoserver/wfs. Since we know that we’ll want to filter our responses, it’s not a terrible idea to make a filter factory now and save it for later. In GeoTools everything is made using a factory. For filters, we need to make a factory using the new keyword, though. Here goes.

public class WFSUpdater {

    private DataStore wfs;
    private FilterFactory filterFactory;

    public WFSUpdater(String url) {
      try {
        URL endPoint = new URL(url);

        XMLSAXHandler.setLogLevel(Level.OFF); // turns off logging for XML parsing.

        // Parameters to connect to the WFS server, namely the URL.
        // You could have others, say if you had to authenticate your connection with a username and password.
        Map params = new HashMap();
        params.put(WFSDataStoreFactory.URL.key, endPoint);

        wfs = (new WFSDataStoreFactory()).createNewDataStore(params);
        filterFactory = FilterFactoryFinder.createFilterFactory();

      } catch (MalformedURLException e) {
        e.printStackTrace();
      } catch (IOException e) {
        e.printStackTrace();
      }
    }

Now that we have a connection, it’s time to make the transaction. As a first timer to GeoTools, I found it difficult to crawl through the documentation. Lots of their classes and methods have been deprecated with no clear hints about what to use instead. I had a hard time finding the right constructors. In what follows everything is hard-wired into the code, but it shouldn’t be all that bad to tweak things so that it works the way you like.

public void updateProperty(String propertyToUpdate, String updatedValue) {
    // This is the layer name on the server.
    String layer = “myns:LayerToUpdate“;
    Transaction update = new DefaultTransaction(“update”); // The handle/ID of this transaction is called “update.” It’s required.

    try {
      // Make the filter.
      Expression property = filterFactory.property(“constraintProperty“);
      Expression value = filterFactory.literal(“constraintValue“);
      Filter filter = filterFactory.equals(property, value); // This is an org.opengis.filter.Filter.

      FeatureStore features = (FeatureStore) wfs.getFeatureSource(layer);

      // Set the transaction. Otherwise, we can’t commit our changes later on.
      features.setTransaction(update);

      // Fetch the property from the FeatureType schema so that we can update it with the new value.
      FeatureType schema = features.getSchema();
      AttributeType atrributeToUpdate = schema.getAttributeType(propertyToUpdate);

      features.modifyFeatures(atrributeToUpdate, updatedValue, (org.geotools.filter.Filter) filter); // There’s that casting again.

      // Record the modifications.
      update.commit();

      } catch (IOException e) {
        e.printStackTrace();
      }
    }
}

Anyway, I hope this saves some people the hassle of tearing through the Javadocs for GeoTools. Also, if there’s a better way to do what I did, please let me know. Happy GIS-ing.

Technorati Tags: geotools, geoserver, opengis, ogc, java, wfs, wfs-t, shapefile, update, request, javadoc, gis, esri, xml