Geotagging at SFO Museum, Part 7 – Custom Writers
This is the seventh of an 11-part blog post about geotagging photos in the SFO Museum collection. In the last post I wrote:
A writer that simply parrots things to a terminal window has limited utility. In the next post we’ll demonstrate how to implement a custom writer that extends these ideas in to something a little more useful.
By default the go-www-geojson application produces data encoded in the GeoJSON format. SFO Museum also publishes all its open data in the GeoJSON format, but structured as Who’s On First (WOF) documents.
In order to integrate the go-www-geojson application with our work that means we need to take the following steps given a GeoJSON record produced by the geotagging application:
- Find the WOF record associated with the image being geotagged.
- Update that record’s
geometrycoordinates with the latitude and longitude of thego-www-geotagcamera’s position. - Update that record’s
propertydictionary to include all the relevent geotagging information so that it can be used to preload an image in the geotagging application (like we demonstrated in part 5) or on a map on the Mills Field website. - Create or update an “alternate geometry” record containing the coordinates for the
go-www-geotagcamera’s field of view. - Publish both the primary and the alternate WOF records.
In the last post I wrote that:
The reason for requiring this level of indirection [in the way the application handles writing data] is that it allows the go-www-geotag application to be unconcerned with how data is output or where it is written to. The application expects to be given a valid writer.Writer instance and after parsing and validating input data assumes the writer will be responsible for everything else.
There’s quite a lot going on in the SFO Museum scenario I’ve just described but the go-www-geotag application remains unaware of these details. We simply create an SFO Museum specific writer.Writer instance and tell the geotagging application to use that when it needs to write data.
In part 2 of this series I wrote that:
Go applications can be pre-compiled in to operating system specific binary applications that don’t require any additional steps or dependencies to run.
That’s one of the great things about developing applications in the Go programming language. One of the challenging things about developing applications in Go, a consequence of their ability to be baked in to pre-compiled binaries, is that those applications need to know about everything they’re going to do in advance.
In order to support SFO Museum’s use case we would need to bundle all of the code required to implement the steps described above with the go-www-geotag application. That’s a lot of functionality which may not be germane to another user of the application. It’s also, potentially, a lot of code that SFO Museum may not want or be able to share publicly. It’s a scenario that would have to be repeated for every custom writer adding unnecessary complexity and size to the final geotagging application.
It would be much better to move the SFO Museum specific code, which we’ll refer to as the WhosOnFirstGeotagWriter writer, in to its own package complete with its own instance of the go-www-geotag application.
In order to facilitate this approach we’ve tried to structure the code used to configure and start a geotagging application, in the go-www-geotag package, in to discrete libraries and re-usable building blocks that can make developing a custom application quick and easy.
Here’s an abbreviated example of the application code from the go-www-geotag-whosonfirst package which is almost identical to the application code in the go-www-geotag package:
import ( "context" "github.com/sfomuseum/go-flags" "github.com/sfomuseum/go-www-geotag/app" _ "github.com/sfomuseum/go-www-geotag-whosonfirst/writer" "net/http" ) func main() { fl, _ := app.CommonFlags() flags.Parse(fl) flags.SetFlagsFromEnvVars(fl, "GEOTAG") ctx := context.Background() mux := http.NewServeMux() app.AppendAssetHandlers(ctx, fl, mux) app.AppendEditorHandler(ctx, fl, mux) app.AppendProxyTilesHandlerIfEnabled(ctx, fl, mux) app.AppendWriterHandlerIfEnabled(ctx, fl, mux) s, _ := app.NewServer(ctx, fl) s.ListenAndServe(ctx, mux) }
In fact, the only difference is the addition of the import statement for the go-www-geotag-whosonfirst/writer package which registers the WhosOnFirstGeotagWriter writer implementation:
import (
_ "github.com/sfomuseum/go-www-geotag-whosonfirst/writer"
)
The mechanics of the Go programming language dictate that sometimes you will need to rewrite an application from scratch in order to add custom functionality. It can be frustrating at moments but we think the overall benefits that Go affords outweigh its few shortcomings and in between we’ve tried to make those rewrites as simple and painless as possible.
In order to start this custom geotagging application with support for the WhosOnFirstGeotagWriter writer you do something like this:
$> cd go-www-geotag-whosonfirst
$> go build -mod vendor -p bin/server cmd/server/main.go
$> ./bin/server \
-nextzen-apikey {NEXTZEN_API_KEY} \
-enable-placeholder \
-placeholder-endpoint {PLACEHOLDER_API_ENDPOINT} \
-enable-oembed \
-oembed-endpoints 'https://millsfield.sfomuseum.org/oembed/?url={url}&format=json'
-enable-writer \
-writer-uri whosonfirst://?reader=fs:///usr/local/data/sfomuseum-data-collection/data&writer=stdout://
2020/04/14 17:02:45 Listening on http://localhost:8080
Different writer.Writer instances are created using a URI-based syntax. The syntax for the StdoutWriter writer described in the last post is stdout://.
The syntax for the WhosOnFirstGeotagWriter writer described in this post is whosonfirst://?reader={READER}&writer={WRITER}. For example:
whosonfirst:// \ ?reader=fs:///usr/local/data/sfomuseum-data-collection/data \ &writer=stdout://
The reader={READER} and writer={WRITER} parameters are themselves URIs for creating abstract reader and writer implementations specific to the Who’s On First project, the details of which are out of scope for this post. In this example we’re saying “read WOF data from a local directory” and “write WOF data to the console”.
When you load the application in a web browser it looks and behaves exactly the same as before. Here’s a screenshot of the application being used to geotag a photograph of Hubert Latham flying over downtown San Francisco, in 1911:
When the Save button is pressed the application “writes” the following data:
{ "id": 1511942249, "type": "Feature", "properties": { "geotag:angle": 48.77566073467381, "geotag:bearing": 76.00314592510365, "geotag:camera_latitude": 37.79147283080962, "geotag:camera_longitude": -122.40383148193361, "geotag:distance": 19737.780983653265, "geotag:target_latitude": 37.83418923027191, "geotag:target_longitude": -122.1858472295593, "src:alt_label": "geotag-fov", "src:geom": "sfomuseum", "wof:id": 1511942249, "wof:repo": "sfomuseum-data-collection" }, "geometry": {"type":"Polygon","coordinates":[[[-122.40383148193361,37.79147283080962],[-122.16144059594401,37.75608316926981],[-122.21025386317457,37.91229529127401],[-122.40383148193361,37.79147283080962]]]} } { "id": 1511942249, "type": "Feature", "properties": { "date:cessation_lower": "1911-12-31", "date:cessation_upper": "1911-12-31", "date:inception_lower": "1911-01-01", "date:inception_upper": "1911-01-01", "edtf:cessation": "1911-12-31", "edtf:date": "1911", "edtf:inception": "1911-01-01", "geom:area": 0, "geom:bbox": "-122.403831,37.791473,-122.403831,37.791473", "geom:latitude": 37.79147283080962, "geom:longitude": -122.40383148193361, "geotag:angle": 48.77566073467381, "geotag:camera_latitude": 37.79147283080962, "geotag:camera_longitude": -122.40383148193361, "geotag:target_latitude": 37.83418923027191, "geotag:target_longitude": -122.1858472295593, "iso:country": "XX", "lbl:latitude": 37.79147283080962, "lbl:longitude": -122.40383148193361, "millsfield:category_id": 1511214215, "millsfield:collection_id": 1511214207, "millsfield:subcategory_id": 1511213363, "mz:hierarchy_label": 1, "mz:is_current": -1, "sfomuseum:accession_number": "2010.174.243", "sfomuseum:category": "Philately", "sfomuseum:collection": "Aviation Archive", "sfomuseum:creditline": "Gift of Charles Page", "sfomuseum:date": "1911", "sfomuseum:daterange_end": "1911-12-31", "sfomuseum:daterange_start": "1911-01-01", "sfomuseum:description": "Black and white photographic postcard depicting early aviation aircraft in flight over San Francisco, facing east toward Ferry Building and San Francisco Bay; photo taken from Nob Hill, identified as “Latham in his flight over San Francisco, January 7, 1911”; typed message from H. N. Cook Belting Company to Union Gas Engine Company. ‘Latham’ is French aviator Hubert Latham.", "sfomuseum:media_ids": [ 1527855487, 1527855491 ], "sfomuseum:medium": "paper, ink", "sfomuseum:object_id": 109526, "sfomuseum:placetype": "object", "sfomuseum:primary_media_id": 1527855487, "sfomuseum:subcategory": "Postcard", "src:geom": "sfomuseum", "src:geom_alt": [ "geotag-fov" ], "wof:belongsto": [ 1511214207, 1511214277, 102191575, 102087579, 102527513, 1511942249, 85688637, 1511214215, 1511213363, 85633793, 85922583 ], "wof:breaches": [], "wof:country": "XX", "wof:created": 1579906665, "wof:depicts": [ 1108830801, 85688637, 85633793, 85922583 ], "wof:geomhash": "b354ab5a6d3fbaff36ca2189db675b75", "wof:hierarchy": [ { "arcade_id": 1511213363, "building_id": 1511214277, "campus_id": 102527513, "concourse_id": 1511214215, "continent_id": 102191575, "country_id": 85633793, "county_id": 102087579, "locality_id": 85922583, "neighbourhood_id": -1, "region_id": 85688637, "venue_id": 1511942249, "wing_id": 1511214207 } ], "wof:id": 1511942249, "wof:lastmodified": 1588278170, "wof:name": "postcard: downtown San Francisco, Hubert Latham’s Antoinette", "wof:parent_id": 1511214277, "wof:placetype": "venue", "wof:repo": "sfomuseum-data-collection", "wof:superseded_by": [], "wof:supersedes": [], "wof:tags": [] }, "bbox": [ -122.40383148193361, 37.79147283080962, -122.40383148193361, 37.79147283080962 ], "geometry": {"coordinates":[-122.40383148193361,37.79147283080962],"type":"Point"}, }
This example is actually very long so it’s been put it in a box with a fixed height and a scrollbar. As you scroll down you can see how and where the default GeoSON data has been transformed and merged with the primary and alternate WOF documents for the photo that has been geotagged.
Both records in the example above contain data that entirely is new, and previously unknown to the go-www-geotag application. Where did it come from? How did the writer know to associate the "id": 1511942249 or "wof:id": 1511942249 properties with the photo? In part 5 of this series I wrote that:
[W]e’ve updated the SFO Museum oEmbed endpoint to include some additional non-standard properties for images…
Most of these additional properties are there to help the go-www-geotag application automatically set the camera position and its field of view for previously geotagged photos. They are optional with the exception the geotag:uri property which is mandatory if you’ve enabled writers in the application.
For example, here’s what the oEmbed response for the photo of down San Francisco looks like:
$> curl -s "https://millsfield.sfomuseum.org/oembed?url=https://millsfield.sfomuseum.org/objects/1511942249/" | jq { "version": "1.0", "type": "photo", "width": "800", "height": "502", "title": "postcard: downtown San Francisco, Hubert Latham’s Antoinette", "url": "https://millsfield.sfomuseum.org/media/152/785/548/7/1527855487_nRYhI6KyRE23XWM2oOPmshKDJvjwzPai_c.jpg", "author_name": "SFO Museum", "author_url": "https://millsfield.sfomuseum.org/objects/1511942249/", "provider_name": "SFO Museum", "provider_url": "https://millsfield.sfomuseum.org/", "geotag:geojson_url": "https://millsfield.sfomuseum.org/data/1511942249/", "geotag:uri": "151/194/224/9/1511942249.geojson" }
In part 6 of this series I wrote that:
The writer.Writer interface defines a single method, called WriteFeature that takes three input parameters … the second of which is a unique identifier.
The value of the geotag:uri property in the oEmbed response is that unique identifier. How that identifier is defined is a detail left to the publisher of the oEmbed response. How that identifier is interpreted is a detail left to the writer that consumes it, in this case the WhosOnFirstGeotagWriter writer.
Just like the example in the last post we’re still just writing data to a terminal window but this time we’re writing different data. In the next post we’ll describe how to publish that data remotely.