Blog posts tagged golang
On the surface this is a blog post documenting the steps to add a new record (an airport) to a catalog of geographic places (the sfomuseum-data-whosonfirst GitHub repository). Scratching the surface, though, it’s really a blog post about how SFO Museum supplements and extends the Who’s On First to meet the needs of our online efforts.
These third-party services that we use offer many benefits but too often we forget that they are not necessarily built for for longevity. Importantly it’s not necessarily their responsibility either. So long as there is a way for SFO Museum to export the things that it posts on a service we can and should take on some of the burden of preserving those efforts for posterity. That is, after all, the business of museums and libraries and archives.
The goal of the “Accession Numbers” project is to compile a catalog of machine-readable patterns for identifying and extracting accession numbers in arbitrary bodies of text for as many museums and cultural heritage organizations as possible.
While this particular instantiation of the geotagging application is scoped to the physical boundaries of SFO the code itself is not specific to SFO Museum. It can be used, in combination with custom databases of map tiles and geographic data, with any image on your computer. This code will continue to be the common infrastructure that we build a SFO Museum application, specific to our needs, on top of.
The bad news is that, when you look closely, there really are that many moving pieces when it comes to something like geotagging photos. The good news is that nearly all of those moving pieces, from the underlying data to the tools to operate on those data, are within the reach of cultural heritage as low-cost and open-source alternatives to the commercial offerings. We may still need to stitch those pieces together to meet the needs of a specific institution but at least they are within reach now.
When I save an image of this view of the runways at SFO, from 2017, the newly created image’s EXIF data contains not only the date and the permanent URL for the map but also the latitude and longitude coordinates for the map’s center point.
One question that’s been raised about the camera/save button in zoomable images is whether or not the new image contains, or preserves, existing EXIF metadata information stored in the original file. The answer yesterday was: No. The answer today is: Not yet, but only because we haven’t enabled it and we will do that soon.
Have you ever wanted to be able to reverse-geocode a point not just in space but also in time? Have you ever wanted to do that date-filtering with fuzzy or imprecise dates, encoded using the Extended DateTime Format (EDTF) ? Have you ever wanted to do both of these things with an arbitrary subset of location records? Have you ever wanted to be able expose these things as a web application and an API that doesn’t need to talk to a remote database? Have you ever wanted to be able to deploy those applications both locally and as serverless applications running on a cloud-provider’s infrastructure? Now you can.
The EDTF specification does all the work of defining the rules and semantics for encoding complex and ambiguous dates in to well-defined and structured strings and the go-edtf packages do the work of decomposing those strings in to values and flags that can be manipulated by computers.
It fosters a practice of actively requesting backups of our activity on these services, as opposed to relying on a mysterious automated system running in the background. I also like that it mirrors our own practice of building services and functionality, like the Mills Field website, from the same open data that we publish for other people to use.
In that spirit this blog post is about four small command-line utilities we’ve written to do our work and are sharing with the wider cultural heritage sector. These tools aren’t specific to SFO Museum and might be useful or helpful to other organizations using a similar infrastructure as ours.
In the same way that we can wrap a traditional web application in a Go program, can we wrap that Go program in a native macOS application? Each platform has its own unique affordances and tolerances. A larger goal for the museum is recognizing the possibilities that each platform affords so that we might be able to treat them as a kind of “kit of parts” to be reconfigured as needed for future projects.
This post is about the go-www-geotag-sfomuseum application. It adds support for authenticating users and publishing data to any service that supports the OAuth2 standard. For the purposes of this post we’re using GitHub to demonstrate our work because we already publish all our open data on GitHub so it is useful for our geotagging application to be able to write directory to their API. In the future we might update our geotagging application to talk to SFO Museum’s own OAuth2 and API endpoints.
In order to support SFO Museum’s use case we would need to bundle all of the code required to implement the steps described above with the go-www-geotag application. That’s a lot of functionality that which not germane to another user of the application. It’s also, potentially, a lot of code that SFO Museum may not want or be able to share publicly. It’s a scenario that would have to be repeated for every custom writer adding unnecessary complexity and size to the final geotagging application.
So far, we’ve got a map and a camera, a global search endpoint, a simple way to query for and display images and enough information to display already geotagged images correctly. Importantly none of these things are specific to our museum or any other institution. In fact there’s nothing about go-www-geotag that is specific to the cultural heritage sector at all. You could use this tool to geotag any set of images. But what about saving the geotagging information that the Leaflet.GeotagPhoto extension produces?
I call this the “building with two-by-fours” stage of development to highlight the importance of not just building things quickly but equally being able to disassemble and rearrange them just as easily. Sometimes this happens at the expense of elegance and finish but that doesn’t diminish their importance or a commitment to address those things.
In the end we may deploy this application for staff as a hosted website on the internet but we would like to have the ability and the flexibility for staff to also run the application locally, from their desktop. The majority of museum staff are not developers and won’t know how or be able, or want, to install the external dependencies that might be necessary for an application written in another programming language to run.
Longer-term, and importantly, it also means the workflows we develop aren’t inextricably bound to Amazon services. Knowing that we don’t have to use AWS and knowing that there is an alternative avenue for accomplishing the same work in the future, should we ever need it, goes a long way towards making it easier for us to want to use AWS in the present.
I am happy to announce that go-iiif version 2.0 has been released. The biggest change in this release is that go-iiif no longer requires the libvips image processing library, by default. As of version 2.0 go-iiif can do all its image processing using native (Go) code. The absence of external dependencies means that go-iiif tools can be compiled in to standalone applications that can be run even if Go isn’t installed on the same computer.
At the end of that first blog post about go-iiif we wrote “An ideal scenario is one where a museum could upload a set of full-sized images to a AWS S3 bucket, wait for Amazon’s computers to process each image … and then find a new set of images to download (along with a reasonable bill for services rendered) in a different S3 bucket.” Today, that is possible.
This is a technical blog post about map tiles, caching, third-party services, so-called “serverless” computing and sustainability. It’s also about improvements to open-source software for managing all of that stuff.
Maybe these maps are simply a fail-safe and only used when nothing else works. Their value then comes from giving us the confidence to try a more sophisticated approach while still having a way to get home safely, so to speak.
This is a technical blog post about image processing. The short non-technical summary is that not only were we able to use open source software to simplify our image processing workflow (and reduce costs) but we contributed our improvements back to the project so that hopefully others in the museum sector may benefit from our work. Yay!