A new geocoder for Who's On First (and TGN) data

With few exceptions, most geospatial tools, tools for working with place and space, are linear affairs. There is only the perpetually updating present. It is often said of geospatial data that if it’s not being updated it is “dead”. This idea is a reflection of the ongoing cost and effort to maintain geospatial data at the scale of the planet. Many, if not most, of the organizations that choose to take on the task are commercial in nature and pesky little things like “the past” are typically understood as a burden. And it’s true: Most people aren’t looking for things in the past (often because they don’t know it’s possible but that’s a question for another day). Cultural heritage organizations, on the other hand, are looking for things in the past as much, if not more so, than they are looking for things in the present. For example, searching for named places is often referred to as “geocoding”.

I’ve written about geocoding in the past, usually in the context of using the Placeholder geocoder. Placeholder, a “coarse” or not-address-level geocoder, is an open source project that uses the openly-licensed Who’s On First dataset. Recently, SFO Museum decided to write its own coarse geocoder, modeled after Placeholder, and written in Go. It is called “geocoder”.

This package contains tools for indexing geocoding databases and querying them from the command line or a simple HTTP API endpoint. The functionality is minimalist by design. It is just an interface to search for places via a query string returning Who’s On First-style place records with stable identifiers, coordinate data (centroid) and a display label. Query strings may also be filtered by the following:

• One or more 2-letter ISO country codes.
• One or more Who’s On First placetypes or custom wof:placetype_alt placetypes.
• Custom bounding box to scope the query (results) to.
• 3-letter language code for a place name.
• Who’s On First language “tag” used to signal Who’s On First classifications for a place name (for example: preferred, variant, historical, etc.)
• Start and end dates represented by ExtendedDateTimeFormat (EDTF) date strings.
• One or more Who’s On First IDs which are ancestors (contain) result candidates.

As of this writing there is no semantic “parsing” of query strings. For example, if you pass in the query “Los Angeles CA” the code does not attempt to scope query results to the state of California. Geocoding remains a special kind of quicksand and not something SFO Museum is looking to wade into right now. The goal in building the geocoder tools was to match or exceed Placeholder results, enable additional query filters (described above) and to be written in Go. While there are still many query strings that will trip up or confuse the geocoder tools; so far it has been right, or “right enough”, most of the time.

Indexing

You use the wof-coarse-geocoder-index tool to create, or “index”, a new geocoding database. Any Who’s On First style dataset can be indexed. For example, this is how you would index the sfomuseum-data-architecture and sfomuseum-data-whosonfirst data repositories:

$> ./bin/wof-coarse-geocoder-index \
	-fresh \
	-exclude-superseded=false \
	-iterator-uri repo:// \
	-geocoder-uri 'sql://sqlite?dsn=sfom.db' \
	/usr/local/data/sfomuseum-data-architecture \
	/usr/local/data/sfomuseum-data-whosonfirst
	
...time passes

2026/08/08 11:21:34 INFO Post-indexing complete time=661.16725ms "time (total)"=6m29.676154166s

Note the -exclude-superseded=false flag. This will be relevant to how we query the database below. Indexing time can depend a lot on the data source. Files on disk (above) can take a while. Indexing Who’s On First Parquet files (produced by the wof-parquet-export tool in the whosonfirst/go-whosonfirst package) is significantly faster, taking only 20-30 minutes to create a geocoding database for all 6 million administrative records in Who’s On First:

$> ./bin/wof-coarse-geocoder-index \
	-fresh \
	-iterator-uri parquet:// \
	-geocoder-uri 'sql://sqlite?dsn=wof-sfom.db' \
	/usr/local/data/whosonfirst-parquet/whosonfirst-data-admin-*.parquet
	
2026/08/08 11:32:00 INFO Iterator stats elapsed=1m0.000199625s seen=319775 allocated="3.2 GB" "total allocated"="38 GB" sys="4.5 GB" numgc=88
2026/08/08 11:32:00 INFO Indexing stats elapsed=1m0.000307583s seen=319765 "average (ms)"=0.09237721451691086

...time passes

2026/08/08 12:01:46 INFO Post-indexing complete time=10m44.596903375s "time (total)"=30m45.71374225s

Note that these Who’s On First Parquet files, shown in this example, are not available for download from the Who’s On First servers yet so you’ll need to create them manually. SFO Museum might provide alternate downloads in the interim. In the meantime, we have posted a downloadable copy of the wof-sfom.db database (6GB) created in this example.

Querying

To query a geocoding database you can use the wof-coarse-geocoder-query tool. For example, here are all the records matching “T3” in the SFO Museum specific database created above:

$> ./bin/wof-coarse-geocoder-query \
	-geocoder-uri 'sql://sqlite?dsn=sfom.db' \
	-query T3

2026/08/08 11:25:22 INFO Query results total=7 page=1 pages=1

id		name		placetype	latitude	longitude	is current	inception	cessation	label
1947304447	Terminal 3	wing; terminal	37.618362	-122.386773	1		2024-11-05	..		Terminal 3, SFO Terminal Complex, San Francisco International Airport, San Francisco, US
1159157307	Terminal 3	wing; terminal	37.618362	-122.386773	0		2017~		2019-07-23	Terminal 3, SFO Terminal Complex, San Francisco International Airport, San Francisco, US
1477855699	Terminal 3	wing; terminal	37.618362	-122.386773	0		2019-07-23	2020-~05	Terminal 3, SFO Terminal Complex, San Francisco International Airport, San Francisco, US
1729792487	Terminal 3	wing; terminal	37.618362	-122.386773	0		2020-~05	2021-05-25	Terminal 3, SFO Terminal Complex, San Francisco International Airport, San Francisco, US
1745882233	Terminal 3	wing; terminal	37.618362	-122.386773	0		2021-05-25	2021-11-09	Terminal 3, SFO Terminal Complex, San Francisco International Airport, San Francisco, US
1763588269	Terminal 3	wing; terminal	37.618362	-122.386773	0		2021-11-09	2024-06-17	Terminal 3, SFO Terminal Complex, San Francisco International Airport, San Francisco, US
1914600841	Terminal 3	wing; terminal	37.618362	-122.386773	0		2024-06-17	2024-11-05	Terminal 3, SFO Terminal Complex, San Francisco International Airport, San Francisco, US

As mentioned, queries can also be scoped to a date range. For example, records matching “T3” whose start and end dates are between 2019 and 2020:

$> ./bin/wof-coarse-geocoder-query \
	-geocoder-uri 'sql://sqlite?dsn=sfom.db' \
	-query T3 \
	-date-starts 2019 \
	-date-ends 2020

2026/08/14 14:39:04 INFO Query results total=1 page=1 pages=1

1477855699	Terminal 3	wing; terminal	37.618362	-122.386773	0		2019-07-23	2020-~05	Terminal 3, SFO Terminal Complex, San Francisco International Airport, San Francisco, US		2019-07-23	2020-~05	Terminal 3, SFO Terminal Complex, San Francisco International Airport, San Francisco, US

Do you remember how the -exclude-superseded=false flag was passed when the sfom.db database was created? Without it all the records for “T3” not marked as current would have been excluded from the database because each had been superseded by a more current record. For more information on why SFO Museum has multiple records for Terminal 3, and other architectural elements, have a look at the paper Mapping Space, Time, and the Collection at SFO Museum. Here’s another example, taken from a database of Who’s On First data indexed without records that have been superseded, searching for the former country of Yugoslavia:

$> ./bin/wof-coarse-geocoder-query \
	-geocoder-uri 'sql://sqlite?dsn=/usr/local/data/geocoder/wof-sfom.db' \
	-query yugoslavia
	
2026/08/14 15:41:33 INFO Query results total=1 page=1 pages=1

id		name			placetype	latitude	longitude	is current	inception	cessation	label
1209522935	Oficina Yugoslavia	locality	-24.05635	-69.82909	-1						Oficina Yugoslavia, CL

And here are the results for the same query against a different database indexed with records that have been superseded:

$> ./bin/wof-coarse-geocoder-query \
	-geocoder-uri 'sql://sqlite?dsn=wof-with-superseded.db' \
	-query yugoslavia

2026/08/14 15:40:49 INFO Query results total=6 page=1 pages=1

id		name						placetype		latitude	longitude	is current	inception	cessation	label
1209522935	Oficina Yugoslavia				locality		-24.05635	-69.82909	-1						Oficina Yugoslavia, CL
1126113567	Socialist Federal Republic of Yugoslavia	country		44.098986	18.839966	0		1945-11-29	1991-06~	Socialist Federal Republic of Yugoslavia
1108955789	Kingdom of Yugoslavia				country		44.098986	18.839966	0		1918-12~	1945-11-29	Kingdom of Yugoslavia
1108955783	Federal Republic of Yugoslavia		country		43.550779	20.94335	0		1992-03~	2006		Federal Republic of Yugoslavia
1108955785	Socialist Federal Republic of Yugoslavia	country		43.971458	20.197023	0		1991-09~	1992-03~	Socialist Federal Republic of Yugoslavia

For more on how Who’s On First manages dates, and relationships, for historical places, have a look at the blog post Tackling Space and Time in Who’s On First. The geocoder will pass the so-called “Brooklyn test” (at least according to New Yorkers) in English:

$> ./bin/wof-coarse-geocoder-query \
	-geocoder-uri 'sql://sqlite?dsn=wof-sfom.db' \
	-query brooklyn \
	-per-page 5
	
2026/08/09 22:20:13 INFO Query results total=135 page=1 pages=14

id		name			placetype	latitude	longitude	is current	inception	cessation	label
421205765	Brooklyn		borough	40.652256	-73.956582	1						Brooklyn, New York, New York, US
85969229	Brooklyn Park		locality	45.11953	-93.350542	1						Brooklyn Park, Minnesota, US
404511829	Brooklyn Park		localadmin	45.11953	-93.350542	1						Brooklyn Park, Minnesota, US
85807925	Brooklyn Heights	neighbourhood	40.696455	-73.995788	1						Brooklyn Heights, New York, New York, US
85871819	Old Brooklyn		neighbourhood	41.434487	-81.70417	1						Old Brooklyn, Cleveland, Cleveland, Ohio, US

As well as other languages, like “Farsi”:

$> ./bin/wof-coarse-geocoder-query \
	-geocoder-uri 'sql://sqlite?dsn=wof-sfom.db' \
	-query بروکلین \
	-per-page 10
	
2026/08/09 22:22:22 INFO Query results total=72 page=1 pages=8

id		name			placetype	latitude	longitude	is current	inception	cessation	label
421205765	Brooklyn		borough	40.652256	-73.956582	1						Brooklyn, New York, New York, US
85969229	Brooklyn Park		locality	45.11953	-93.350542	1						Brooklyn Park, Minnesota, US
404511829	Brooklyn Park		localadmin	45.11953	-93.350542	1						Brooklyn Park, Minnesota, US
85807925	Brooklyn Heights	neighbourhood	40.696455	-73.995788	1						Brooklyn Heights, New York, New York, US
85871819	Old Brooklyn		neighbourhood	41.434487	-81.70417	1						Old Brooklyn, Cleveland, Cleveland, Ohio, US

Deploying

In addition to the command line query tool there is also the wof-coarse-geocoder-server tool which will start a web server exposing a machine-readable endpoint for querying records and returning results as GeoJSON. The server also has a “demo” flag which will serve a simple web application, relaying query string to its own query endpoint, and displaying those results on a map and in a table view. We’ve set up a demonstration server (of this demonstration server) here:

This demonstration server does NOT support geocoding for the entire planet. It only contains records from the sfomuseum-data-whosonfirst data repository; a subset of all the Who’s On First records that we know involve objects in the SFO Museum Aviation Collection. To illustrate this point, the demonstration server will “fail” the Brooklyn test, described above, since we haven’t associated any records in our collection with the borough (or any of the other Brooklyns in the world) yet. If you want to deploy your own geocoder database and server with global coverage it basically boils down to this: You’ll need to run an actual server somewhere.

It is not practical to deploy this sort of service as a short-lived, on-demand, cloud function like Amazon’s Lambda Function URLs. The dream for services like the geocoder is to be able to store the data itself (6GB) in a cloud-based storage service, like AWS S3, and then query it from a cloud function, like AWS Lambda. While it is actually possible to do this, the response times are so prohibitively slow as to make it impossible. The same is true for bundling the 6GB data file inside a container image and then running that container image as a Lambda function. These are all nice ideas but for one reason or another they aren’t practical yet.

Careful readers will note that the demo server we’ve pointed to is, in fact, running as an AWS Lambda Function URL. We’ll explain a little more about how that works below and it depends on the fact that it is using a very small database. So, to repeat: Anything planet-scale still requires running something server-shaped.

Experiments

Virtual and embedded file systems

The demonstration server, above, actually bundled its entire geocoding database as part of an embedded filesystem in the server application itself. The database itself is approximately 30MB, for a little over 1,600 records, which fits inside the limits that AWS imposes on the size of the applications used as Lambda functions. This points to some interesting possibilities. Not all geocoders need to have global coverage. The ability to deploy purpose-fit and scaled-down geocoding applications to low-cost, on-demand billing services are both attractive and intriguing.

Or it may simply be desirable to bundle a geocoding database with the application that uses it for the sake of simplicity and portability. While not the default behaviour it is possible should you need to do so. Bundling databases in an embedded filesystem will require you to write some code (a small amount of code) manually so consult the documentation for details.

It is also possible to configure the geocoder tools to read data from a database hosted on a remote web server or storage service like AWS S3. This is not always the most responsive option but it is an option. It is absolutely not fast enough for use with AWS Lambda functions, but it will work from your local computer. Here’s an example querying the Who’s On First database that we created earlier (and have made available for download:

$> bin/wof-coarse-geocoder-query \
	-geocoder-uri 'sql://sqlite?vfs-enable=true&vfs-base=https://static.sfomuseum.org/geocoder&vfs-dbname=wof-sfom.db' \
	-query-timeout 15 \	
	-query gowanus
	
2026/08/12 12:51:39 INFO Rewrite geocoder URI to enable VFS uri="sql://sqlite?dsn=file%3Awof-sfom.db%3Fvfs%3Dvfs1%26mode%3Dro"
2026/08/12 12:51:43 INFO Query results total=2 page=1 pages=1

id		name			placetype	latitude	longitude	is current	inception	cessation	label
85865587	Gowanus		neighbourhood	40.678529	-73.987462	1						Gowanus, New York, New York, US
102061079	Gowanus Heights	neighbourhood	40.682373	-73.987939	-1		2012				Gowanus Heights, New York, New York, US

Note the additional query parameters passed to the -geocoder-uri flag as well as the extra-long allowable query time defined in the -query-timeout flag.

Thesaurus of Geographic Names (TGN)

There is experimental support for indexing place records from Getty’s Thesaurus of Geographic Names (TGN). This support manages to map Getty place details on to the internal Record data structure used to store information. Wherever possible TGN placetypes are mapped to their Who’s On First equivalent. When a match is not found that place is assigned a Who’s On First placetype of “custom”. Both parts of a TGN placetype (its numeric identifier and string label) are indexed, separately, as alternate placetypes. The placetypes mappings can be found in x/tgn/placetypes.json. These choices may contain errors or inaccuracies and we welcome your feedback if you think that is the case.

These placetype mappings are also used to construct a Who’s On First style hierarchy. This hierarchy is important because, as of this writing at least, it is what is used to generate a fully-qualified label for a place. Language qualifiers (variant, preferred, etc.) are almost certain to contain mistakes. I do not fully understand (yet) how TGN defines these things so this will probably require some finessing.

In the case of TGN, specifically, there is little likelihood of ID collision with existing Who’s On First (WOF) IDs, for example if we created a database containing both TGN and WOF records, but it highlights ongoing considerations about the need to change some things about the internal data representation, like the need to move away from numeric identifiers and towards something like machinetag-based identifiers. (If you’re interested in questions like this then Bill Dolan’s recent blog post The Siren Song of Global Identifiers is worth reading.)

To create a TGN-specific geocoding database use the wof-coarse-geocoder-index-tgn and a TGN XML data release (consult documentation for details). For example:

$> go run cmd/wof-coarse-geocoder-index-tgn/main.go \
	-geocoder-uri 'sql://sqlite?dsn=tgn.db' \
	-tgn-data ~/Downloads/tgn_xml_0126.zip

...time passes

2026/08/11 23:12:05 INFO Indexing complete time=1h6m16.185558333s

$> du -h tgn.db 
4.5G	tgn.db

Once your database has been created it can be used “as is” with the wof-coarse-geocoder-query and wof-coarse-geocoder-server tools. (We have posted a downloadable copy of the one we made (4.5GB).) For example:

$> ./bin/wof-coarse-geocoder-query \
	-geocoder-uri 'sql://sqlite?dsn=/usr/local/data/geocoder/tgn.db' \
	-query USSR

2026/08/14 13:53:20 INFO Query results total=1 page=1 pages=1

id		name							placetype					latitude	longitude	is current	inception	cessation	label
6006211	Soyuz Sovetskikh Sotsialisticheskikh Respublik	custom; 81508; empire; former nation; state	60		98		-1						Soyuz Sovetskikh Sotsialisticheskikh Respublik

The former USSR is not (yet) part of the Who’s On First dataset but it is part of the TGN dataset.

What’s next

Not much, to be honest. There are a couple of novel features which are being investigated but the immediate next steps are to let things “bake” for a while and to understand what works and what doesn’t by using these tools for a while. There are almost certainly query, ranking and performance improvements to be made. If you spot any of them we’d love to hear from you about that. All of the code for these tools is available as open-source software from our GitHub account: