Making the SFO Museum geocoder less Who's On First specific, reusable JavaScript components and experimental vector search

Towards the end of the last blog post about SFO Museum’s Who’s On First-focused geocoder, I wrote:

In the case of indexing Getty Thesaurus of Geographic Names (TGN) records there is little likelihood of ID collision with existing Who’s On First (WOF) IDs, for example if we created a database containing both TGN and WOF records, but it highlights ongoing considerations about the need to change some things about the internal data representation, like the need to move away from numeric identifiers and towards something like machinetag-based identifiers. (If you’re interested in questions like this then Bill Dolan’s recent blog post The Siren Song of Global Identifiers is worth reading.)

Although the geocoder is a tool is primarily targeted at Who’s On First-shaped documents, internally those documents are transformed in an abstract representation which now looks like this:

type Record struct {
	// Id is the unique identifier of the place.
	Id string `json:"geocoder:id"`
	// ParentId is the unique identifier of the parent place.
	ParentId string `json:"geocoder:parent_id"`
	// Name is the primary name of the place.
	Name string `json:"geocoder:name"`
	// Country is the ISO 3166‑1 alpha‑2 country code of the place.
	Country string `json:"wof:country"`
	// Placetype is the primary Who's On First placetype of the place.
	Placetype string `json:"wof:placetype"`
	// PlacetypeAlt contains any alternative placetypes stored in
	// the Who's On First `wof:placetype_alt` property.
	PlacetypeAlt []string `json:"wof:placetype_alt"`
	// Hierarchies contains the ancestor hierarchies for the place.
	// Each hierarchy is a map of placetype to ancestor ID.
	Hierarchies []map[string]string `json:"geocoder:hierarchies"`
	// Centroid is the geographic centroid of the place.
	Centroid *orb.Point `json:"geo:centroid"`
	// Bounds is a slice of bounding boxes that enclose the place.
	Bounds []orb.Bound `json:"geo:bounds"`
	// Inception is the EDTF representation of the start date of the place.
	Inception string `json:"edtf:inception,omitempty"`
	// Cessation is the EDTF representation of the end date of the place.
	Cessation string `json:"edtf:cessation,omitempty"`
	// PopulationRank is an integer that indicates relative population size.
	PopulationRank int64 `json:"wof:population_rank,omitempty"`
	// IsCurrent indicates whether the place is current (1), not current (0)
	// or unknown (-1).
	IsCurrent string `json:"mz:is_current,omitempty"`
	// Tokens contains tokenised names and concordances indexed for full‑text search.
	Tokens map[string]map[string][]string `json:"tokens,omitempty"`
	// VectorEmbeddings holds pre‑computed embeddings for the place.
	VectorEmbeddings []*VectorEmbeddings
}

Since writing that last blog post the Record data structure no longer assumes numeric identifier records for place records but instead allow arbitrary string-based identifiers. As a practical matter that means Who’s On First identifiers take the form of wof:id=101751119 instead of 101751119 and means that applications consuming data from geocoder will need to parse the string tag to derive the numeric Who’s On First identifier. Some thought was given to enforcing the use machinetag-style “triples” for all identifiers but that seems unnecessary or, more accurately, it feels like a decision that the software doesn’t need to make for you.

This is a small change on the outside but a big change on the inside. The underlying database still uses numeric identifiers, keeping an internal mapping of the string identifiers, in an effort to keep the overall size of the database as small as possible. These changes result in a 20-25% increase in the overall size (8GB instead of 6.5GB) but that feels like an acceptable trade-off for the increased flexibility of string-based identifiers.

The hope is that this change will make it easier to use the geocoder tool for other data sources without the need to maintain a separate lookup tables of string-to-numeric identifiers. We like using Who’s On First as a source for geocoding places but there are lots of available data sources out there and these changes were made with the goal of accommodating them. For example, querying OpenPOIs records labeled “museum”:

JavaScript components

We have also published a collection of reusable JavaScript components to display modal dialogs for performing geocoding queries targeting geotagging (deriving coordinates for a place name) and georeferencing (deriving labeled place identifiers for a place name) uses. There is comprehensive documentation for all the components but the short version, in the form of four JavaScript functions, is:

• sfomuseum.geocoder.query – Send a request to the sfomuseum geocoder API; specifically to the API provided by the wof-coarse-geocoder-server tool.
• sfomuseum.geocoder.georeference – Open a modal dialog that allows a person type a place name, pick a result and attach a label to that place, returning both the calling application.
• sfomuseum.geocoder.geotag – Open a modal dialog that only allows a person to pick a place (no label), returning that place to the calling application.
• L.control.geocoder – Add a Leaflet control to a map which opens the geotag dialog when clicked.

The demo geocoder application we’ve posted online has been updated to include demonstrations of the geotag dialog (and Leaflet map control):

As well as the georeference dialog:

Remember: This demo application does NOT contain records for the entire planet. It only indexes the slice of places contained in the sfomuseum-data-whosonfirst repository. Here are some screenshots of georeferencing application. The application launches with a single button for adding a new georeference:

Clicking the button will open a modal dialog where you can enter a label and a query term for a place name. Entering a name will trigger a call to the sfomuseum.geocoder.query method. Results will be written to the select menu below the query input.

When a place is selected the map (in the modal control) will zoom to that place and show its label.

When you click the “Select” button the modal dialog will close and the label and identifier for the place selected will be written to a table.

Multiple labeled georeferenced locations may be added. That’s all this application does. The sfomuseum.geocoder.georeference method takes care of opening and closing a modal dialog, querying the geocoder API endpoint and returning the names of selected places (and their labels) in a JavaScript Promise. For a complete example have a look at this code. The geotagging and georeferencing components do not support all the parameters that can be passed to the geocoder query API, for example scoping queries to a specific placetype or bounding box. These will be added in time and if you have other features you’d like to see we’d love to hear about them.

The individual JavaScript and CSS source files are available from:

Bundled and minified versions of those source files are available from:

Vector embeddings search

Finally, there is still-experimental support for storing and querying vector embeddings for place names in geocoding databases. This is enabled by passing the -embeddings-index flag to the wof-coarse-geocoder-index tool and/or the -embeddings-search flag to the wof-coarse-geocoder-query tool. As of this writing a single embedding is generated for the unique set of names for each (language + language tag) pair for each record. Is this the best way? I don’t know. Because it takes much longer to generate and store a lot of embeddings, and because Who’s On First records often have a lot of different names (and languages), it seemed like a reasonable compromise just to prove that storing and querying vector embeddings was feasible. There is more work, and more investigating, to do.

Here’s an example of what querying a vector-enabled geocoding database for places matching “sfba” (the acronym for the “San Francisco Bay Area”) looks like:

$> ./bin/wof-coarse-geocoder-query \
	-embeddings-search \
	-embedder-uri 'ollama://' \
	-geocoder-uri 'sql://sqlite?dsn=us-vec384.db' \
	-query 'airport sfba' \
	-per-page 10

2026/08/21 10:16:40 INFO Query results total=29 page=1 pages=3

rank			id			label										placetype		latitude	longitude	is current	inception	cessation
0.5294566750526428	wof:id=102527513	San Francisco International Airport, San Francisco, California, US	campus			37.61799	-122.370943	1		1948~		..
0.5966955423355103	wof:id=102530873	Yuba County Airport, Olivehurst, California, US				campus			39.097801	-121.57	1				
0.606959342956543	wof:id=102528839	San Carlos Airport, San Carlos, California, US				campus			37.511902	-122.25	1				
0.6179097890853882	wof:id=102527337	Santa Barbara Municipal Airport, Santa Barbara, California, US		campus			34.427974	-119.837133	1				
0.6228018999099731	wof:id=102527529	Norman Y Mineta San Jose International Airport, California, US		campus			37.363728	-121.928755	1		2001-11	..
0.641245424747467	wof:id=404517201	Airport Township, Missouri, US						localadmin		38.741639	-90.359883	1				
0.641245424747467	wof:id=85926473	Airport, California, US							locality		37.632083	-120.979923	1				
0.641245424747467	wof:id=420539489	Airport, Philadelphia, Philadelphia, Pennsylvania, US			neighbourhood		39.885787	-75.213489	1				
0.641245424747467	wof:id=1729434409	Airport, Missouri, US								locality		38.750377	-90.363144	1				
0.6423512697219849	wof:id=102528473	Flabob Airport, Jurupa Valley, California, US				campus			33.9894	-117.40997	1

One thing you’ll notice is that both the index and query applications specify an “embedder”, as in an external source for generating vector embeddings for the text that is stored in the geocoding database as well as the text used to query for places. Under the hood, the geocoder software is using the sfomuseum/go-embeddings package to abstract those sources (providers) behind a common interface. There have been some recent updates which I think will help further hide these kinds of details and extra steps but until then the easiest thing is to use the vector-enabled geocoder in concert with an easy-to-install application like Ollama.

Querying vector embeddings in the “demo” server is NOT enabled by default. This functionality depends on the presence of the ngxson/wllama Javascript library and the WebAssembly binary in addition to the bert-bge-small/ggml-model-f16.gguf (large language) model. All of these assets could be loaded remotely but one of the design criteria for the API/demo server is that all its assets are bundled locally. The wllama and bert-bge-small assets are not bundled with this repository to prevent unnecessary bloat; there is a handy Makefile target for downloading them locally.

The ngxson/wllama package provides WebAssembly bindings for the llama.cpp library which, in turn, enables the ability to create vector embeddings client-side in a web browser which is kind of bonkers amazing when you think about it. The WebAssembly binary still depends on a third-party model to derive embeddings. The ngxson/wllama package uses the bert-bge-small/ggml-model-f16.gguf model in its examples and is only 69MB (rather than, say, 10 or 20GB) so that’s what the geocoder demo uses too. At least for the time being.

Vector-enabled search is unlikely to replace traditional geocoding approaches any time soon. Querying for “mont royal” returns Montreal. Querying for “khmer rouge” returns Cambodia and Hồ Chí Minh City, in Vietnam, which is not incorrect historically. Anything else may get weird because large language models are weird. For example, searching for “ham and cheese” and getting back results for towns called “Sandwich” is… kind of not-wrong? Either way, it suggests some interesting possibilities going forward.