Coloring Books Pages For the SFO Museum Aviation Collection

We have launched a new experimental feature on the SFO Museum Aviation Website: Coloring book pages (or sheets) for a subset of the objects in our collection. As of this writing those items are model airplanes, scarves, airline bags and a small number of other objects that were added during testing.

You can see all of the objects with coloring book pages at:

Objects with coloring book pages are displayed with a icon underneath their title. Clicking on the icon will open a PDF file in a new window with a stylized, black and white outline of that object for you to print out and color as you see fit. For example, the coloring book page for this object:

Looks like this:

Each coloring page also has a QR code linking to that object on the SFO Museum Aviation Collection website. As mentioned earlier this is still considered experimental work which means not all coloring book sheets are as good as others. Specifically: Sometimes the outlines are too dense and sometimes the outlines are missing enough detail. We’ve also seen some sheets where the object itself is drawn smaller than it should be. All of these things will be addressed in time but we are excited to tell you about this feature so that you and/or your children can get busy coloring!

In the Updating the SFO Museum Wayfinding Service - Publications blog post I wrote:

Custom publications are a first attempt at designing and producing artifacts which can be thought to “follow a visitor out of the building” (or the museum (which also happens to be the airport)). Think of them as training wheels towards achieving that goal. We believe that the objects and public art works on display at SFO Museum and SFO are worth seeing in person but that doesn’t, and shouldn’t, preclude the ability to enjoy these things when you are not at SFO.

Coloring book pages should be thought of as a second attempt at producing a similar artifact that can follow you “out of the building”. In practical terms because there aren’t public printer stations at the airport and because most people don’t travel with a portable printer these artifacts are likely to be printed before a person even gets to “the building” (the airport) but you get the idea. These coloring book sheets can be downloaded to tablet devices and opened (and colored) in a variety of applications so if you are looking for something to do while you wait for your flight once you’ve arrived at the airport, or something to do while you’re in the air, these can be a fun and creative activity.

The rest of this blog post gets in to some of the technical details for how these coloring book sheets are produced. If that’s not of interest or if you just want to start coloring you can stop reading now and start downloading coloring book sheets at:

The technical details

Generating the outlines for coloring book sheets is a three-part process: First an image is simplified using the visioncortex vtracer command line tool, which produces an SVG file. Second, that SVG file is turned back in to a raster image using the Batik SVG rasterizer. Finally, the outline is derived from that (second) image using Michael Fogleman’s contourmap application and saved as a third and final image which is embedded in a PDF document.

vtracer is written in the Rust programming language, Batik in Java and contourmap in Go. Because we write most of our code in Go, including the code that produces and then publishes the final PDF files we include on the collection website it would be convenient if the vtracer tool could output a raster image and if it could be called from library code in Go. At a glance adding code to the vtracer library to rasterize its own SVG output shouldn’t be too hard but I have only done a little bit of Rust programming and I’ll confess I got a bit lost trying to understand the documentation for the resvg rasterizer. Likewise, while it appears possible to call Rust libraries from Go it also looks sufficiently involved that it risks becoming a wild goose chase. Any help, or suggestions, with either of these things would be welcome. The original version of this workflow used an SVG rasterizer written in Go but it was often very (very (very)) slow. Batik has been actively developed for nearly two decades, remains the gold standard for SVG rasterization and is fast so it wasn’t a hard decision to adopt it until the vtracer tool can be updated to generate raster images.

That’s a lot of moving pieces but it is possible to bundle them all in to a single Docker container which can be run from a containerized AWS Lambda function so we can generate (and more importantly regenerate) coloring book sheets on demand quickly and inexpensively. Because of the complexities described we have not released any general-purpose tooling for this workflow but we have published the code we use for people to consult as a reference, in particular the steps to derive an outline for any given image:

Early on in the process we experimented with using the image-emboss tool that I wrote about in the Extracting Subjects from Images in Swift (and gRPC) blog post to do image segmentation to only derive outlines for the principal “subject” in an image. This was to account for the fact that while most of our collection images have backgrounds that are “just grey” they still have enough variation in them that, when they are used to derive contours, they produce a lot of distracting artifacts.

In the blog post about the image-emboss tool I wrote:

Note that the underlying “subject lifting” code merges two objects in to one and excludes four others entirely. This is a good reminder that as impressive as the technologies used to perform these tasks are they remain fallible and the reasons they choose to do, or not do, something often remain opaque. While they herald exciting new possibilities they are still tools which should only be used with supervision and a good understanding (or at least awareness) of when and where they may not meet the needs of your specific project.

One of these “opaque decisions” manifested itself with this image of a Braniff Airlines 727 plane which when “embossed” (or “segmented”) is missing an entire wing.

Given that we always planned to include model airplanes in our initial set of coloring book sheets these kinds of errors argued against using image segmentation without more investigation. We also considered using Apple VisionKit’s VNDetectContoursRequest APIs but quickly determined that they have many of, if not the same, problems that image-emboss tool does. For example:

Eventually we settled on using the vtracer tool to flatten each image and accepting that a certain amount of background artifacting would be acceptable and might even present some interesting creative possibilities when outlined.

During this process I spent a little bit of time to determine whether we could use Stable Diffusion’s image to image tool to derive new images to use for creating outlines. For example, would it be possible to create images of specific airplane models or uniforms without any branding or livery? It is worth noting that I didn’t try very hard or train the model on more than a single image but the initial results were not promising. Given this image of a Fiji Airways A320 aircraft:

And the prompt “Model airplane of an unpainted Airbus A330-200 seen from nose to tail on a grey background” the image that was produced was incorrect and incomplete, depicted an aircraft that is almost certainly not airworthy and either a grim prediction or a poignant commentary on modern passenger seating.

So, there is work to do all around. There is work to do to better configure the settings we use to generate outlines using our current workflow in an automated fashion. There is work to do to better understand how we might use machine learning processes which probably means training our own machine learning models to use in that work. As always we welcome any suggestions and improvements and, in the meantime, happy coloring!