Using Coyote to Describe the World

Published paper: Using Coyote to Describe the World

In November of 2015, the Museum of Contemporary Art Chicago (MCA) launched a website with a rare accessibility feature. The website team had committed to making all of the images on the site accessible to the widest possible audience—particularly visitors with vision impairment—through visual description. To do so, the MCA worked with Sina Bahram and his team at Prime Access Consulting to develop a workflow tool, backing service, and API to support a distributed description workflow. The description project, Coyote, is an outstanding illustration of the principles of universal design, which argue that products designed to be useful to one community (in this case, people who are blind or have low vision) are likely to benefit a variety of users, not just those with disabilities. Invented to solve an unresolved need at the MCA and within the community, Coyote has in a short time expanded well beyond its original scope, bringing together a team of expert and poetic describers, passionate accessibility advocates, and open source developers and bringing needed attention to the ways that visual description might become a useful aspect of museum practice.
In addition to discussing the policy, institutional, and technical implications around large-scale image description, (including both short and long descriptions), we also plan on presenting the latest enhancements, made possible by a museum technology grant from the Knight Foundation, to Coyote. These include a greatly enhanced representational model that can track visual descriptions of real world objects in addition to images, integration of concepts from the semantic web to facilitate rich search, the development of an organizational model that is used to offer a centralized Coyote instance to multiple institutions in a cloud-hosted version of the software, and enhancements to the Coyote API to facilitate broader third-party access such as Coyote being used for a treasure-hunt-like game based on visual descriptions.

Bibliography:
Full citation list to accompany the paper, but we are aware of the decades of research that has gone into image description. This includes modern guidelines; such as, those published by Art Beyond Sight, work from Benetech, WGBH, the Diagram Center, and Smith-Kettlewell, just to name a few. Furthermore, in the context of digital accessibility, our work is deeply rooted in, and implements, the recommendations proposed by the W3C in the WCAG 2.0 guidelines.