Reflections from Art UK on building and operating a digital infrastructure for public art collections across the UK
AbstractIn this Paper Art UK will reflect on its experience building and operating a shared digital infrastructure for over 3,000 public art collections across the UK and supporting these collections through mass digitization initiatives. At a time of funding pressures within the cultural sector, sharing resources and costs between collections makes sense. Whilst geography imposes limits to sharing physical resources, in the case of digital resources, geography is irrelevant. Digital infrastructure is eminently sharable, allowing collections – irrespective of size – to take advantage of scale and technology. The paper will introduce the range of platforms created by Art UK to serve the UK’s public art collections. This includes the Art UK website, the Art Detective crowd-sourced specialist knowledge network (Winner of the Best of the Web Award 2015); the Art UK Shop, a commercial platform that will generate commercial income for participating collections; the Collection Portal that allows collections to upload and update records; the Permissions Portal that allows UK collections to control their image licensing. The paper will describe the national digitisation programmes to record art in public collections starting with the oil painting project (completed 2012) and the recently launched project to digitise the nation's sculpture holdings both inside collections and outdoors in the UK's parks and squares. The paper will provide insights into how such an initiative might possibly be rolled out in other countries. Five challenges in particular will be discussed. 1) How to collaborate at a national level? 2) What legal framework is required? 3) What is the technology solution? 4) How do you attract audiences? 5) How do you make it financially sustainable?
Keywords: infrastructure, collaboration, crowdsourcing, learning, art, sharing
In this paper Art UK reflects on its experience building and operating a shared digital infrastructure for public art collections and supporting these collections through mass digitisation initiatives.
Showing art online creates substantial public and economic benefit. It dramatically improves access to artworks and collections across the UK. It allows audiences to find out about artworks in store whilst encouraging visits to the art that is on display, thereby promoting cultural tourism. It opens up learning opportunities for audiences of all ages, inspires creativity and is vital for research. The digital records provide an audit of condition and whereabouts. Finally, it offers collections valuable commercial-income generating potential.
However, lack of digital image records, funding pressures and limited resources within public collections have meant that few UK institutions outside the nationals and some large regional museums have their art collections on their own websites. At the same time, the vast majority of the nation’s art collection is in store or buildings without routine public access. In brief, what is publicly owned has not been publicly accessible. The work of our charity has focused on transforming public access to the nation’s art collection. We have put in place a comprehensive (technical, administrative, legal and editorial) solution to a key part of the nation’s cultural digital infrastructure challenge.
3. The work of Art UK
The Public Catalogue Foundation (as we were previously called) was launched in 2003. It is a registered charity. It was set up to create a photographic record of all the oil paintings in public ownership. The original idea was for it to be published through a series of county-by-county printed catalogues (84 were published). Photography was completed in 2012.
Cataloguing agreements allowed for the project going online eventually. In 2008 we approached the BBC about a possible online partnership and in 2011 the first digital manifestation of the project was launched on bbc.co.uk as Your Paintings. This proved successful in terms of audience reach but, given the nature of the BBC’s public service remit, did not allow us to do anything commercial on the site, raise funds or even show funders’ names. To allow for a more sustainable business model, Your Paintings was retired in 2016 and replaced with an upgraded site, https://www.artuk.org/ run independently of the BBC but with the BBC remaining an important project partner. At that point Art UK took on its current name.
Fig.1 Art UK Home Page
An algorithmic-based tagging project was launched in 2011 generating over three million tags but is temporarily suspended. This was supplemented by tags generated through image-recognition software developed by Oxford University’s Engineering Science department. We are seeking funding to re-launch Tagger later this year.
Art Detective, a digital knowledge network of crowdsourced specialised information for collections needing to fill in gaps in their knowledge about their artworks was launched in 2014. This won the Museums and the Web ‘Best of the Web’ award in 2015.
The Art UK Shop has been set up to allow collections to generate commercial income from their image assets and sell other museum merchandise. This was launched in pilot in late 2016.
The second major digitisation initiative – with the National Lottery as the lead founder – to record the nation’s sculpture collection has just got under way. Collections are also starting to upload existing digitised works on paper.
The core team comprises a dozen FTE staff. The Sculpture project will see the team expanded by over 30 FTE for a fixed 3-year period.
4. Shared Infrastructures
At a time of funding pressures within the UK cultural sector, sharing resources and costs between collections makes sense. Digital infrastructure is eminently sharable, allowing collections – irrespective of size and geography – to take advantage of scale and technology.
We see substantial potential for UK museums and other public collections to do more to share collection management systems, digital asset management infrastructure, online collection websites, online giving platforms and e-commerce platforms. For public art collections, www.artuk.org is a step in that direction being a digital infrastructure shared by 3,250 art collection venues. Around a third are museums, the others being universities, historic houses (through the National Trust and English Heritage), national collections such as those of the Government Art Collection, hospitals and other public buildings.
Fig 2. Hogarth Tryptich in Bristol City Council Offices
The economic argument for sharing digital infrastructure is beyond refute. All parties benefit from the infrastructure at a substantially lower cost than would have been the case if they had purchased the infrastructure alone. If each of the 3,250 collection venues on Art UK created its own online art collection for £10k it would cost a lot more than £32.5m and that is before running costs are considered.
However, beyond that, it must be stressed that it is only through the use of a shared digital infrastructure that most public art institutions digitised by Art UK are able share their art collections online.
For medium-sized and smaller collections it is rare to find online collection portals and almost impossible to find collections where all their art collection is shown on their website. Funding pressures and lack of relevant in-house expertise has meant that building an online collection portal is just not feasible.
5. Digitisation Challenges
Similarly, lack of resources has meant that only a small proportion of public art collections have digitised their collections. During our UK-wide oil painting digitisation project we used existing images where we could. However, nearly 90% of the paintings were photographed afresh mainly due to the absence of photographic records.
As a result of the oil paintings project, we made over 200,000 artworks digitally available. The high resolution images were given to collections for free.
Ahead of our Sculpture digitisation project a 2015 survey of over 450 collections indicates that 67% of public collections do not have any high-resolution images of their sculpture. The survey also told us that 70% of public organisations do not have any of their sculpture online. Of the 30% that have some online, only 1% claimed to have their entire sculpture collection online. In this ‘digital’ age, 46% of collections reported that they held some or all of their sculpture records on paper.
Our UK oil painting digitisation programme benefited from scale and strong economics and the same will apply to the forthcoming sculpture project. Including all related research, processing, editorial, copyright clearance, management and overheads, the total cost (excluding building the website) was approximately £6m over ten years.
(i) Art UK technical architecture schematic
Fig 3. Art UK Platform overview
(ii) General overview
Art UK is more of a platform than a website. The overall infrastructure (summarised above) has evolved over a number of years and now offers a wide range of services to meet the needs of public users, partner collections, project partners, Art UK staff and a growing set of technology suppliers.
The Art UK platform is increasingly component based with extensive use made of an integration and access layer to share data and services between the different elements.
Fig 4. Interface Layer
The interface layer for Art UK comprises 7 components built over the last 2 years with 3 different suppliers. All components are developed to a common set of style and branding guidelines. With the exception of the Permissions Portal, all interfaces are fully responsive across a range of platforms.
Table 1 Interface Components
|Art UK public interface||User profiles facilitate additional functionality for public users, partner collections, Art UK project partners and technology suppliers.
Art UK makes extensive use of Elasticsearch and uses a range of indexes to allow users to browse, search and retrieve a range of record types (artworks, artists, venues, etc.) at speed.
|Art Detective||Dedicated interface to show discussions, groups, discoveries etc. and links to/from Art UK main website. All content is stored in the Qi CMS.||Keepthinking /
|Collections Portal||Self service portal to allow authorised partner collections to update key content and manage their artworks (edit, upload etc.) and venue details.||Keepthinking /
|Permissions Portal||Used by collections and artists to assign licences to artworks.
Supports a range of licence types including All Rights Reserved, Creative Commons as well as commercial licences for use in Art UK projects.
It is a lightweight web application component that fully integrates with the Art UK content management system (Qi – see below) using the Qi API. All user, collection, artist and artwork data is read from Qi and the system creates new licence records directly in the Qi database.
Licence data for artworks is synchronised using CIIM and Elasticsearch on a daily basis.
(but migrating to Amazon AWS)
|Donate||Donate uses the Internova WebShop framework and includes a set of web-based workflows to facilitate one-off or recurring donations and pledges from benefactors.
Donor details are stored in the WebShop database.
|Art UK Shop||The Shop has been set up as a marketplace service to allow collection partners to sell artwork-related products (print on demand and licences) as well as a range of collection-based merchandise through a single website.
The Shop was built using Internova’s WebShop ecommerce framework. WebShop functionality supports product selection journeys (for print size, framing options etc.) as well as shopping basket, checkout and secure payment processing.
Products, customer data and online transactions are stored in the WebShop database but fully integrated with the Art UK database using Elasticsearch (to identify and import new artworks with commercial licences) and the Qi API (for user identification management).
|Commercial Portal||Authorised Art UK collection partners participating in the Shop use the Commercial Portal to manage products (upload, edit etc.), receive and process orders as well as view the reports dashboard to track KPIs.
The Commercial Portal has a fully-responsive interface and integrates with the underlying WebShop database. User management is controlled through permissions set in Qi.
(iii) Integration & access layer
The integration and access layer facilitates sharing and integration of Art UK data and services across the range of interface components developed over time and by separate suppliers.
Table 2 Integration and Access
|Qi API||A fully featured, read/write API to the Qi content management system (see Qi below).
The Qi API allows third party developers to consume Art UK data (based on permissions granted) as well as write new data to the Qi CMS.
Qi API includes a wide range of types covering all key data entities and associated relationships – artworks, artists, collections, licence types, etc.
The Qi API user management is controlled in Qi.
|CIIM||CIIM (pronounced “sim”) is a third party framework designed to process and aggregate library, archives and collection content for digital publication.
For Art UK, CIIM is configured to extract data from Qi and prepare for indexing using Elasticsearch. CIIM extracts changed data from Qi on a daily basis.
In addition to Qi data, the CIIM is used to aggregate event data from Culture 24 based on the 3,254 venues participating in Art UK.
|Knowledge Integration /
|Art UK Search API||The Art UK search API uses the Elasticsearch full text search engine.
All key landing pages (artworks, artists, venues, stories, topics, events etc. have been designed to work with Elasticsearch indexes that support facet-based search.
Landing pages also make use of an autocomplete suggestion feature.
|Knowledge Integration /
(iv) Content management layer
Art UK uses two key content management systems. The main system (Qi) manages all the collection-based content and the WebShop CMS is used to manage products, orders and payment transactions for the Shop. Where relevant, the two content management systems are integrated to avoid duplication and the integration & access layer (Qi API and Art UK Search API) allows the two systems to share and update content.
Table 3 Content Management
|Qi CMS||Qi (from Keepthinking) is the central content and collection management system for Art UK, Art Detective, Tagger (suspended), Collections Portal and the Permissions Portal.
Qi is fully web-based. Rather than having to maintain separate systems for the collections data, images and the website content, Qi is used to manage all key content types in a single database.
All user registrations are managed in Qi and shared across other interface components.
Qi is also used to manage the online image uploads to Amazon S3 and associated image workflows (image resizing, embedded metadata processing and digital watermarking).
Art UK staff are distributed so having a web-based content management system allows both editorial and data management activities to take place anywhere.
Qi is PHP / MySQL based and built using the CodeIgniter framework.
Access to the Qi API is managed using Qi.
|WebShop CMS||The Shop website is managed using the WebShop CMS developed by Internova.
An integrated workflow (running nightly) identifies artworks in Qi with related commercial licences (for print on demand or licensing) and pulls these records into the WebShop CMS for presentation in the Shop as a products.
As well as managing the Shop website, the WebShop CMS is also used to track and manage collection merchandise, online orders and payment transactions.
The WebShop CMS integrates with Qi for user identification management. For example, an Art UK registered user can save a group of artworks in a folder. This folder can then be used in Art UK Shop as the basis for a bulk order.
WebShop CMS uses Microsoft SQL Server database.
|Symfony||Symfony is used as the application framework for the Permissions Portal.
The vast majority of content for the Permissions Portal is pulled from and updated back to Qi (using the Qi API) but some static content is stored in Symfony.
(but migrating to Amazon AWS)
(v) Image management layer
Table 4 Image Management
|Art UK online images||All online image for Art UK are stored in Amazon S3.
High resolution JPEG images of artworks are created by Art UK (from TIFF masters) and uploaded to Amazon S3 using Qi.
Qi includes a number of image workflow routines that create optimised versions for Art UK as well as embedding detailed object metadata into image files. Art UK images also include digital watermarks (Digimarc).
|Art UK Shop images||Website images (products, lifestyle etc.) are stored and managed using the WebShop CMS and stored in the Private Cloud managed by Internova.
Images for artwork-related products are automatically pulled from the Art UK Amazon S3 source.
High resolution images (used for printing or image licensing) are retrieved on demand from the Art UK Master Image server.
|Art UK master images||The original, master images for artworks are managed by Art UK and currently stored on an Art UK Master image server.
This is currently located on the Art UK internal network.
(vi) Collection content challenges
Art UK comprises 215,000 artworks from more than 3,250 different collections and venues. During the initial catalogue phases artwork metadata and images were originally provided to Art UK on spreadsheets (based on CDWA Lite) and discs collection by collection. Over time collections may have acquired (or disposed of) artworks or created new (higher resolution) images of their artworks. Keeping this content up to date on Art UK presents a number of challenges.
The launch of Art UK (in February 2016) was an opportunity to develop a self-service portal for collections. Rather than have to rely on emailing content changes to the Art UK editorial team why not allow collections to manage aspects of their own content for themselves? The Collections Portal was borne from that premise.
The Collections Portal currently allows collection users to add a new artwork, update an existing artwork (change metadata or upload a new image), upload a batch of new images or change details of a collection venue:
Fig 4 Art UK Collections Portal menu
The batch process is based on entering data into a spreadsheet template and uploading the related images to Dropbox. Qi workflows validate the data and show errors. It’s useful for small batch updates of new records.
The Collections Portal doesn’t yet allow users to a) edit a batch of existing records b) delete a single artwork or c) delete a batch of existing artworks. Using a spreadsheet template works but there is an overhead for collections to create data in that format.
We were conscious that the initial functionality of the Collections Portal only addressed core needs. This was probably adequate for smaller collections but not really useful or sustainable for the larger collections with with frequent amendments. Some form of data harvesting approach is required here.
When thinking about possible data harvesting solutions for large collections we are aware of potential barriers – do collections have technical resources to integrate with a 3rd party service such as Art UK; a wide range of existing technology architectures and software systems used by collections; limited resources (people and budget) at Art UK; what record formats to use and how to handle collection-based data mapping; how to address bulk image updates?
We determined a few key objectives for harvesting data from larger collections – automatically, update artworks but flag some key changes to Art UK, minimize manual checking and editing, minimise Art UK house style changes, adopt a well-used data exchange format (such as LIDO) and expose Art UK data as Linked Open Data.
Adoption of a data exchange format such as LIDO would be vital. Rather than establishing an Art UK specific schema it makes sense to leverage an existing, lightweight exchange format as any investment in the project will have benefits for the collections beyond Art UK.
One of the key issues for Art UK has been artist identification. The current master artists list on Art UK has approaching 40,000 different artists. Historically, individual collections have tended to manage their artist authority records separately. Where the same artist has works in many Art UK collections it’s vital that the Art UK database has a unique record for the artist.
The Art UK artist records in Qi have already been linked to DBpedia and Wikidata. Using established, prolific, 3rd party identities seems to be the best approach to ensure consistency across collections. The challenge ahead will be to persuade the larger Art UK collections to enhance their own artist authority records with Wikidata ID’s. The added benefit of using Wikidata ID’s is the ability to leverage existing links to third party resources (such as VIAF, Oxford Biography Index, ULAN, Library of Congress, etc.).
Art UK is now starting to work with a group of larger collections to undertake further research and develop a core scope and specification for a data harvesting project.
7. Key Success Factors
We are still on this journey and are learning all the time from our experiences and the experiences of others. As starters, the following factors seem key.
At heart, the Art UK initiative is a massive collaboration. In total there are 3,250 venues on the site; there are some 650 Partner Collections who pay an annual fee to contribute to platform sustainability and access commercial opportunities; there are 14 project partners including the BBC, Culture 24, and two universities; there are three extant Steering Panels with some 50 members including many representatives from museums and other public collections.
All of the participating collections that share the Art UK platform have a stake in it. Ensuring the collections – large institutions and small – are involved in developing the initiative through our Steering Panels (https://artuk.org/about/advisors) has been a key aim of Art UK’s governance, and representatives also sit on our Board and Advisory Board. Henceforward, the Art UK Steering Panel will hold some meeting outside London and we will be opening up these meetings to collections beyond those on the Panel to ensure we have the maximum input from the sector. We will also be doing an annual survey of collections’ needs. To bring us closer to our Scottish collections and audiences we will open a small office in Glasgow later this year.
We see linking back to participating collections as a key part of this collaboration and do this as much as we can. Where collections have their artworks on their own websites we link back to these. We will also provide participating collections with dashboards to see their traffic to the Art UK site and which of their artworks are the most popular.
Fig 5 The Discussions Page from Art Detective
The part of the digital infrastructure where collaboration works in a particularly exemplary fashion is Art Detective (https://www.artuk.org/artdetective/). This is a digital network that links collections in need of information about their artworks on Art UK with art historians, curators and members of the public who can provide expert knowledge on missing sitter names, topography or unattributed artworks.
(ii) Legal Framework
Legal agreements have been vital to the way that we have digitised public collections and regulated the use and re-use of images on Art UK and on our social media platforms. 2,800 legal agreements, signed over the past 14 years, underpin Art UK’s relationship with the 3,250 venues that are featured on Art UK and underwrite the trust put in our organisation. In addition to this, thousands of image reproduction agreements have been signed by artists and estates. Approaching 40,000 artists are represented on Art UK.
Until recently agreements were recorded on paper but now they are going digital as collections and artist rights holders are being invited to upgrade to new digital agreements via our Permissions Portal website. The Permissions Portal is a self-service online interface. It allows collections and artists represented on Art UK to update their agreements with us and choose to further open up the rights in their images by attaching Creative Commons licences, which are then made available to users of the Art UK website. Around 30% of collections are agreeing to Creative Commons licenses and 23% of artists/estates. It allows collections to amend licenses in an easy way for the whole collection or artwork by artwork and make their images available for commercial products and licensing opportunities on the Art UK Shop, in order to generate revenue for participating partner collections.
The legal framework has been put in place with a considerable pro bono contribution from London Lawyers Freshfields Bruckhaus Deringer.
(iii) Approach to audiences
Our partnership with the BBC has been key to helping us learn how to bring artworks from the long tail of records on our database (using tags) to the surface through the stories we tell and matching people’s broader interests with subjects represented in art. Working closely with the BBC has shown us that the BBC can play a powerful role in sharing the UK’s rich cultural assets with the public through digital and broadcast.
Fig 6 The most popular painting since 2011 – Due to the BBC White Queen drama series
Whilst the relationship with the BBC has changed since we retired Your Paintings, the relationship remains strong. The TV series Britain’s Lost Masterpieces, in which a number of artworks discovered on Art UK, have been re-attributed to the likes of Rubens and Jordaens, has Art UK as its partner. BBC Arts features branded stories from Art UK and sends traffic to our site (around 2% of our traffic comes from the BBC).
Average monthly uniques in last 3 months of 2017 approached 180,000 visitors. The organization and website is still very much below the radar and significant work remains to be done to raise its profile. Interestingly just over 50% of traffic comes from overseas, indicating that audience-facing digital infrastructures can play an important role in promoting a country’s ‘soft power’. Average time on the site is around 8 minutes.
Around 25% of our audience comes to Art UK for academic or professional reasons (April 16 data). 10% of the audience come to the site to arrange visits to collections. When we surveyed the Your Paintings audience in 2013, 40% said that seeing art online encouraged them to want to go and see the real thing and 14% claimed to have gone to see an artwork in the flesh as a result of seeing a work on our site.
Broadening our audience and reaching people excluded from culture is a key focus for us as our audience data has underperformed benchmarks for 16-24 audiences and BAME audiences. In an 18 month initiative funded by Arts Council England we have been looking to broaden our audience whilst measuring the impact of a variety of micro digital interventions. This work will be completed in March and will be shared with the UK museum sector and at the Museums and the Web conference.
We are particularly aware that re-use of our aggregated data could be very powerful and funding allowing will be looking at how we make our data more open and linkable.
Raising revenue for digital projects remains a constant challenge for most museums and organisations such as ours in the UK. What is most problematic is raising funds to cover the core running costs of maintaining and operating digital projects to the scale that is required. And, of course, by their very nature, such infrastructures need long-term funding. The success of Art UK needs to be measured on a multi-year view, not the next six months.
From our experience, in the context of the mixed funding model that prevails in the UK, some regular public funding needs to be part of the Art UK funding mix. In our view, the case for this is indisputable. The economic savings resulting from creating one piece of shared digital infrastructure that all public collections can share, means the public purse does not need to fund multiple initiatives amongst the thousands of public collections that would like to put their artworks online. Spreading our FY2017 total costs (including significant exceptional capital costs) across 3,250 venues amounts to c.£250 each.
Our ten-year oil painting project was c.75% funded by the private sector (trusts, corporates and individuals) and only c. 25% funded by the public sector. In recent years this balance has reversed and we are far more dependent on public funding than we have been historically.
However, longer term, we aim to return to our previous funding split and have been developing a funding model to achieve this. There are a number of elements to this.
A key one is subscription income from participating collections that become Partner Collections. Annual subscriptions of between £50 and £1000, depending on collection size, are now being made by 650 of the participating collections (with a good number funded by charitable trusts – for example the Wellcome Foundation paying for all the medical institutions with art). The incentive to be a Partner Collection is that only Partner Collections can generate commercial income through the Art UK Shop. This subscription income will grow as new collections become Partners and existing ones add artworks, thereby going up the fee scale. This ‘earned income’ is supplemented by some commercial income from digitisation services (private sector and public sector), and will be aided by a growing range of Art UK retail products which in due course will include limited edition prints.
In addition, there is a range of voluntary income streams. Historically the grant-giving trusts were a key income stream and will start to be again. Some of this funding is core funding but much has been and will be project-related with some contribution to core costs. Our Benefactor scheme encourages individuals and corporates to support us with donations between £1,000 and £10,000 per year. This has been successful without giving many benefits in return but in future Benefactors will be invited to a range of events at collections and UK art dealers (who themselves support us through a Dealer Circle). We will also be looking at membership schemes, corporate sponsorship of digitisation and renting out parts of our infrastructure to other organisations or groups of organisations. We are also going to explore the idea of creating an artworks in store rental scheme that uses the Art UK site to allow people to find works to rent. This will also generate income for Partner Collections.
Raising funds for artwork digitisation remains challenging (such funding covers not just photography but creation of metadata, curatorial and technician time etc). However, we have recently raised £3.8m from the Heritage Lottery Fund (National Lottery) and various match funders to digitise the nation’s sculpture of the last thousand years. This will contribute to core costs to a degree. It is a three-year project.
9. Focus in 2018
A key focus for 2018 is rolling out the Art UK Shop which, like the main site, constitutes shared infrastructure shared by Partner Collections on Art UK. This will allow them to generate commercial revenue from their image assets (fulfilled by our supplier Internova) and sell other merchandise (fulfilled by themselves). Art UK does not share in the transactional revenue stream.
Fig 7 Art UK Shop Home Page
A second focus in 2018 and 2019 will be growing the learning resources on the site and associated physical activities, which will include Masterpieces in Schools, in which we will take sculptures from collections into schools. In 2013 we took real paintings by the likes of Monet, Gainsborough and Turner into schools. We have also launched a Write on Art essay prize with the Paul Mellon Centre aimed at 14-17 year olds encouraging them to write about any artwork on Art UK. We will also be adding a number of films related to sculpture and paintings. Some BBC film archive is already on the site.
As part of our Sculpture project, we will be sharing our digitisation experience through training smaller collections as we go round the country photographing sculpture. This project will also offer significant volunteering opportunities.
Andrew Ellis and Adrian Cooper January 2018
Ellis, Andrew and Cooper, Adrian. "Reflections from Art UK on building and operating a digital infrastructure for public art collections across the UK." MW18: MW 2018. Published January 21, 2018. Consulted .