The Digital Footprint
Catherine Devine, Microsoft, USA
AbstractThere used to be just the museum, and then there was the museum and the website. And then with each year came more digital channels: apps, social, virtual reality, and voice. Now, there is a proliferation of digital channels which are ever-increasing and becoming fragmented. Websites, social, apps, mobile Web, voice and chat interfaces, virtual reality, notifications, wearables. We have more channels and increasingly more data than we can manage. We are drowning in data but unable to draw meaningful insights from that data. This paper looks at the journey the American Museum of Natural History has taken to define and monitor the institutional digital and physical footprint. This allows us to extend beyond traditional measures of website traffic to understand the channel mix, shifts between channels over time, and understand initiatives that drive growth in overall reach. In addition, it is about understanding key behavioral characteristics of the audience. Rather than looking at audience behavior in a single channel, we are looking at behavior across the integrated physical and digital experience or journey. Also, to understand how visitors move across digital channels as well as between digital and physical channels (the museum). That is, how they move between the website and email and apps and the physical museum for example. Finally, it is focused on identifying insights that support decision-making rather than generating reporting. Leveraging this data to drive decision-making in the museum that delivers on the museum's goals and objectives, by confirming hypotheses about the audience.
Keywords: insights, analytics, data driven, digital
Section 1: Introduction
Prior to the advent of websites twenty years ago, the physical museum was the primary interface with the museum along with traditional media and advertising channels. The measurement of physical attendance at the museum was often the primary measure of the museum audience. However, the advent of websites brought a whole new significant channel into the mix. In the last decade, with tools for measuring website audience becoming broadly and freely available, in particular Google Analytics, the metric of users, sessions (or visitors and visits) along with page views has become the primary means of articulating the museum reach in digital channels.
The digital space has evolved rapidly, and in the last decade we’ve seen a proliferation of digital channels, in particular mobile with apps and Web, as well as social. In the last five years we have seen search emerge as a key channel not just for discovery of content on our websites, but also as a source of content itself about the museum. In 2017, we saw voice become mainstream with Siri, Alexa, Google Home, and other manifestations of voice on mobile and voice-specific devices. The rapid change in technology and the always emerging technologies indicate that digital will continue to grow to even more channels, resulting in even more fragmentation in understanding the digital audience. The days of the website have not yet disappeared, but websites are no longer the only valid focus in measuring the digital audience.
The objective of this paper is to outline a framework that we’ve introduced to more accurately measure the growth or contraction in reach and engagement of museum audiences. It is specifically focused on measuring the overall digital audience, not just the website, but also recognizes that the physical audience is a valid channel as well that is deeply integrated. We term this as the “total footprint,” with an emphasis for this paper on the digital footprint.
This paper outlines the following:
- the museum objectives of measuring audience reach and engagement;
- the limitations of current digital audience measures;
- a framework that is more reflective of the current experience, particularly as it has emerged in the last five years;
- the benefits and limitations of this proposed framework;
- a position on where I think we are heading as we transition to journey analytics.
Section 2: Background and history
For some time, measuring the digital reach of a website has been focused on the Google Analytics concepts of user (visitor), session (visit), and page view.
- Visits (otherwise known as sessions) reflects how many times a website was accessed in a given period;
- visitor (otherwise known as user) reflects the same measure but doesn’t double count individuals who visited more than once. If the same person visited five times they are counted as one visitor and five visits;
- page view is independent of how many people visited, but rather how many pages in total were viewed multiplied by the number of times they were viewed.
It was often concluded that growth in all these standard measures correlated to an improved website. However, sometimes these measures encouraged the wrong behavior. That you have maximized page views could mean you have many people really engaged in what you have to offer, or it could mean that people can’t find what they are looking for. If you redesign and see a decrease in page views it could actually be a good thing, as simplifying the site has made it easier for people to find what they are looking for.
The reason we use these measures is because these are the easiest to measure and track. However, the easiest to track does not always mean that it is measuring the right behavior. In addition, as we have seen websites diverge into desktop and mobile, we see significantly different uses of websites in mobile, which makes these measures even less insightful.
Forrester (2018) in it’s Digital Intelligence Playbook takes a very sophisticated view of insights and analytics processes and practices. Specifically, it makes the point that analytics need to become action. Simple measures do not allow us to take holistically informed action in our organizations.
Proliferation of digital channels
Since 2007, with the advent of the smartphone, and then the rapid growth in adoption of mobile technologies, we have seen mobile emerge as its own separate digital channel, and within mobile, in the form of mobile Web or apps. To a lesser extent, we have also seen wearables emerge.
Many other digital channels have also emerged, some specific to museums, and others more generally. Social has emerged as a significant platform of its own, as has search, and specific to museums has been in-gallery interactives. Search results and app store marketplaces such as Apple’s App Store and Google’s Google Play have become examples of places that the museum is reached even though those properties are not controlled by the museum.
It used to be that people would visit your website to understand what time the museum is open, or what events are on. Now, many people will default to channels such as Facebook to look up events, because that’s where you go to find out about any event, not just museum events. They will use Google Search as the front door to your website and obtain opening hours right on the Google results page, negating the need to visit your site. There are many more examples. If people want to know what to see or even if they should visit, they are visiting Yelp or Trip Advisor. People still come to websites, but it is no longer the only channel through which this information is available to them.
Channels continue to emerge, and this is expected to continue. This year, it is voice-enabled platforms and chatbots, such as Siri, Alexa, Google Home, and shortly Apple Home.
In many cases, the measurement of these channels lags the introduction of the channel. We saw this with apps where for many years the only visibility you had to usage was downloads. Only later did more sophisticated analytics develop about behavior in apps. Downloads became the default measure in the same way that users, sessions, and page views became the default measure. Sometimes this lack of insights early on in a channel’s emergence contributes to the mindset of defaulting to only measuring websites.
We expect the future to see many more emerging channels and the increasing fragmentation of digital channels, with potentially measurement systems that lag the mainstream. Kihn (2016) discusses further the fragmentation of digital channels and measurement tools.
Physical audience measures
Many museums have typically measured physical attendance. However, the Mmseum can be reached in many other ways, such as through its programs. Even physical measures of attendance can be enhanced to be more encompassing.
Issues with current measurement systems
The measurement and focus of website visits, visitors, and page views as a representation of the digital audience is a limited view. It limits the real audience and impact of the museum in digital channels. It influences the mindset that the Web is the main digital channel, and it doesn’t reflect the relative weight of the website against other digital channels. As new digital channels emerge, and we see mainstream adoption, such as social and search, it can create the appearance that the website is contracting in usage. The Web has become one piece of a much larger picture, rather than being the whole picture.
Despite the discussion as to whether Web measures are the right measures, the proliferation of channels has also resulted in different measures in different channels, and in some cases, limited to no measurement. In addition, some measures represent activity in a period and some are defined as lifetime totals. This makes this issue even harder to resolve if you want to look at the total digital audience in aggregate, or how it has evolved over a time period. Social is measured by likes and follows; apps by downloads; websites by visits, visitors and page views. As digital is easier to measure than physical channels, it does not mean that we shouldn’t consider channels that are more difficult to measure within digital, or in considering physical channels. This is the fallacy of measuring what is easy rather than measuring holistically and accurately.
These different measurement systems present the problem of “how do you look at the audience in aggregate,” as they are not apples and oranges. More importantly, how do you look at trends in the aggregate as well as at the individual channel level as some channels contract, while others grow and emerge. All of these channels are connected to create an overall connected museum experience, with both digital and physical channels, and between the many digital channels. This measurement of the connectedness of the experience will be discussed later in this paper.
Section 3: Why measure digital footprint?
Before we move on to a more encompassing way of measuring reach and engagement, it is worth pausing to discuss why we want to measure the total footprint in the first place. Reach is a core goal of many museums, because it tells us how many people have been exposed to the museum’s mission. At the American Museum of Natural History, our mission includes the communication of science; and core to that mission is reaching as many people as possible. You could debate that reach isn’t a sufficient objective, and that is true. It is equally important that you reach people effectively, and that is as important to us.
Reach can be considered as a funnel. The larger the funnel is, the more people you will reach, and the more people who will ultimately be engaged in the science.
Equally important to total reach is understanding the relativity of each channel to the total reach as individual channels contract and grow, and the relative influence and effectiveness of each channel on engagement. This is particularly important in an era where channels emerge and disappear overnight. It’s a statement that the world has changed and will continue to change.
An understanding of which channels are the most influential and effective also drives an understanding of how to optimize investments. This is important in a world where the channel with the most influence can change rapidly.
Section 4: Approach
This section details the proposed framework for calculating the following index:
- overall reach;
- overall engagement;
- measuring movement in that index over time; movement in the index reflects overall growth or contraction.
A detailed worksheet can be found at the end of this paper.
The framework is designed to identify the following:
- all audiences;
- their size in their native measurement systems;
- normalize those measurements;
- calculate an aggregate index.
The index measures the following:
- trends over time in aggregate;
- trends at an individual channel level;
- changes in relativity of each channel.
It is key to emphasize that the results are an index and not an absolute measure of the reach.
The key components of the framework are the following:
1. Audience channels
Firstly, we look at all the ways that the museum is discovered or engaged with. That is, all the channels that the museum has a presence in, both digital and physical.
For example, digital channels could include Web, mobile, apps, social, interactives, search, apps marketplaces, e-mail, digital ads, voice. Traditional channels could include museum attendance, program attendance, and traditional media channels such as print and ads. Moonka (2015) discusses the concept of understanding all the touchpoints in your marketing mix.
The core categories of audiences in 2017 were as follows:
Physical Museum: attendance from visitors for general admission and programs
Members and Donors: number of members and donors
Website : page views across website
Search: impressions on first page of search results, and of info boxes on branded search terms
Social: impressions on posts made by museum
E-mail: e-mails distributed
Ads: digital and traditional impressions
Media: digital and traditional impressions
Print: distribution of print materials
Voice: voice searches for museum
Third party websites: activity on websites such as Yelp or Trip Advisor for museum brand
To measure reach for each channel, we included the raw reach measure for that channel.
For example, on e-mail it was e-mails sent; for ads it was impressions; for museum visitors it was attendance; for Web it was page views. These are all “apples and oranges” measures, and we understand that. The first step was to make a statement about measurement in its raw form before normalizing it.
3. Normalizing and weighting impressions
The challenge with raw impressions is that they are not apples and oranges in each channel. A museum visitor is a museum visitor, and you can have reasonable confidence on that. The same is true with website visits. However, something like the number of people who could have seen a digital ad overstates the audience, or rather doesn’t overstate the audience, so much as it cannot be compared using raw numbers with a museum visit.
So, we defined weightings to each channel. The only purpose of the weighting was to try and come closer to being able to equate the apples and apples value of impressions in each channel. It is definitely a subjective measure. Intentionally, weightings do not refer in this case to what we think is the relative importance of each channel. It measures the under/overstatement of the measure relative to other measures.
For example, a website visit has a higher weighting than an ad impression. Why? Because a website visit required someone to do something, intentionally access museum information, whereas an ad impression may only have been exposed to someone and they are more peripherally aware. If we didn’t apply weightings, ad impressions would significantly overstate the audience relative to Web visits, and incorrectly infer that it was a more influential channel.
Schoenfield (2014) discusses in more detail the challenges with aggregating and standardizing measures across channels to obtain that big picture view.
4. Engagement and weighted engagement
To measure engagement, we took an engagement metric in each channel and performed a very similar exercise to what we performed above for reach.
We listed the engagement measure in its raw form. As an example, for digital ads we used weighted impressions as the reach metric, but weighted clickthroughs as the engagement measure.
We identified a weighting for engagement in the same way we did for reach, so that we could better compare one channel to another.
The key output is a set of indices for weighted reach and engagement that are then measured over time to reflect the following ratios:
Reach: Overall growth or contraction in audience size
Weighted Audience (this period)
Weighted Audience (previous period)
- weighted audience of 228m this period versus
- weighted audience of 227m last period
This reflects 0.44% growth in aggregate reach.
Negative values represent contractions.
Engagement: Overall interactions level growth or contraction
(Weighted Interactions (this period) / Weighted Audience (this period)
(Weighted Interactions (previous period) / Weighted Audience (previous period)
- engagement this period is 42m versus
- engagement last period is 40m.
Engagement is (42m/228m)/(40m/227m) reflects 4.5% increase in engagement.
Relative growth of channel: Overall channel level growth or contraction
Channel Audience (this period)
Channel Audience (prev period)
For example, web channel is 50m page views versus 48m last period.
Overall channel growth is 4.2%.
It is important that this ratio is comparing the channel’s growth to itself not the channel’s growth relative to the overall reach. That is the next measure.
Relative importance of channel: Overall channel level growth or contraction relative to aggregate
(Weighted Audience for this channel (this period) / Weighted Overall Audience (this period))
(Weighted Audience for this channel (previous period) / Weighted Overall Audience (previous period)
For example, Facebook channel is weighted at 8m against a total weighted audience of 228m. In previous period, it was 7.5m against a total weighted audience of 227m.
Overall relative importance of Facebook has grown 6%.
I want to emphasize that with all of these measures, measures such as weighted audience of 227m do not reflect the actual audience. It is not 227m and should not be interpreted as anything other than an index. This is one of the greatest concerns with using numbers: that they can be interpreted as more accurate than they actually are because they are numeric.
We should treat the overall calculation as an index. The key is in the relative movement over time, positive or negative, and the impact campaigns have in influencing those directions. We do not want to say that our audience reach is 100m for example, we want to say that is has grown 5% year on year. Within that we are seeing 20% growth in social, 5% contraction in Web, 25% contraction in media, and 6% increase in museum visits.
It is important to emphasize that the model is an MVP (minimum viable product). It has many limitations which we plan to continue to develop.
The first key limitation is that it does not measure duplication of individuals across channels. If an individual engages with us across multiple channels, we will see them as multiple people and are effectively double counting them. This is why it is so important to be careful about how the numbers are interpreted. Despite the limitation of double counting of individuals, which has existed in traditional channels for many years, it does represent many important pieces of information. In particular, it tells us the relative growth, contraction, or influence of individual channels.
The second key limitation is the inherent subjectivity of the weighting. The weighting system is imperfect. It allows for some normalization of data to enable channels to be compared relatively, and is better than present day systems, however those weightings are subjective and will differ from institution to institution. Key is to retain consistency to again measure trends over time.
The third key limitation is that until you have 12 months of data, it doesn’t account for seasonality of the data.
The fourth limitation is that of attribution. That is, recognition that activity in one channel influences activity in one channel. For example, how e-mail influences Web usage. How does a museum visit influence use of an app or other interactive?
Finally, as has been stated several times, it is absolutely key not to interpret the index as an absolute measure of the audience, but rather an index to measure trends over time.
6. Don’t let perfect be the enemy of good
It’s important to emphasize again that this is an MVP. I will be the first to admit that this is not a perfect model. However, it is a more representative model of audience than only focusing on Web visits or app downloads. Is it better to consider and attempt to measure your holistic audience than it is to live in a world where measuring is simpler but does not take into account the entire picture.
This model is about measuring trends over time in aggregate, and in individual channels. It is also most importantly about understanding what drives growth, and measuring the efficacy of initiatives in driving growth, as well as attempting to isolate a campaign’s influence on growth from behavioral trends in the audience towards that channel. It’s important to not only report on aggregate audience, but also to influence growth in that audience through campaigns. The understanding of the impact of campaigns helps you understand which levers you have to influence reach and engagement, and which are the most effective.
It also allows you to assess how many resources to allocate to whatever is the latest channel, and to be more informed about when to ramp up capabilities and investments there. If, for example, Snapchat is an emerging channel, but is only representing < 1% of your aggregate audience, is this the right time to allocate resources to it? It may be; it may not be until it reaches a higher percentage, but it allows you to have this informed conversation in your organization. Similarly, if you see a trend away from the Web over time, this helps you have an informed conversation as well. Do you want to focus on increasing its influence, or do you want to divest resources into other channels?
7. The Future: Experience or journey analytics and prioritization of resources
One of the key limitations that we discussed was understanding that people exist in multiple channels, and so there is effectively an element of double counting of individuals in this model. That is, a museum experience is effectively a connected experience where visitors transition many channels, digital and physical, over time, in their interactions with the museum. The right measures are the measurement of the journey or experience, or journey analytics. van den Brink-Quintanilha (2017) discusses in detail the journey analytics process and tools landscape and makes projections about the future. However, the tools and mindset are not yet sophisticated enough and available for us to do this. Digital and physical channels are the building blocks in the delivery of an overall experience, as people move between both digital and physical channels, and between many digital channels in their overall view of a museum experience. They don’t distinguish channels, they just see it as the museum.
Finally, this model can be layered with costs incurred per channel that provide for informed decision-making on resource allocation. This is particularly key as traditional channels give way to digital channels, and within digital channels, those channels emerge and die rapidly.
Today’s focus on website visits is a narrow view of what is really happening in the digital space. There is both a proliferation of digital channels, and the integration of digital channels and physical channels across a connected experience that has emerged rapidly in the last five to ten years. In the absence of tools to measure the experience, we have attempted to develop a framework that measures the digital footprint and growth/contraction trends over time to understand which are the most influential channels at any point, which channels are emerging and may warrant investment, and which are contracting and may require remediation or divestment of resources.
Forrester. (2018). Digital Intelligence Playbook for 2018. Consulted March 7, 2018. Available https://www.forrester.com/playbook/The+Digital+Intelligence+Playbook+For+2018/-/E-PLA680?objectid=PLA680
Kihn, M. (2016). “Customer Journey Analytics: A Manifesto.” Gartner. Published March 4, 2016. Consulted March 7, 2018. Available https://blogs.gartner.com/martin-kihn/customer-journey-analytics-a-manifesto/
Moonka, N. & B. Kee. (2015). “The Path to Better Measurement: Analytics and Attribution.” Think With Google. Consulted March 7, 2018. Available https://www.thinkwithgoogle.com/marketing-resources/data-measurement/path-to-better-measurement-analytics-attribution/
Schoenfeld, A. (2014). “Why Cross-Channel Measurement is a Problem for Social Marketers.” Simply Measured. Consulted March 7, 2018. Available https://simplymeasured.com/blog/why-cross-channel-measurement-is-a-problem-for-social-marketers/#sm.0001ie0tijdbpessyda2av5q472uz
van den Brink-Quintanilha, J. & A. Causey. (2017). “The Forrester Wave: Customer Journey Analytics Visioning Platforms, Q3 2017.” Forrester. Published September 14, 2017. Available ://www.forrester.com/report/The+Forrester+Wave+Customer+Journey+Analytics+Visioning+Platforms+Q3+2017/-/E-RES137895
Devine, Catherine. "The Digital Footprint." MW18: MW 2018. Published January 15, 2018. Consulted .