Charting the Chartists:
A 2016 Survey of Data Visualization Professionals

Introduction

Over several decades, numerous studies have been conducted on data visualization practices, including a nice collection of fresh research being conducted in academia and the commercial world today. By data visualization, I mean any visual depiction of data, such as charts and graphs, maps, interactive data experiences and even more esoteric data displays that verge on the territory of art.

Most of this research has focused on the study of visual perception (how humans process and understand what we see) and best practices in data visualization– what does and does not work in creating visuals that connect, inform and inspire viewers to take action.

But there has been no research into the practitioners doing the work themselves:

  • Who does data visualization?
  • In what kinds of organizations and what departments within them, do these professionals work?
  • What kinds of data visualization are they doing, and for what purpose?
  • Is their work having an impact, and if so what kinds?
  • Why are some of these people having more impact than others – i.e., what makes some more successful at their work than others?

These questions are being (or soon will be) asked at organizations everywhere as this field grows and matures.

This survey is an attempt to begin to shed light on the state and nature of data visualization work. During the summer and early fall of 2016, we collected nearly 100 surveys (99 to be exact) from data visualization professionals online. The respondents were self-selecting and marketed to exclusively via social media.

Key Findings

  1. Practitioners are using an incredibly large array of tools to do their work, while some market leaders are beginning to emerge.
  2. “Increasing understanding” is the primary goal of data visualization but after that, the opinions vary.
  3. Good data and trained staff are the keys to success, while “figuring out the story” in the data remains a critical challenge for many.
  4. Those who are measuring their outcomes expect spending on visualization to increase, with nearly one in five expecting significant growth.

Leaders vs. Laggards

One of the most interesting ways to evaluate survey data is to compare “leaders” in some activity (data visualization, in this case) with “laggards.” What do those who are setting the trends and having greater impact in their organizations doing differently than those with less impact? In this survey, two questions helped us separate the leaders from the laggards.

Respondents were asked: “How good a job is your organization doing visualizing data overall?” The answers available were on a five-point Likert Scale from “Very Poorly” to “Very Well”. This question, while certainly not scientific proof of “leadership” or “lagging behind”, can be used to gauge an organization’s confidence in its performance (or lack thereof). This is the source for all of “the Confident” vs. “the Unsure” comparisons below. Each of these two groups consisted of 41 survey respondents.

Respondents also answered “If you are measuring outcomes, are you seeing a ROI (return on investment) (financial or otherwise) on your visualization projects?” This question also offered a Likert scale from “Very positive” to “Very negative”. “Positive ROI” (PROI) respondents are those who answered “Very positive” or “Somewhat positive” and “Neutral to Negative ROI” (N2NROI) are the rest. There were 28 PROI respondents (53% of those measuring their outcomes) and 25 N2NROI (47%) respondents. As one might expect, 75% of the PROI group are also in the Confident group. The N2NROI group is a bit more evenly distributed between the Confident (44%) and Unsure (56%).

What is the difference between these two groupings of “leaders” and “laggards”? It’s hard to make broad conclusions. However, when possible, I point out their differences and offer conclusions.

Primary Findings

1) Tools! Tools! And more tools!

The most surprising finding from this survey centers around tools being used. When asked “What are the top 3 software tools your organization is using for data visualization?” the answers were incredibly diverse.

Our question surfaced more than 62 different tools in use. And that is even after aggregating responses such as “Our own Platform” and “Other imagery and charts” into one “Other” grouping as well as grouping any mention of Adobe software into one set.

Equally as interesting as the diversity of tools used is the type of tools so many respondents rely upon. Excel – a spreadsheet for running calculations and creating charts, which was released more than 30 years ago – got the most mentions (43), followed by a second set of popular tools: Tableau (26), Adobe Suite (25), and D3 (21). The next closest was R, with 12 mentions. And most of the rest had one or a maybe a few mentions.

Is this an established industry with old-guard monopolies that dominate market share and mindshare? Is this an immature industry with plenty of room for new tools to proliferate and dominate? Or is this a maturing industry where standards are getting set and companies are building toward the next monopoly?

I think the answer is a resounding “yes” (to all three, kind of). It is difficult to accurately predict what the future holds for data visualization tools. Old dominant tools die hard. Meanwhile, new tools are popping up left and right. But among the newer tools, Tableau and D3 are becoming clear standards. Is there room for someone else to come along and eat their lunch? No doubt, given the vast array of tools in play.

What about the Confident and Unsure? Are they using tools differently? The Confident are using Tableau and Excel more than the Unsure by large margins, while using D3 and Adobe tools less than the Unsure by equally large margins. What does this mean, exactly? It’s very hard to say. But it is interesting and worthy of further investigation.

The PROI and N2NROI respondents had different patterns of tool use compared to the Confident and Unsure, indicating a clear distinction between measurable success and confidence. For instance, Tableau is being used about equally across both groups, and all three of the other top tools are being used much more extensively among the PROI group. Are those who are seeing success, then, more likely to work with more “custom” tools like D3 and Adobe? Or are they more likely to use a wider array of top tools? It is impossible to draw direct conclusions, but again this is an area that warrants more research.

2) No. 1 Goal of This Work: Enhance Viewer Understanding of Data

People visualize data for all kinds of reasons. No surprise there. But what was surprising (and comforting, really) was that 90% of respondents identified “increase understanding” when asked “When your organization is visualizing data, what are your key goals for that work?” A far distant second was “influence the influencers”, cited by 40%. When I teach data storytelling and visualization, I argue that increasing understanding should be the primary goal, so it is gratifying to see that nearly all of my peers agree!

Once again, the responses were pretty diverse, with multiple respondents (in fact, more than 10% of respondents in each case) identifying the same key goals from the provided answer choices. (The one lone exception was “receive votes”. This survey was running during the thick of a presidential election, so perhaps all the political operatives were busy with their candidates and not answering surveys!)

The biggest difference between the Confident and Unsure was the fact that nearly twice as many of the Confident (32%) mentioned “improve brand perception” as a goal, which can be difficult to measure. Meanwhile, “gain media coverage” was a much more important goal for PROI (36%) compared to N2NROI (12%), which is easily (and therefore frequently) measured. So it makes sense that those seeking media coverage are more likely to feel they’re getting a PROI than those seeking a less tangible outcome.

3) Key Success Factors: Good Data and Trained Staff Top the List

Multiple questions in the survey are helpful to determine the challenges and secrets to success in data visualization.

One of the most exciting findings in this survey was around ROI. Among those measuring the outcomes of their work (which is only about half of all respondents), 53% said that they are seeing a “very positive” or “somewhat positive” ROI. And just 4% of respondents saw any negative ROI. So 96% of respondents report a positive or at least neutral ROI. Everyone who reported a “very positive” ROI belonged to the Confident group.

What leads to success? “Good data” (71%), “trained staff” (70%), and “the right tools” (61%) dominated the responses. While “culture” wasn’t a multiple choice option, it did pop up amongst the respondents’ manual inputs associated with an “other” choice. For instance, one person said, “Having a staff that is enthusiastically ‘pro-data’ and equally ‘pro-data visualization’ from senior to line staff.”

Interestingly, a similar question was posed as an open response question and the results differed a bit from the multiple choice. In open response, tools dropped near the bottom of the list and respondents focused on training and talent, culture, ROI, and quality data and storytelling more heavily.

The primary difference between the Confident and Unsure addressing the multiple choice question was that the Confident recognized the need for good data at a very high rate (83% compared to 59%). Once again, the PROI and N2NROI varied significantly from the Confident and Unsure. Fully 86% of the PROI group identified trained staff as a key success factor, compared to just 60% of N2N ROI. And, understandably, the N2NROI group were much more likely to identify “sufficient funding” (24%) than the PROI group (11%).

When asked “What are the hardest challenges faced when visualizing data in your organization?”, once again we had multiple respondents (more than 10% each) selecting every response available. “Figuring out ‘the story’” (48%), “finding the time” (41%), and “cleaning up the data” (40%) led the way. The Confident (59%) struggled more with “finding the time” than the Unsure (24%). Meanwhile the Unsure have a harder time finding talent (39%) and budget (29%) than the Confident (22% and 15%, respectively).

The biggest variances between the PROI and N2NROI folks were once again different from the Confident and Unsure. The PROI struggle “figuring out the story to communicate” (61%, compared to 40% of N2NROI) while the N2NROI struggle with “analyzing the data” (32% compared to 21% of PROI). Are the N2NROI less likely to “struggle to find the story” because they tend to easily figure it out or because they aren’t even thinking about story at all? Do they really struggle with data analysis or is it just that the respondents consider that the hard part, because they are the ones doing that work? (This could easily be as much about their pride in the difficulty of what they do as it is about “struggling”.)

It is worth mentioning that, just as culture was a factor for respondents describing critical success factors, it also appeared as a barrier to overcome for many. “A cultural shift towards data visualization is needed,” one respondent shared.

4) Spending: Vast Majority of Budgets are Under $250,000 Growth

Based on their own estimation, answering “How much do you think your organization is spending on data visualization in 2016?”, most of our respondents (51%) are spending less than $50,000 per year on it and 87% are spending less than $250,000. The group of survey takers skewed toward smaller organizations (73% come from companies with fewer than 1,000 people), so this is not surprising. There was no significant difference between the Confident and Unsure, except to note that all of those who reported spending more than $1,000,000 per year belong to the Confident group.

More interesting than the gross spending is how respondents see spending changing over time. Almost no one (4%) said they expect spending on data visualization to shrink in the coming year. 43% expect spending to grow somewhat or significantly. The Confident are more likely (7% compared to 2% of the Unsure) to say it will grow significantly, but that is a small number. The PROI are much more confident, with 61% expecting growth (compared to 28% of N2NROI) and 18% expecting significant growth. Interestingly, even the N2NROI expect significant growth at high levels (12%), compared to the Confident and Unsure.

Conclusions

So what does all this mean for the data visualization professional? The data points to several conclusions in how they can improve their work and their organization impact.

Choose your tools wisely
The data visualization toolset is giant and growing, yet also consolidating around some leaders. Your organization needs the right tools to do the work and you need to select tools that will make it easier to find talent to use them now and in the future. You need tools that are easy to use and sufficiently powerful, yet also tools that allow you to create visuals that speak to audiences.

Know Why You’re Doing What You’re Doing
This may seem obvious, but you need to really think about why you’re visualizing data in the first place. You don’t need to do it because everyone else is, you need to do it to “increase understanding”, as almost all respondents agree, but also to achieve specific goals. Know your goals, measure your results and change what you’re doing to improve those outcomes.

Overcome The Challenges
Good data, trained staff, culture, and figuring out the story are the things people are finding challenging in this universe. So build an organization that is data-centric and believes in visualization as a powerful tool for communicating data. And provide the training and support to a broad swath of employees who will get on the visualization bandwagon, even if they aren’t directly involved in the work.

Follow the Leaders
Based on a few of the questions and the variance, and how similarly the PROI and N2NROI together stack up against the rest of the data set, one might conclude that regardless of the actual ROI, just measuring results alone is likely to make one a “leader”. Both the PROI and N2NROI are expecting growth in spending in data visualization next year at higher rates than any other group. Both also were more closely aligned overall in what they saw as the primary success factors and challenges in contrast to the differences between the Confident and Unsure.

Next

As with most surveys – especially surveys that are the first of their kind, this one brings up more questions than it answers. I would love to follow up with a lot of questions, such as, but not remotely limited to:
– Why are you using the tools you’re using? And if you identified that better tools are a key success factor, what does that mean, exactly?
– Based on your identified goals, challenges and success factors, what can you do in 2017 to achieve/overcome? For instance, training is important – what do you and your colleagues need training on, specifically?
– How are you measuring your success?
– How is your culture shifting?

The data visualization practice is a young and growing field. And it is a subset of activities within many different roles and departments in any organization. Defining it is difficult but can shed light on what works and what doesn’t for all of us working in this area. As it matures, we should see things like the number of tools used shrink and consolidate and budgets and spending increase, while also getting even more clear direction on what the primary challenges and success factors are.

Next year, we will do this survey again and I hope to have the support and involvement of even more people in the community. If you can think of questions you would add to next year’s survey, or improvements and adjustments to those asked this year, please share them with me! And don’t hesitate to share any other thoughts as well.

Note

This online survey (hosted on SurveyMonkey.com) was developed in Spring 2016 and shared online until October 2016. It was marketed exclusively via social media. Due to its small number of respondents and this non-random sample, it can not be considered scientifically accurate, but does provide interesting insights. In the summary report above, all percentages have been rounded for simplicity’s sake.

Additional Questions & Data

All of the additional questions and answers are below, and the full raw results are available for download. Just email bill@beehivemedia.com to request the responses.