For much of their fifty-year lifespan, analytics were “artisanal”—hand-crafted, slow, and expensive to create. The goal was typically to create a report or dashboard using descriptive statistics, although there were occasionally some predictive regression models too. But there was another attribute of artisanal analytics that I have seldom mentioned until now: they were individually-oriented. Like an individual artist or craftsperson, the analytical artisan created analytics for him or herself, perhaps facilitated by a support person. At most a more senior decision-maker might see the results of the analysis—after all, this was the “decision support” period.
But now we are in a new era in which data, analytics, and AI are increasingly considered key organizational assets. They are typically created by teams for teams, and the many producing teams might serve consumer teams that are colleagues, senior executives, customers, suppliers, or an entire ecosystem. Today, for an analytics artisan to create an analytical program for him or herself is considered a waste of value and time. And the highly centralized approach, in which a corporate business intelligence group prepares data, dashboards, and analytics for thousands of users, can be even slower and more expensive.
This shift to “collaborative analytics” has implications for the tools we use, the processes we employ to create and distribute analytics, and the way we organize analytical talent. We need to make it easy to share analytical initiatives and insights throughout, and even outside, our organizations. This trend hasn’t been discussed much, so I thought I should talk with some leading analytics companies in order to explore it. Aginity, a software company whose offerings (particularly a set of SQL collaboration products) support collaborative analytics, suggested a couple of likely suspects (the company sponsors some of our work at the ROAI Institute). I already knew the Chief Analytics Officer of a large health insurance company who was focused on this issue—more on him and the company later—but I had not heard of Nick Marr or the company he cofounded, Clear Box Retail.
Collaborative Analytics at Clear Box Retail
Clear Box Retail, a company that mostly operates in the U.K. and Canada—but is moving into the U.S.—was founded on the idea of sharing data and analytics across company boundaries, and doing so rapidly. You are probably familiar with the idea that there are companies like Nielsen and IRI that gather scanner data from supermarkets, organize it into who sold what when, and eventually share it with consumer packaged goods (CPG) companies that sell through that channel. It works fine for comparing firms’ market shares long-term, but it takes a long time—sometimes weeks—for the data and analyses to make its way back to the manufacturers. And while it’s good to know your market share, the manufacturers often aren’t sure what to do about it.
Clear Box provides store-level scanner data—as well as inventory, promotion, shelf space allocation, and sometimes wholesaler data—and shares it with manufacturers. They find out what happened yesterday by 9AM the next day. Clear Box gives them not only descriptive reports but also prescriptive recommendations—“You only sold six boxes of your product in Store 477 yesterday because that’s all they had in inventory.” The path to action is clear—in that case, get to that store and tell them to reorder quickly.
Clear Box is the ultimate in collaborative analytics. It has to gather data in a variety of formats and make it available in a variety of different formats. It works with retailers, wholesalers, and manufacturers, and multiple groups within each type of organization. The company uses a variety of tools to make this possible, including Aginity Premium, Snowflake’s cloud warehouse, and Microsoft PowerBI. Some consulting is involved, but the tools are largely self-service. Most of its clients still rely on humans to look at the data, visit some stores, and recommend actions to resolve problems. However, Marr suggests that many of the problem resolution steps can be automated with the right collaborative analytics. For some online retailers like Amazon, they already are to a substantial degree.
For Clear Box customers, it’s a perfect storm: data is exploding, resources are declining, and everyone is trying to develop capabilities to keep up. A collaborative analytics partner is just what these companies need.
Collaborative Analytics at a Healthcare Payer
A large healthcare payer is also moving toward collaborative analytics. In its case one of the biggest problems that collaborative analytics can help to address is multiple versions of the truth. The company has many different groups that practice analytics, and they are coming to meetings with different perspectives on what’s happening with members, providers, employees, and so forth.
The company’s Chief Analytics Officer is investing in a platform to provide “data as a service” for this type of reporting and analysis. He noted, “We have many thousands of employees who are pretty good with data and work with it every day. But they spend 80% of their time pulling the data together, and they should be spending 80% analyzing it. We hope our new platform will solve that problem. But we are also trying to improve their data literacy—we’ll have different versions of it for the different user personas, and different tools for them as well. Some will have static dashboards, some dynamic dashboards, and some more sophisticated tools. We of course want to move the majority of them to self-service. Then at some point we hope to move them toward more automated data discovery.”
The other situation in which collaborative analytics are necessary, he argued, is when a business group has a high level of analytical skills, and collaboration with the central analytics group is desirable. “Take the actuaries, for example,” he pointed out. They have people who are heavily data and insights driven, and they are very hands-on with analyses. If my central analytics group is working with them it’s a level of collaboration where you’re co-developing, and the lines are blurred between what we do versus what they do. But our collaborative processes aren’t well-developed. We need to figure out a way to work with them smoothly as partners.”
These data and analytics problems, he argued, have real business implications for the company that are holding back its growth. “We need to hold on to more of our members when they reach 65 and go on Medicaid. We need to influence our members to manage their own health better, and also to influence physicians to provide value-based care. We’ve done a lot on identifying fraud, waste, and abuse, but we could go further there too. There are just so many opportunities, and it’s really important for us to get the right analytics to the right person and not let our data issues get in the way.”
Call these issues what you want, but I like the term “collaborative analytics.” It signifies that analytics are no longer artisanal, and should be addressed as an activity in which the entire enterprise is engaged. Both Clear Box Retail and the healthcare payer have data and analytics issues that involve collaboration across their company and with customers and suppliers. Giving each individual a spreadsheet or a visual analytics tool, and pointing them to a set of databases, no longer does the job—if it ever did.