How Much Data Is Too Much Data?

Sure, new tech allows you to capture minute-by-minute data on how your sales team is doing. But will all of those numbers help your business?

Big data is hot, and it’s not hard to see why. In some contexts tracking minute-by-minute what employees are doing results in significant efficiencies. Take big box retail, for example. A new wearable device called Theatro that combines data tracking with wireless communication for employees, is making a significant impact at the Container Store where it’s being tested, allowing staff to communicate and find information more easily, but also helping management deploy employees according to where they can have the most impact.

“Once you understand the data, you can begin to do predictive behavioral analytics regarding finding the right job for people,” SVP of Marketing at Theatro Patrick Fitzgerald told Yesware. Early studies with the Container Store found that the system helped them reclaim 3-5% of their labor hours, he said.

That’s a pretty open-and-shut case for big brick and mortar retailers, but would this sort of constant data gathering and analysis benefit sales pros working in other sorts of environments? We’re at the beginning of a revolution and the promise of this field of technology is clearly vast, but many experts also suggest there are a few good reasons to be cautious as you ponder the right way to use big data at your business.

Ready for data that’s simple, actionable, and shows you what’s working as an individual contributor and what’s not?

What Gets Measured Gets Gamed

Everyone knows the old management saw about ‘what gets measured gets managed’, but the era of big data may be highlighting a less often acknowledged corollary to that principle: what gets measured often also gets gamed. It a problem Felix Salmon summed up neatly in his fascinating recent Wired article about “quants” and big data. “The reason the quants win is that they’re almost always right—at least at first,” he concedes. “They find numerical patterns or invent ingenious algorithms that increase profits or solve problems in ways that no amount of subjective experience can match.”

But that’s only the first stage of the process. Next, he asserts, often comes a fall from “data-driven paradise.”  Salmon writes: “The more a field is run by a system, the more that system creates incentives for everyone (employees, customers, competitors) to change their behavior in perverse ways—providing more of whatever the system is designed to measure and produce, whether that actually creates any value or not.”

Salmon writes about NSA spying, finance and even policing, but that same process is a danger in everyday business management as well, according to David Hoffman, professor and area chair of organizational behavior at the University of North Carolina’s Kenan-Flagler Business School. “There is some truth in both of the following statements: ‘what gets measured, gets done’ and the ‘law of unintended consequences,’” he told Yesware. “If organizations over-emphasize one particular metric, then the organization will get performance on that one metric. But, it might come at the cost of other important criteria of performance and success.”

Which isn’t to say data — even “big data” — doesn’t have its uses. Just that one should be incredibly careful about using data to incentivize (or evaluate) individuals. “Organizations need to be very thoughtful in how they collect, evaluate, and act on these data. For these types of approaches to be successful, there needs to be a foundation of trust between management and employees,” Hoffman believes. “Using aggregated, big data to identify patterns of interaction or team dynamics that lead to better performance seems to be very reasonable. If the organization is using these data to learn and identify best practices and then these best practices are subsequently taught to others – this seems like a great use of these data.”

“But, if there is any suspicion that these data will be used to single out and punish certain employees, then you start to have a crack in the trust foundation which, I believe, could lead to a number of unintended consequences,” he warns.

Big Data and Bias

The danger that data may turn your workplace into an adversarial game, weakening your team’s esprit de corps and shifting the focus from outcomes that truly matter, isn’t the only danger experts fret about when it comes to deluge of data washing over organizations. Using data for hiring raises other troubling questions, as INSEAD professor Herminia Ibarra pointed out recently in an article for the Harvard Business Review Blog Network.

“The beauty (and danger) of big data is that it’s not limited to the tests a person takes voluntarily as part of the hiring process—it can also scour our digital traces to find leading indicators correlated with on-the job performance. The vast number of data points that miners marshall afford them surgical precision in discerning which attributes correlate best with success in different jobs,” she writes. Combing the data for correlations between personal attributes and performance turns up all sort of unexpected findings, apparently, including one startup’s startling finding that the best predictor of coding prowess was a love of a particular Japanese website.

Cover that conclusion in data and it looks like science. State it without the numbers and you have an old-fashioned stereotype. “What is the difference between the pattern recognition afforded by big data, and profiling on the basis of gender, race or class?” Ibara asks.

She may be worried that big data will reinforce bias and shut out candidates that that don’t match companies’ pre-existing patterns of success, but not everyone thinks thing will play out that way. Don Peck, who wrote a lengthy article on people analytics for The Atlantic, reports coming away from the experience cautiously optimistic. He began reporting “worried that people analytics, if it worked at all, would only widen the divergent arcs of our professional lives, further gilding the path of the meritocratic elite from cradle to grave, and shutting out some workers more definitively. But I now believe the opposite is likely to happen,” he writes.

Companies already profile and suffer from bias. Now at least they’ll be looking at more variables than where you went to school and the jobs that educational background helped you gain access to, he hopes — but it’s only a hope. The truth is the science is too new to tell. Perhaps, then, Ibarra’s conclusion is the soundest: “Before we go whole hog on our embrace of this evidence-based revolution, shouldn’t we follow its tenets by actually conducting some studies about the diversity dynamics of this brave new world of talent management?”