statistical noise

The Benefits of Statistical Noise

26 August, 2020

In this article Ruth Schmidt, associate professor at the Institute of Design, Illinois Institute of Technology, talks about the importance of diversity of thought and ideas when designing systems – from business plans to public policy. Her work sits at the intersection of behavioural economics and humanity-centred design, combining strategic design methodologies and behavioural insights to inform effective, ethical solutions for complex system challenges.

She says the heightened tendency to tune out some data as unimportant is a well-known side effect of expertise, which encourages us to become highly attuned to some signals and patterns at the expense of others.

Noise, defined as extraneous data that intrudes on fair and consistent decision-making, is nearly uniformly considered a negative influence on judgment that can lead experts to reach variable findings in contexts as wide-ranging as medicine, public policy, court decisions, and insurance claims.

Across many contexts, our enthusiasm for clearing out noise sometimes leads us to throw out useful—even vital—information, especially when top-down approaches dominate bottom-up contributions to define what matters. In assuming that our values and lived experiences are universally shared, we risk insufficiently interrogating how they deviate from those for whom we are designing. Our assumptions of a shared premise simply beg to be made noisier when we are creating solutions for others whose lives, interests, and needs may fundamentally differ from our own.

Notions of what’s relevant can differ wildly depending on who’s perceiving the data, and over-enthusiastic noise removal can inadvertently over-simplify results by throwing out complexity. In other words, when situations lack a single right answer, trying to reduce unwanted variability by eliminating noise can inadvertently eliminate useful information as well.

But judging “what we see” in a stable environment with a history of reliable data differs from judging “what will be” in a complex systems context with multiple social factors.

In these cases, rife with independent variables and future unknowns, we can benefit from intentionally inserting noise into decision-making in the form of speculative scenarios that force us to concretely grapple with the potential downstream implications of our choices.

Inspired by insights from the field of military intelligence, Gregory Treverton described two kinds of problem-solving challenges: puzzles, which can be resolved by doggedly accumulating and digesting the right information to reveal latent truths, and mysteries, which demand speculating about ambiguous or future contingencies to help us make sense of the information we’ve already got.

Solving puzzles is largely an interpretive exercise, where finding the solution consists of organizing the right information in the right way. This is the job of detectives and diagnosticians: in these cases, there is a bull’s-eye to aim for. Eliminating noise can help us arrive at these solutions more effectively and efficiently in part thanks to a shared, underlying agreement about what data is relevant to make those decisions and the collective benefit of less crime, or better health care, when the system works well.

Solving mysteries, on the other hand, relies on how we define, or frame, the nature of the problem itself. Unlike puzzles, mysteries are often situated in an uncertain future, where there may be multiple valid conceptions of what we’re even solving for. This is more the domain of hiring employees and crafting business strategy, where making smart decisions may require us to question our assumptions about what data registers as important. In situations like these, removing noise in the interest of fairly and consistently applying relevant criteria may contribute to a false sense of objective precision that is not as desirable as we may think.

Yet many things in life—academic publishing, health care, and housing policy among them—require addressing individual challenges within the context of complex systems, simultaneously solving for puzzles and mysteries. As behavioral designers, we must compel ourselves to deeply and empathetically understand both the needs of the people we are designing for and the systems in which they operate, and critically question what our legitimate desire for fairness and consistency leaves out. If we don’t, our well-meaning efforts to reduce noise may inadvertently strip away essential signals, causing us to miss patterns, gaps, and perspectives in data that deserve our attention.

Behavioral Scientist, Ruth Schmidt, 24 August 2020. Read the full article here.

Cranlana Centre for Ethical Leadership’s programs include the 2 day Executive Ethics, 6 day Executive Colloquium and year-long Vincent Fairfax Fellowship. We also deliver online and tailored corporate programs. Find the right program for you.

Photo by Valentin Salja on Unsplash