Nevin's Data "Points"

  “Sure, it will probably be more work, and generate fewer clicks,” commented one industry source, “but it’s the right thing to do.” Yes, after years of relying on uninformed “guesses” from individuals ignorant of their financial needs and situation, the retirement industry, major media outlets, and a large number of academics have made a commitment to focus on actual data, rather than hypothetical extrapolations from incomplete datasets. Another explained, “we always thought that exaggerating the depth of the retirement crisis would encourage people to save more — but that turns out not to be the case.” Those projections affixed labels like “ magic ” to those extrapolated numbers based on surveys of uninformed workers, which not only ignored real differences in incomes, location and age, but were typically also averaged to further obscure accurate results. Likely fueled by previous reports of needed retirement savings,  surveys of individuals routinely exaggerated  the r...

I recently ran across a survey that claimed 7 in 10 DC plan sponsors were “taking steps” to solve the retirement income challenge… but that looks to have been “aspirational.” While the survey’s [i] intro cautioned that there was more to be done, that struck me as a remarkably high (and reassuring) finding, though it didn’t mesh with my sense of the world at present. Sure enough, turns out, there is apparently a retirement income “journey”—one that apparently has several stages—all of which were (apparently) classified as “steps.” Those included: 34% – INITIAL (my emphasis, their wording) stages of learning about retirement income approaches 14% – in the process of better understanding participants’ retirement income needs 8% – in the process of evaluating specific retirement income solutions/products 7% – implementing/implemented a retirement income solution/product Indeed, the survey goes on to comment that another 8% have evaluated these type solutions, and decided not ...

When you see a headline that confirms your sense of the world, you’re naturally predisposed to embrace, remember (and these days “share”) it as a validation of what you already perceive reality to be. Indeed, as human beings, we’re drawn to perspectives, surveys, and studies that validate our sense of the world. This “confirmation bias,” as it’s called, is the tendency to search for, interpret, favor, and recall information in a way that confirms our preexisting beliefs or hypotheses. It also tends to make us discount or dismiss findings that run afoul of our existing beliefs—even if the grounds supporting that premise are shaky, sketchy, or (shudder) downright scurrilous. Here are some things to look for—likely in the fine print or footnotes—as you evaluate those findings. There can be a difference between what people say they will (or might) do and what they actually will. No matter how well targeted they are, surveys (and studies that incorporate the outcome ...

As human beings, we’re drawn to perspectives, including surveys and studies that validate our sense of the world. This “ confirmation bias ,” as it’s called, is the tendency to search for, interpret, favor, and recall information in a way that confirms our preexisting beliefs or hypotheses. It also tends to make us discount findings that run afoul of our existing beliefs. In its simplest terms then, when you see a headline that confirms your sense of the world, you’ll be naturally inclined to embrace and remember it as a validation of what you already perceive reality to be. Even if the grounds supporting that premise are shaky, sketchy, or (shudder) downright scurrilous. Here are some things to look for – likely in the fine print – as you evaluate those findings. There can be a difference between what people say they will (or might) do and what they actually will. No matter how well targeted they are, surveys (and studies that incorporate the outcome of surveys) must rely on w...

I joined EBRI with a passion for the insights that quality research can provide, and a modest concern about the dangers that inaccurate, sloppy, and/or poorly constructed methodologies and the flawed conclusions and recommendations they support can wreak on policy decisions.  While my tenure here has only served to increase my passion for the former, on (too) many occasions I have been struck not only by the breadth of assumptions made in employee benefit research, but just how difficult – though not impossible – it is for a non-researcher to discern those particulars. We have over the past couple of years devoted some of this space to highlighting some of the most egregious instances, but as I close this chapter of my professional career, I wanted to share with you a “top 10” list of things I have learned in my search to find reliable, objective, actionable research: There are always assumptions in research; find out what they are. Garbage in, garbage out, after all (the harde...