Lessons Learned

I joined EBRI with a passion for the insights that quality research can provide, and a modest concern about the dangers that inaccurate, sloppy, and/or poorly constructed methodologies and the flawed conclusions and recommendations they support can wreak on policy decisions.  While my tenure here has only served to increase my passion for the former, on (too) many occasions I have been struck not only by the breadth of assumptions made in employee benefit research, but just how difficult – though not impossible – it is for a non-researcher to discern those particulars.

We have over the past couple of years devoted some of this space to highlighting some of the most egregious instances, but as I close this chapter of my professional career, I wanted to share with you a “top 10” list of things I have learned in my search to find reliable, objective, actionable research:
  1. There are always assumptions in research; find out what they are. Garbage in, garbage out, after all (the harder you have to look, the more suspicious you should be).
  2. Just because research validates your sense of reality doesn’t make it “right.” But just because it invalidates your sense of reality doesn’t necessarily make it right, either.
  3. Take the conclusions of sponsored research with a grain of salt.
  4. Self-reported data can tell you what the individual thinks they have, but not necessarily what they actually have.
  5. Sample size matters. A lot.
  6. “Averages” (e.g., balances/income/savings) don’t generally tell you much.
  7. There’s a certain irony that those who propose massive changes in plan design, policy, or tax treatment, frequently assume no behavioral changes in response.
  8. When it comes to research findings, “directionally accurate” is an oxymoron.
  9. In assessing conclusions or recommendations, it’s important to know the difference between partisan, bipartisan and nonpartisan.
  10. In an employment-based benefits system, the ability to accurately gauge employee response to benefits change is dependent on the reaction of the employers who provide access to those benefits.
One of the things that I’ve always loved about the field of employee benefits was that there was always something new to learn, and with each position along the way I have gained a new and fresh perspective.  I’ll always treasure my time here as a member of the EBRI team, the opportunity I’ve had to contribute to this body of work – and I’m looking forward to continuing to draw on EBRI research for insights and analysis in my new position, as I have for most of my professional career.

That said, I’ll close by commending to your attention one of my favorite “lessons” – a quote attributed to Mark Twain, and one worth keeping in mind along with the 10 “lessons” above:

“It’s easier to fool people than to convince them they have been fooled.” 

Here’s to not being fooled.

- Nevin E. Adams, JD

Comments

Post a Comment

Popular posts from this blog

Do Roth and 401(k) Pre-Tax Holders Really Spend Differently?

Image
An interesting—and somewhat counterintuitive—report came out last week, one that cast doubt on the “common wisdom” regarding Roth versus traditional pre-tax savings. The assumption underlying the research [i] was that those who had not yet paid taxes on their savings (the traditional pre-tax savings) would be hesitant to tap into those savings and trigger taxes, certainly more so that individuals that had already paid those taxes. Instead, the research suggested that the opposite occurred; that individuals who had saved on a pre-tax basis actually withdrew more/sooner—but with a twist.  This they hailed as good news. They noted that the research suggests investing in a CT (current-taxed, or Roth) plan could help ease concerns about outliving funds—because they spend at a lower rate (though they saw this as a negative for those who saved on a deferred tax (DT) basis).  They even managed to find a silver lining in the “cloud” of faster spending by the pre-tax crowd beca
Read more

Is the 401(k) Really a ‘Horrible’ Retirement Plan?

Image
  A recent “news” post about a blog post finds some curious “faults” with the 401(k). The news item carried the provocative title “ ‘Rich Dad’ Robert Kiyosaki Reveals Why the 401(k) Is a ‘Horrible’ Retirement Plan .” It originally ran on GoBankingRates.com, but was subsequently picked up in syndication on Yahoo Finance (at least). For the uninitiated, Robert Kiyosaki is one of those “I’ll help you get rich the way I did” types, providing that road map via books (notably “Rich Dad, Poor Dad”) and seminars. He offers comparisons to decisions made by his dad (poor dad) in contrast to those of the father of his best friend (rich dad).  Suffice it to say that his dad worked hard and saved the old-fashioned way—which is NOT his advice to you. That’s included previous pearls of wisdom like “ Why Saving Money is the Wrong Way to Prepare for Retirement .” Apparently rather than saving for retirement, we’re supposed to “look for an investment that will pay for your desired standard of living. Ma
Read more

Shifting the 401(k) ‘Balance’?

Image
A week or so ago, I came across an announcement that IBM was making changes to its 401(k). More specifically that, effective next year they were going to replace their matching contribution in their 401(k) with an employer contribution to a cash balance plan. [i] In the days that followed, the news was picked up in a couple of different trade publications—the implication being that this might be signs of a new shift in plan design. Heck, even Teresa Ghilarducci weighed in , championing the “evolution” to a defined benefit structure from the “flawed” 401(k). She never misses an “opportunity.” Readers here are likely familiar with the basic concepts of a cash balance design. Technically a defined benefit plan, it’s generally referred to as a “hybrid” because it also has a number of participant-friendly aspects that it shares with a defined contribution plan, notably an account balance (though it’s a “notional” one) that is shared with participants. The benefits accumulate
Read more