A common observation about Millennials is the generation’s penchant for spending on experiences over goods like cars or homes. In fact, a recent Harris Group study found 72% Millennials plan to focus on “experiences rather than physical things” in their future spending.

That has led to some hand-wringing about a young generation spending too much at bars and restaurants, but it’s not really spending habits that are the biggest problem for Millennials in general. The most significant financial challenge facing 20-somethings and 30-somethings is a structural problem in how they will be able to save and invest for retirement.

Saving for a distant retirement is always a hard concept for younger Americans, who are more inclined to live in the now. And the slow death of pension plans over the last few decades has made things even harder.

But in addition to these typical hurdles, Millennials also face a mathematical double whammy of lower lifetime earnings and lower investment returns that will make their challenge even more difficult.

On the income front, consider a Yale economist’s study of earnings from college grads, which showed “large, negative wage effects to graduating in a worse economy.” In some cases, they saw as much as $100,000 less in cumulative earnings over the next two decades vs. those who graduated into more favorable times. Many Millennials, of course, hit the job market smack dab during the Great Recession.

And on the investing front, consider a recent McKinsey & Co. report titled “Diminishing Returns: Why Investors May Need to Lower Their Expectations.” The name says it all, but the gist is that the consulting firm forecast 4.0% to 6.5% returns annually from the stock market across the coming years — down dramatically from the 7% to 10% annual gains common over the past 50 years or so.

More: Expand this story & watch video, tap here