I'm 31 years old and spent my 20s stuffing money into retirement accounts — $75K in a Roth IRA and over $100K in a 401(k) — but I'm struggling to buy a home. Was maxing out a big mistake?

But should retirement be our only goal? And how much should we put away?

Is maxing out a mistake?

Tom is 31 and has taken to heart the mantra to save. He earned an above-average income throughout his 20s, and by regularly contributing a portion of his income to retirement accounts and benefiting from employer matching and solid investment returns, has already built a sizable nest egg. He has $75,000 in a Roth IRA and more than $100,000 in a 401(k), as well as an emergency fund with six months’ worth of expenses.

He’d also like to buy a home and start a family with his wife in the next few years. However, despite their savings, most of it is locked away in retirement accounts and they don’t have enough cash for a down payment. Tom is now wondering if he made a mistake by being so focused on his retirement goals.

Using retirement funds for a down payment

There’s some good news for Tom. He’s in a situation where he has enough saved that if he aims to retire at 65 and earns a conservative annual return of 7% investing his retirement funds, he could stop contributing today and have around $1.75 million at retirement. So, he may be able to direct a substantial portion of his savings or take-home pay going forward to a down payment and still be financially prepared to retire when the time comes.

He can free up some of his retirement funds by withdrawing from his Roth IRA. Since contributions to a Roth IRA are made after-tax, you can withdraw them without paying any penalty. In addition, if you’re a first-time homebuyer and your account has been open at least five years, you can withdraw up to $10,000 in earnings that would normally be subject to a 10% early withdrawal penalty if you're below age 59.5.

If allowed by his employer, Tom may also qualify to take a loan up to $50,000 out of his 401(k). This needs to be repaid into the plan with interest, but if you’re using the funds to buy a principal residence, then the term to pay it back can extend beyond five years. If the loan isn’t repaid, it will be considered a distribution and incur taxes and potentially the 10% penalty for an early distribution.

Creating a goals-based financial plan

Tom’s dilemma brings to light the value of having a goals-based financial plan. This type of plan is designed to meet specific financial goals, not just build wealth. It’s based on your unique personal circumstances, such as age, income and risk tolerance, and might include goals such as building an emergency fund, paying down debt, buying a house, paying for your kids’ college education or retiring comfortably.

Start by outlining and prioritizing your goals, then assessing your current financial situation. Once you know where you stand and where you want to go, you can design a plan to reach your goals.

Despite his concerns, Tom is still in a good place. He’s established effective saving habits, which he can re-engage if he decides to adjust his financial goals. He may also further benefit himself by focusing his efforts with the help of a financial planner to achieve both his family and retirement dreams.