17. Richmond, Virginia
The Virginia capital is steeped in history and culture — and if you want more of that, Washington isn't far. But Richmond's notoriously bad schools and rapidly rising housing costs may be why more people want to move away than move in.
The school system is known for its low test scores and graduation rates, and local home prices have jumped nearly 9% in the last year to an average $244,000.
Richmond's strong job market probably isn't driving anyone away: Unemployment was a stunningly low 2.6% in April. The area's major employers include credit cards giant Capital One and used-car retailer CarMax, which is based here.