Almost all of them have to work to live somewhat comfortably. They also have to do their own housework and childcare in addition to paid labour, which rich people do not. Both of those, especially in combination, are stressful. Stress ages you.
Once you get to a certain amount of wealth, you never have to do labour again, paid or otherwise, unless you want to.
Not to mention that they can’t afford dieticians, personal trainers, aestheticians, etc., as the rich do.
If they stopped working, they would eventually have to worry. That is the fundamental difference.