One line of thinking I use starts with a desire to do no harm.
I see neglecting my own health and joy as a choice for harm. Finding your joy becomes more of a neutral action rather than a selfish choice.
Your fulfillment matters as much as anyone's; it also fuels your ability to help others.