Unfollowing, as well as blocking and muting, hold absolutely no moral values; being unfollowed/blocked/muted does not mean you did or say anything bad or wrong
Equally, anyone who use these tools should not be demonized for curating their online space, regardless of the reason