I mean, I get what you're saying, but right wing parties get elected in times of prosperity as well.
Racism is just their appeal to the masses. Even now, you describe the median voter as "clueless" as if they're almost a passive party being duped into Hitlerite fascism.