Why hasn't this been done instead of Congress giving themselves vacations so often?
Under current House rules, it takes only one single member of Congress to kickstart the process of house speaker removal, but a simple majority of those present and voting to actually remove the Speaker.
