Here's how I look at it. You can obey all the traffic laws drive carefully and still get into an auto accident. That fact though wouldn't justify driving a 100 miles an hour in an old junker with bald ties and bad brakes with the idea being you can get into an accident either way.
So you hire the best men in the county to call a game and because they are human you will still get some bad calls
Still the number and nature of the bad calls is much worse when you hire guys that fired from the lLingerie Legue or division 3 college ball.
To get back to your question, refs are like any other employee. If they do a poor job and don't perform up to expectations you eventually let them go. You generally don't fire them for one mistake though. The Giants didn't fire Eli after his first interception or bench their number one pick running back just because he fumbled once....