[Richard]

And if he gave a crucial decision in Australia's favour at a crucial time?

IMO taking away any remote suspicion of bias, especially in such an important game, is the only way to go.

Everyone who is serious about the matter realises that independent Umpires are not a result of bias, but a result of the suspicion of bias. However good an Umpire is, that will never, ever go away. The only way to stop it is to remove all Umpires from matches involving their own team.