Harbaugh has had what - 4 gigs in 16 years? All leaving for greener grass.
That's an average of 4 years each. I'll stand by my assertion. He had pretty much worn out his welcome at each place before he left. Notwithstanding the one Super Bowl appearance (which you really do have to give him credit for), he never quite lives up to the expectations that he leads fans to believe he will achieve when he arrives and then he grates on them and the organization with his odd behavior and demanding personality.
Most people I have heard that have had interactions with him come away not being fans. I am not a UCLA or Rick Neuheisel fan by any means, but I have heard him tell several stories about what a jerk Harbaugh is.