The actor, who only lasted one series playing the Time Lord in 2005, says he left after he 'clashed with bosses'.
In an interview with Radio 4's Loose Ends, he explained: "I don't think it's important that I left - I think it's important that I did it in the first place.
"I'm still there - I was in David Tennant, I was in Matt Smith, I was in Peter Capaldi. I'm always there in spirit."
"Myself and three individuals at the very top of the pyramid clashed so off I went.
"But they're not here to say their side of it so I'm not going into detail."
The 'Safe House' actor went on to say he believed he had made mistakes in how he played the role.
"I think I over-pitched the comedy," he explained. "If I had my time again, I would do the comedy very different - but I think where I did possibly succeed was in the tortured stuff - surprise surprise!"
Since leaving 'Doctor Who', Christopher has gone on to bag himself other TV and film roles including 'The Shadow Line', 'Blackout', 'Fortitude' and 'Thor: The Dark World'.