According to journalist Tom Bower, author of Simon Cowell's unauthorised biograpy, Simon reckons the four stars would have brought in the most viewers.
In an interview with heat magazine, The Intimate Life Of Simon Cowell writer claimed: "His ideal line-up this year probably would have been Cheryl and Dannii with Gary and Louis.
"He would have very much liked Dannii there – he thinks that would have brought in a lot more viewers and it would have been a better show.”
It was revealed recently that Simon was so keen to get Dannii back on the show this year, offering her up to £2 million to join the panel.
According to new details in the music moguls ‘unauthorised’ biography, Simon urged the show’s team and ITV to splash out millions in order to secure a deal.
Simon is quoted as telling ITV director of television Peter Fincham: “Bring her back to X Factor… everyone will be looking at her t*ts.”
Dannii confirmed previously she was offered the chance to return, but decided it wasn’t the right thing for for. She said: “It didn’t fit in with arrangements I’d made for Australia’s Got Talent.
“I have been with AGT since the first episode and it’s filmed in my home town. I had four incredible years on The X Factor but the show has moved on and so have I.”
Dannii added: “The X Factor will always have a special place in my heart. I’m still in contact with the contestants.”
Meanwhile Tom Bower claimed that Simon Cowell will definitely pop up in The X Factor 2012 final this year.
He said: "There’s no doubt he thinks he’ll come back for the final. The role that he’ll play will become clearer in a few weeks."
The X Factor 2012 continues on Saturday and Sunday on ITV1.