James and Ella were in the bottom two earlier this month, a shock result that saw the judges send the vote to deadlock.
Ella left, with James having received more public votes, knowledge that he says has made him believe he can get to next month's final in Manchester.
He said: “It’s given me an added boost — I don’t fear anybody now,” he says.
“To go up against Ella and come out on top, I feel I might have booked my place in the final.”
Speaking to The Sun recently, James thanked the show for helping his personal life too, having been sleeping rough after a rocky relationship with his parents.
“I’d still be living that life if I’d not had The X Factor — it has changed my life,” he told the newspaper.
“My lowest point was when I didn’t have anywhere to stay. A couple of times I had to sleep on a park bench or something like that. I just felt alone and had trust issues. I was lashing out at the world as I didn’t really understand my situation.”
Now things are on the mend, with James revealing: “I decided I would go back to the roots of any problems.
“I took up any issues that I had with my family and everyone was really mature about it. It wasn’t easy but it all worked out.”
He continued: “It’s been a while since I’ve felt part of something,” he admits.
“I love Jahmene — he’s just an amazing guy. Rylan’s very funny, he’s a joy. And I’m close to Josh in Union J.”
James added: “Going from the contrast of a s*** life to having the best one possible, and winning the biggest show on TV, it would be the greatest sense of achievement,” he says.
The X Factor 2012 continues this weekend on ITV1 and ITV1HD with the final four acts singing songs that they hope will get them into the final.
On Sunday's results show, one more act get the axe and there will be performances from Pink and Tulisa.