The pair, who spent much of their time on the Xtra Factor flirting with one another, were spotted on an apparent date last week.
Caroline and James went spent a lengthy four hour evening together at London’s Groucho Club earlier this week, looking rather cosy.
"James was staring at Caroline really intensely all night, he couldn’t take his eyes off her," an onlooker told heat magazine. “A few people interrupted them to congratulate James – in fact at one point someone brought over a pair of massive y-fronts to sign which made them both laugh – but more or less they were left alone."
Speaking about his crush on Caroline previously, James said: “I think Caroline Flack is beautiful. Maybe we’ll see in the future. Things like this have to develop.”
According to the Daily Mirror, the pair have arranged to meet up again now that the show has finished.
“James built up a reputation as a ladies’ man over the series and made no secret of the fact he was enjoying all the female attention he was getting," a source told the tabloid. “But while he has been playing the field he has always had feelings for Caroline. She made it clear nothing could happen while the show was still on, but as the weeks went by they got closer and closer.
“At the wrap party on Tuesday – which was held at the Hippodrome Casino in London – they spent almost the entire evening together and there was a lot of flirting. They left separately but arranged to meet again.”