Hannibal also left troops to guard his own camp. Didn't the remaining legionaries attack Hannibal's camp as well and Hannibal had to relieve them after holding the field? So Hannibal wouldn't have as much troops.The topic of numbers at Cannae is not one that I have thought too deeply about, certainly not enough to have a strong opinion, but to show the other side of the debate, I'll quote Lazenby 1978, Hannibal's War, 75-76:
'But the most important question concerns the number of legions Varro and Aemilius Paullus eventually had at Cannae. Polybius states quite unequivocally (3.107.9) that it was decided "to maintain the struggle with eight legions, a thing which had never happened among the Romans before, each of the legions having up to 5000 men, apart from the allies," and this, if true, means that it was decided to raise four new legions, each of 5000 men, and to bring the existing four under Servilius and Atilius up to the same strength. He also believed that an equal number of allied infantry was levied, and consistently with this says that the Romans had 80,000 infantry at Cannae (3.113.5). These figures have been doubted, because Livy (22.36.1-4), although he reports this tradition, also records a variant one - that only 10,000 fresh troops were raised - and it is argued that the smaller is to be preferred a priori, and that Hannibal's tactics at Cannae are unintelligible if he was really outnumbered by nearly 2:1 in infantry. But the a priori argument is not necessarily to be accepted - it is very difficult, for example, to accept the lowest of the three estimates Livy gives (29.25.1-4) for the army Scipio took to Africa in 204, and some scholars would accept the highest. But a more serious objection to accepting the tradition that only 10,000 fresh troops were raised for the Cannae campaign, is that this would have only given the Romans a slight numerical advantage, even if the four existing legions under Servilius and Atilius were strengthened to 5000 men, and that as many allied soldiers were levied as Roman. For even if we accept that all the 10,000 were citizen soldiers, that would still only have meant 30,000 citizen troops in all, with 30,000 allied infantry in addition. But Hannibal had 40,000 infantry (3.114.5), and some of the Roman infantry were bound to be left to guard the camp on any day of battle - Polybius indeed believed that 10,000 were left to guard the larger of the two camps, and Appian (Hannibalic War 4.26) mentions 5000 in the smaller, for what it is worth. Thus Varro and Paullus, we are being asked to believe, were left to face Hannibal's 40,000 infantry with possibly as few as 45,000 of their own, while their 6000 cavalry (3.113.5) were actually outnumbered by Hannibal's 10,000 (3.114.5). If this was the case, the Romans had certainly learnt nothing from the Trebbia, for there they had tried to fight Hannibal with a slight numerical superiority in infantry, and with fewer cavalry, and they had lost. Moreover, it might be argued that Hannibal's tactics, far from being unintelligible if he were really outnumbered by something like 2:1 in infantry, would be unintelligible if his infantry on the field was nearly as numerous as the enemy's.
Finally, it is perhaps significant that Livy's circumstantial account of the Roman losses at Cannae, and of those taken prisoner or who escaped, presupposes the higher number: in particular, if twenty-nine military tribunes were really killed (22.49.16), there must have been at least five legions present, since there were only six such officers to a legion. In other words, Livy's figures for the Romans killed, and for those who survived whether as prisoners or fugitives, support Polybius' figures for the size of the Roman army, and it cannot be too strongly stressed that Polybius is quite emphatic about his figures, though he obviously realised that they were unusually high.'
Logically, if there were two camps for the Romans logic dictate that they must face an opposing camp of Hannibal, or otherwise what is the purpose of maintaining a camp with a legion in it?