Given that all the surviving non-IE languages in Europe except Basque entered within the past 2000 years, why would you expect more in Germany or France? (Treating the west end of the Eurasian steppes as not Europe, so Finnish and Hungarian do not count). They are far away from places where large numbers of non-IE speaking immigrants could arrive, and the Roman empire strongly encouraged people to learn one of the big four (Latin, Greek, Punic, or Aramaic). There are hints of a dozen or so non-IE languages such as
Lemnian and
Etruscan in the Mediterranean before Augustus, but the evidence fades afterwards, because the people who could write and put up monuments mostly wanted to use a language that other powerful people could read.