While I agree that historical reality is something we should keep an eye on, one would be hard-pressed to find a historical culture of Neolithic homestead farmers who start to coalesce into a single people in response to encounters with flying, supernatural monsters. To say nothing of the rumors of Orcs, or even the mysterious and (lightly) super-powered Elves.
Doesn't mean we have to throw historical cultures out the window, but we can certainly take a bit of artistic license.
So, what the story that has been decided upon calls for is a hall or longhouse that can _hold_ somewhere around 700-900 people in a dire emergency, or _house_ about 100-200 people for a week or two.
The Iriquois had structures that if scaled up.could certainly fill the bill
View attachment 3708