Having now seen Episode 3, I can understand these concerns better. In the first two episodes, we are introduced to this culture and their way of life. It's fairly idyllic, if also simple hunter-gatherer as well. So, yes, the characters are dirty, and have baggy clothes, and messy hair with stuff stuck in it. They also come across as simpletons, to some extent. But...overall, the portrayal is strongly positive. So, it doesn't seem as though there would be much to complain about, if there is a real-world connection. Who wouldn't like these Harfoots?
My impression of Episodes 1 and 2 was that the leader (Sadoc) was perhaps the only literate character. Now, we know that Nori also is expected to be able to read. So, literacy may not be universal among them, but it's more widespread than just one person. We still don't know the origin of the book and scrolls, though.
But Episode 3 shows the migration happening, in a caravan where each family must pull their own cart. It is also the case that if you fall behind...you are left behind. There is an ominous ruthlessness to their approach of 'no one left behind'. It is also revealed that the punishment for breaking their many secrecy rules is ostracization/exile. So, maybe not so idyllic after all, and maybe not so entirely likeable. Which is fine - no actual culture is beyond reproach or criticism, so it makes sense that there may be aspects of this one that are...not so great.
But that is where it becomes tricky if the audience is being invited to make real-world parallels. The accent may be pointing Irish (I wouldn't know), but the caravan method of travel could point to the Roma. And there are a lot of existing stereotypes and prejudices there as well.