I don't recall if he was worked into the Season 3 script outlines at any point. Are we supposed to add his cameos to the outlines retroactively as we think them up?
I like the idea that his territory used to be much larger. It could be all of Eriador, Enedwaith, Minhiriath, Ossiriand, and most or all of southern Beleriand (Taur-im-Duinath, the lower Sirion, etc.).
About endearing him to the audience, instead of just having him appear in the background we could have him (briefly) singing nonsense, or gathering flowers, or talking to random animals. Or even show scenes out of "The Adventures of Tom Bombadil" and "Bombadil Goes Boating".
In Season 5 we could show the Hadorians or Folk of Haleth in Ossiriand (before meeting the Noldor). They could catch sight of him singing in the background and somebody could exclaim "Look, it's that crazy man again!"