The group noticed a notch missing, but as you say, the blade melted away. I don't think they would have had any reason to think that whatever bit was missing didn't do the same. In fact, between Weathertop and Rivendell we have an exchange between Sam and Aragorn:
‘What is the matter with my master?’ asked Sam in a low voice, looking appealingly at Strider. ‘His wound was small, and it is already closed. There’s nothing to be seen but a cold white mark on his shoulder.’
‘Frodo has been touched by the weapons of the Enemy,’ said Strider, ‘and there is some poison or evil at work that is beyond my skill to drive out.
Sam, at least, doesn't appear to have considered that a shard could still be causing trouble, and even Aragorn seems to not know the exact nature of the malady.
Also, I don't think Gandalf is implying that Elrond didn't think there was a splinter left. If he's contrasting with anyone in particular, I'm guessing it's with the other hobbits who were hopeful because of Elrond's reputation.
