I really don't know how to respond to that one. It mainly happens when Hispanic people hear me speaking to DD in Spanish in New York. In France, people don't comment on what color Spanish-speakers are (at least not to us directly).
DS has Aspergers and was a very precise and early talker. I got asked how old he was. He was 20 lbs at a year with all his teeth. A lot. Even people who could guess ages well had trouble guessing.
The weirdest thing was the woman who noticed DS's perfect double crown (head worls) at the grocery store and told me they were the mark of the devil. We were deep in colic so she wasn't far off the mark.