I've been reading about neutron stars, and apparently they can be virtually undetectable if they are not producing pulsars, or are not part of a binary system.

That said, how can we know that any "missing" matter in the universe is dark matter, and not simply neutron stars that we haven't yet found?