From millionaires to Muslims, small subgroups of the population seem much larger