What evidence can you find to indicate that slavery was an issue for all the colonies, not just those in the south?

We are often led to believe that slavery was restricted to the southern colonies. What evidence can you find to indicate that slavery was an issue for all the colonies, not just those in the south? Why do you think the belief that slavery and all its problems was confined to the southern colonies and later to the southern states continues to have credence in contemporary society