A quick bonbon! The global distribution of human beings by altitude: a histogram showing the number of people living at every elevation. Not surprisingly, coast-loving humans are a low-altitude species, and the distribution of humans is quite a bit lower than land in general — not even counting ice domes and barren deserts. Quick take-away: when you look out from the top of the Washington Monument, you are higher than half of everyone else in the world.
I also found some additional data for the early decades of American slavery: 1790 data for what's now Tennessee, plus small tweaks to coastal South Carolina and Indian lands in Kentucky before 1820.