people weren't necessarily people

The Indian Removal Bill, including the Five Civilized Tribes, was favored by President Andrew Jackson who made it the law of the land. Whites now had a chance at Indian lands. 

An earlier reference to Jackson here.