The passport is included in the 50 inventions that shaped the modern economy. The modern passport is a relatively recent invention. For most of history, travel documents consisted of a letter of protection, rather than a formal booklet. Governments used these documents to prevent skilled workers from leaving, as well as to control the movement of people from town to town.
It looked like passports were going to disappear in the early 20th century, since free movement of labor ensures that resources will be used most productively. Instead, World War 1 and the resulting concern for security motivated governments to implement additional restrictions on the movement of people. The restrictions evolved into modern passports and border controls, along with the restrictions on where we can work, live, and travel that we live with today.