Census microdata, such as that produced by the NAPP project, can help illuminate interesting issues surrounding work and labor. Investigating the history of work using microdata must begin with the questions asked by the census enumerators about employment. These varied depending on the country and year, but were generally quite simple. For example, the U.S. 1880 census form had two questions pertaining to work, and a third that hinted toward labor as well.

Historians often try not to fall in love with our sources, but sometimes I just can’t help it. For me, aside from the chatty personal journal (what historian can resist?) and the underground mine maps I’ve studied for years, my greatest fondness may be for big compilations of small bits of data, called microdata. What’s microdata? It’s small bits of information that, by themselves, might be virtually useless, but when aggregated and analyzed can show bigger trends.

This post is an explanation of the napptools scripts, including how they transform NAPP download files into a SQLite database. The tools napptools consists of three script programs: napp2csv.sh: A Bash script that uses traditional unix tools cut, sed, and tr to chop a NAPP data file into its respective columns, guided by a SAS-format command file. This also creates secondary tables in .CSV format from the variable descriptions in the SAS file.