Data to Prototype With

In case anyone else wants to play about with the same data I’ll be using to prototype with, the site to go to is — home of the famous Wisconsin Longitudinal Study, the WLS.   Their data is wonderful stuff, believe me.   I don’t like the way social surveys are done, but I think this the best of a poor lot, at least.    Very good for our purposes.  I have downloaded by the Comma Separated Value, CSV, format data and the Stata data, hoping that the R documentation is correct in saying that R can import Stata data files.   I don’t actually want to use R, though it is a great package, not to mention free, but from R it is easy enough to export it in a format that I can read and manipulate using Python.  I am more concerned about the variable information in the Stata files than the actual user response data, which I could easily read using the CSV files.

All this is for prototyping, doing things with the data that I have mentioned in earlier blog posts.   I will only use a subset of the data, since dealing with more than a little of that massive data mountain (37 megabytes in zipped CSV) would be too much for one person, and I am only prototyping anyway.  On the other hand, I do want the prototype to be capable of expansion and refinement into something real, so I’ll try to avoid limited the data capacity.

I am still going to be posting requirements analysis and design information from time to time, but I need to get more contact with the data and do a bit of coding in order to — well, to keep from being too bored, frankly.   Dealing with data is fun, coding is fun, the paperwork is not.   Not fun, but important.  Don’t think I am minimizing its importance.

The last time I dealt with data from the WLS was 2002, and the survey had not reached 2000 yet.  Now the whole dataset covers the test group from 1957 to 2007, which is a lot better, and more supplementary studies have been added. 

Well, this is something to keep me busy, and will take a while.  If anyone wants to get involved, please, get involved.  — dpw

This entry was posted in Uncategorized and tagged , , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *