WLS SAS Catalog or Command Files and Bit Arrays

I was a bit worried about getting the formatting data out of the SAS Catalog or Command files in the Wisconsin Longitudinal Study.  These are ASCII files, and necessary to read the CSV data files, but they are designed to be read by SAS, not by some program I might write.   The answer to this is surprisingly simple.  I used an ordinary text editor to strip off the first few lines and all the format lines at the end, then saved the file as a variable and value file,  the whole of which is in one single format, like this:

value SEXRSP /* sex of respondent */
      1 = ‘male’ 
      2 = ‘female’ ;

Then I did the opposite, saving only the format lines, which maps variables to possibly new names:

format    DEATYR DEATYR.;
format   GROUP91 GROUP9A.;

These two new files are easily readable by a program which will be easily writeable.

It seems that the Python package Bitarray will work.  It returns only a one dimensional array of bits, but as many as necessary can be put in a list.   This will create something like a two dimensional array of bits.  I  need columns to be in the one dimensional arrays, so I will have to transpose the data.  Did you know that you can transpose a two dimensional list of lists in a single line of Python code, using the map and zip functions?  I don’t know if it will work for bit arrays yet, and especially don’t know if it will work for something huge, but I’ll try.  — dpw

This entry was posted in Uncategorized and tagged , , , , , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *