Internal legibility and external availability

From Naked Prey by John Sandford published in 2003.  They are trying to track down the owner of a red Jeep Cherokee.  The auto registration data might be from Minnesota but equally likely might be in any of the surrounding states.  

“Did you hear anything from St. Paul about tracking down the Cherokee?”

 “If you go back a month, you can find maybe thirty Cherokee transactions.  We’ve got the names on those, and we’re working with North and South Dakota, Missouri and Iowa. I think Iowa’s in, haven’t gotten word from the others yet. I’m not sure South Dakota is computerized enough to get what we need that quick.”

I don't know why this especially crystalized a memory but it does.  

That period circa1995-2005.  In the late 1990s we were implementing ERPs at a blistering pace, making companies legible to themselves, to use James C. Scott's phrase (from Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed).  Often, legible for the first time.  

Then, simultaneously from 2000-2005 we were building the first web interfaces for companies.  They had the capacity to leverage internal legibility into an external presence.  

Corporations were one thing, they tended to move in broad alignment, though at different paces, toward common informational goals.

States and Agencies were a whole different beast, as reflected in the above description where all the states have the data (paper forms) but only some of them have it usefully available (it has been digitized) and even fewer have it accessible (via the internet).

It reminded me of a project circa 1986 or 1987.  We had a Florida utility company client which had some matter before the Florida Public Utility Commission.  We had been retained to help construct the evidentiary base for their position and in order to do so, we needed to empirically compare their operation and cost structure with other utility companies in Florida.

All the data we needed was on the annual FERC Form 1 submitted to both federal government and to the Florida Public Commission by each utility.  FERC Form 1 had the advantage that it was used nationwide, had existed for a long time and the data was broadly reliable.

It had the disadvantage that it was hardcopy.  If we wanted the information for all Florida utilities we had to travel to Florida to get the forms.  

I was given the job of flying down to Tallahassee for a week and both photocopying all the forms for all the Florida utilities as well as beginning to capture the data for analysis in Excel.  No scraping tools or even OCR.  This was a straightforward transcription from paper to digital data.  

Even at the time it seemed a ludicrously cumbersome mechanism for capturing data but for all that it took time and was expensive, it made sense to do as the value to the case far exceeded the cost.

Almost certainly, today everything is online.  In fact, I just went online and got the data I collected in Florida way back then, free and immediately.  What took me a whole week to capture in 1986, while paying for travel, hotel and meals, could be done free in perhaps three-to five hours today from my desk,                                                     

But, per Sandford, there was an odd window in there when data availability was patchy owing to whether the company or entities had the internal legibility (ERP implementation) or not in combination with the degree to which they were willing to put any data online.  Today they virtually all have the internal legibility and almost all make most or much of that data available online.

We have travelled a long distance in a short time.