The Very Spring and Root

An engineer's adventures in education (and other musings).

This content shows Simple View


Boskone: Dataliths – Digging the Idea of the Programmer/Archaeologist


Dataliths: Digging the Idea of the Programmer/Archaeologist
Our GOH Vernor Vinge has posited that as computing-based civilizations age, layers upon layers of legacy code build up in vast — let’s call them dataliths. Who gets to dig through them for valuable info? How do they do it? Isn’t our data already in pretty deep doodoo in this regard?
Janice Gelb (M), Charles Stross, Vernor Vinge, Gary D. McGath, Dana Cameron

Stross opens by mentioning that we are obsolescing file formats at an ever-growing rate. Some of this is intentional, related to corporate greed, media consumption, and encryption, and DRM. Brings up the example of Microsoft’s .lit format, which was trying to compete in the ebook market but then was discontinued. Now the license servers for the protected format have been shut off, because there is no financial incentive to keep them up, which means that content that people bought in .lit is now unreadable.

McGath points out how ironic it is that we are in an age of so much data, yet so much of it is actually inaccessible. Calls it an approaching digital dark age.

Stross starts nerding out about an idea he has had for super-compressed solid state information storage, called memory diamond. Carbon-12 would be a 1 and Carbon-13 would be a 0 for example, and you could compress information into a very dense space. Note: I think this is a cool idea, but I also note that it still considers information storage in the paradigm of binary digits. What about quantum computing, which is near on the horizon? Or other natural phenomena which have many more possible states that two, and in which information could be usefully embedded?

Vinge talks about how we have so much redundancy of information. The same file has zillions of copies around the world, all of which have to be stored. He then moves to a larger topic of what do we do when civilization falls? How will we preserve our knowledge and culture for future generations and civilizations? We can’t rely on a particular data format that is proprietary and would never be resurrected. We would need stacked layers of ever more complex generations of data, that could be read and reinterpreted after a fall.

Cameron (the archaeologist): We would need something like a Rosetta Stone for data, for future civilizations to access our culture. I am trying to think about what an archaeologist of the future would want to know, and how best to store and format that information for them.

McGath counters that it is a tricky thing to try and determine the line between what we want to preserve and what we should preserve.

Cameron: culture through the eyes of individuals is the holy grail of archaeology and anthropology. With data we have an amazing opportunity to have that continuous spectrum of the broad down to the specific and then back up again. Even the mundane details of everyday life would help inform theories and ideas about the macroscopic scale system.

Stross wonders about convergence instead of divergence, citing figures that 80% or so of the operating systems out there have converged to either iOS or Android, and these have very similar architecture and heritage. (iOS is from Unix/BSD line and Android from Linux). Note that he is including the huge number of mobile devices out there, which more and more are outnumbering actual “computers” in the desktop and even laptop sense. 

Audience questions conclude with interesting discussion about the role of libraries, particularly public libraries in storing, archiving, and retrieving the data of an age. Calls for innovation on this front.