George Orwell’s clas­sic alle­go­ry, Ani­mal Farm, presents many per­spec­tives on human behav­iour and soci­ety. One of these is how peo­ple can be led and manip­u­lat­ed through the con­trol of infor­ma­tion. In the sto­ry, the Sev­en Com­mand­ments formed a de fac­to con­sti­tu­tion for the Ani­mal­is­tic soci­ety. Since only a hand­ful of ani­mals could read, the rest were depen­dent upon what they were told was writ­ten. Grad­u­al­ly, the writ­ing was cun­ning­ly altered to the ben­e­fit of the pigs above all oth­er ani­mals, and the pop­u­lace was taught to not trust their rec­ol­lec­tions of what was writ­ten in the past.

What made this sub­ver­sion pos­si­ble was the inabil­i­ty of most ani­mals to read. The two ani­mals that could read (aside from the pigs) chose not to do any­thing about what they saw. Amongst oth­er things, the right to access and read infor­ma­tion is an impor­tant cor­ner­stone of democracy.

This is where open file for­mats come in. As our lives become increas­ing­ly defined by elec­tron­ic records, there needs to be a way for inde­pen­dent view­ing and audit­ing. Paper is eas­i­ly read, but com­put­er files require soft­ware to decypher them. Imag­ine if you need­ed spe­cial (and expen­sive) glass­es just to read the let­ter that you your­self wrote only a few years ago.

There has been a fair amount of dis­cus­sion in the press regard­ing the Open­Doc­u­ment and the so-called ‘Open’ XML for­mats. The pri­ma­ry focus of this report­ing thus far has been on the polit­i­cal and tech­ni­cal facets. This is slow­ly chang­ing, as the impor­tance of long-term data preser­va­tion and free­dom of infor­ma­tion become appar­ent to ordi­nary folk.

The BBC has pub­lished a report on the prob­lem, and dis­cuss­es how the UK Nation­al Archives are attempt­ing to deal with it. Alas, it appears that they have opt­ed for a short-sight­ed approach, rely­ing on vir­tu­al­i­sa­tion of old­er oper­at­ing sys­tems and appli­ca­tions, through a direct part­ner­ship with Microsoft. With this approach, the for­mat decoders/viewers (not to men­tion the oper­at­ing sys­tem and soft­ware per­form­ing the vir­tu­al­i­sa­tion itself) remain closed in source and spec­i­fi­ca­tion, and one must deal with a cum­ber­some vir­tu­al machine just to view a document.

Where is the guar­an­tee that files can be read hun­dreds of years from now, just as we can do today with paper doc­u­ments such as the his­toric Magna Car­ta? How does this part­ner­ship ben­e­fit me, an ordi­nary cit­i­zen who might wish to view ten- (or even two-) year-old pub­lic doc­u­ments that are only avail­able in a pro­pri­etary elec­tron­ic format?

It’s both sad and frus­trat­ing to see that his­to­ry is yet again repeat­ing itself. Whilst the con­tents of the Domes­day Book can still be read near­ly 1000 years after com­ple­tion, the dig­i­tal BBC Domes­day Project was ren­dered vir­tu­al­ly unread­able a mere 16 years later.

Thank­ful­ly, there are efforts to cre­ate an infra­struc­ture for long-term preser­va­tion and man­age­ment of dig­i­tal doc­u­ments. To start with, there are open for­mats such as Open­Doc­u­ment and PDF. The Aus­tralian Nation­al Archives have long been sup­port­ers of Open­Doc­u­ment, to the extent that they are stan­dar­d­is­ing upon it. Putting their mon­ey where their mouths are, they are build­ing a com­plete­ly open source (GPL, no less) data man­ag­ment sys­tem that any­body can use or improve to suit their needs. Michael Car­den gave a great talk [Ogg video] at this year’s about this tech­nol­o­gy, known as Xena [PDF]. Whilst their UK coun­ter­parts seem to have for­got­ten that access to data is not just a priv­i­lege for those able to make exclu­sive agree­ments with pur­vey­ors of lock-in tech­nolo­gies, the Aus­tralian Nation­al Archives have been striv­ing to ensure that nobody is left out of the dig­i­tal revolution.

Four legs good, two legs… bet­ter? Let’s pre­vent this sub­ver­sion from happening.


LotD:  Mex­i­can ‘world’s rich­est person’

Four legs good, two legs bad! / Sridhar Dhanapalan by is licensed under a Creative Commons Attribution-ShareAlike CC BY-SA 4.0 licence.