“This is an attempt to stop a war. I hope it is not too late and that somehow, magically perhaps, peace will prevail again.” (Cohen 1980)
Let’s begin this tale of the conflict you’ve probably never heard about with a quotation from Gulliver’s Travels:
It began upon the following occasion. It is allowed on all hands, that the primitive way of breaking eggs, before we eat them, was upon the larger end; but his present majesty’s grandfather, while he was a boy, going to eat an egg, and breaking it according to the ancient practice, happened to cut one of his fingers. Whereupon the emperor his father published an edict, commanding all his subjects, upon great penalties, to break the smaller end of their eggs. The people so highly resented this law, that our histories tell us, there have been six rebellions raised on that account; wherein one emperor lost his life, and another his crown. These civil commotions were constantly fomented by the monarchs of Blefuscu; and when they were quelled, the exiles always fled for refuge to that empire. It is computed that eleven thousand persons have at several times suffered death, rather than submit to break their eggs at the smaller end. Many hundred large volumes have been published upon this controversy: but the books of the Big-endians have been long forbidden, and the whole party rendered incapable by law of holding employments. During the course of these troubles, the emperors of Blefuscu did frequently expostulate by their ambassadors, accusing us of making a schism in religion, by offending against a fundamental doctrine of our great prophet Lustrog, in the fifty-fourth chapter of the Blundecral (which is their Alcoran). This, however, is thought to be a mere strain upon the text; for the words are these: ‘that all true believers break their eggs at the convenient end.’
For Jonathan Swift, the wars of religion and generations-long rivalries between Britain and France were precisely as arbitrary as a decision about which end of a boiled egg to crack. Since it really doesn’t matter, everyone should do as they see fit, and ‘break their eggs at the convenient end’. The question remained: which end would that be?
For modern day computer engineers considering which byte order to set their hardware and software to, the decision is still fairly arbitrary, but it matters at least a little which end is adopted. Rather like driving on the left or right, no one really cares which is which as long as everyone agrees on a convention, but also like driving on the left or the right, no one is prepared to shift their preference, and when pressed will provide justifications for the status quo. Unlike driving on the left or right, however, those justifications do have some reasoning behind them. It actually is possible to come up with a list of why a particular byte order is useful for particular purposes, and therefore to prefer one order or the other on somewhat rational grounds.
Danny Cohen’s article, the one that announced the war in the first place, lent its central metaphor of big endians and little endians to the concept of byte order:
“Endianness describes how multi-byte data is represented by a computer system and is dictated by the CPU architecture of the system. Unfortunately not all computer systems are designed with the same Endian-architecture. The difference in Endian-architecture is an issue when software or data is shared between computer systems. An analysis of the computer system and its interfaces will determine the requirements of the Endian implementation of the software. ” (Intel white paper, quoted in Blanc and Maaraoui 2005:2)
For instance, Intel x86 architecture is little-ended (see LaPlante and Mazor 2006 for a history), while Motorola architecture is mostly big-ended. ARM architecture and some others, meanwhile, have switchable endedness, which tends to be set one way or the other by default. The distinction between Intel chips used in most IBM compatible PCs and Motorola chips used in most Apple computers meant that the byte order issue was, to say the least, overshadowed by intense marketing competition and rivalry during the 1980s and into the 1990s. Behind the embryonic “I’m a Mac – I’m a PC” rivalry lay another much less glossy rivalry, between big-endian and little-endian architecture.
In seeking a resolution to these fairly arcane distinctions in computer architecture, Danny Cohen identified a number of alternatives, which, as it turns out and though he probably didn’t know it, relate quite well to the four cultural biases or worldviews identified by the anthropologist Mary Douglas in her Grid-Group Cultural Theory . The first was continued ‘holy war’. This is what Lilliput was pursuing against Blefuscu and, according to Cohen an appropriate metaphor for the standoff between big-enders and little-enders in computer communications architecture. It seemed likely to continue because:
“Each camp tries to convert the other. Like all the religious wars of the past, logic is not the decisive tool. Power is. This holy war is not the first one, and probably will not be the last one either. The “Be reasonable, do it my way” approach does not work. Neither does the Esperanto approach of “let’s all switch to yet a new language”.”
An alternative was to somehow supply a big man like Gulliver to come and sort out the problem from on high. Here we see the strong Grid – strong Group approach of the Hierarchical cultural bias:
“We would like to see some Gulliver standing up between the two islands, forcing a unified communication regime on all of us.”
Yet another alternative offered by Cohen was to trust to chance and abide by the dictates of fate:
“How about tossing a coin ???”
This is a nod to the strong Grid – weak Group cultural bias of Fatalism.
Cohen, like Swift before him, was not terribly optimistic that conflicts over arbitrary decisions would be resolved amicably. He forecast:
“Our communication world may split according to the language used. A certain book (which is NOT mentioned in the references list) has an interesting story about a similar phenomenon, the Tower of Babel. Little-Endians are Little-Endians and Big-Endians are Big-Endians and never the twain shall meet.”
Cohen was writing in 1980. Computer engineers have now had thirty years to resolve the war of the byte order. So how have they fared?
What happened next? How did things pan out? Which end of the egg triumphed?
Ten years after Cohen’s article, in 1990, the IEEE Computer Society was advised that ‘the endian wars continue’.
Slightly ironically perhaps, the author of the paper in IEEE Micro promoted a big-endian resolution to the dispute.
In 1996 Verts noted that the quasi-religious fervour of the debate was still evident:
“You may see a lot of discussion about the relative merits of the two formats, mostly religious arguments based on the relative merits of the PC versus the Mac. Both formats have their advantages and disadvantages.”
He listed a number of common file formats and observed that they required differing byte ordering. For instance GIF image files were little endian while JPEG files were big endian. However, some file formats written for big endian architecture actually required little endian byte ordering and vice versa. For example, QTM format for Quicktime movies, written for Motorola big endian architecture, required little endian byte ordering, while WPG, Wordperfect Graphics format written for PC little endian architecture, required big endian byte ordering.
Phew.
A 2001 introduction to endianness claimed that the matter hadn’t been resolved:
“Unfortunately, both implementations continue to be prevalent. Embedded programmers must be aware of the issue and be prepared to convert between their different representations as required.”
A 2004 Intel white paper identified opportunities and guidelines for adopting ‘endian-neutral code’
Twenty five years later in 2005 endianness, according to Bertrand Blanc, remained a problem to be overcome, although the prospects for endian-neutrality seemed stronger than previously and the idea of a big-endian takeover appeared considerably less attractive.
In 2008, endianness was still an issue to be considered, (“We must take endianness into account when analyzing memory accesses.”) although in binary analysis a technique called ‘memory normalization’ went some way towards resolving it.
By 2009 resolution was well in sight, by means not of rapprochement, but of established conversion protocols. To give just one example, the Qt toolkit was developed to enable portability of binary applications across platforms.
“To ensure cross-platform compatibility for binary data, you need to specify the order explicitly when writing and again when reading. By using a QDataStream to handle binary file formats, endianness no longer is an issue. You simply specify the byte order to use and then use the stream operators, and it just works.” (Thelin 2009)
This kind of conversion is as simple as the following line of code:
stream.setByteOrder( QDataStream::BigEndian );
War over? Not quite. A note of warning was sounded by Werner Benger (2009) in relation to reading binary data for visualization purposes:
“you might still have to deal repeatedly with supposedly solved problems, such as byte orderings (little-endian vs. big-endian)”
In his article in Computing in Science and Education he commented:
“A big hurdle for establishing a standard file format for scientifc data is the incompatibility of data structures themselves. Not all file formats can cover all data types that occur in scientifc simulations and visualization. A standard file format that enables true interoperability across independently developed applications would need to cover all different cases within the same model. At this point, philosophical beliefs come into play: many people simply don’t think this is possible at all. Rather, the mainstream approach appears to be “special problems need special solutions,” which results in myriad specialized file formats and a list of special cases to be handled by some visualization application. Such beliefs are hard to overcome.”
Benger considers three partial solutions to the problem of data visualization in the shape of three potentially universal file formats, Open DX, HDF5 and F5. He favours F5 but sees this as only a partial solution. He points out that
“No one wants to invest the time needed to implement file converters at the expense of research hours.” (2009: 102)
So here we are, thirty years into the holy war proclaimed by Cohen. And how far have we come?
It is probably fair to say that peace has in fact broken out, but it is a de facto truce rather than defeat or victory for one side or the other. Endianness persists and still causes portability and file-readability issues. To the extent these have been overcome it is largely by means of conversion algorithms rather than by somehow abolishing either big-endedness or little-endedness.
How did peace break out? A giant like Gulliver did not appear to make a ruling one way or another. The toss of a coin was not made to decide arbitrarily which protocol to follow. The ‘war’ was not ‘won’ by either big-endian or little endian architecture. Both still exist and thrive.
In the absence of the Strong Grid approaches of Gulliver or the Fatalist coin toss progress has been by means of Weak Grid measures. Individualism has prevailed in the sense that there is still real choice in endian architecture. The ‘mainstream’ approach identified by Benger, that ‘special problems need special solutions’ is a classic Individualist position: there is no such thing as standard. But Individualism couldn’t go it alone: the incommensurability and lack of portability it entailed would have seriously damaged the prospects for interoperability, were it not for the Egalitarian moves of conversion software.
In the absence of either Hierarchy (a strong regulatory regime that would have cared about byte order) or Fatalism (a system devoted to the the promotion of contingent events) the Individualist bias dominated the endian debate: may the best byte order (for my purposes) win. This could indeed have led to a split in the world of communication. Like the humans at the fall of the tower of Babylon, the computers could have finished up talking past one another in mutually unintelligible languages. Yet what saved the day was the Egalitarian intuition that faced with difference we should recognise our underlying similarities and simply translate our way to harmony. It turns out that byte order is very well suited to translation.
Perhaps if the Cultural theory analysis, or something like it, had been adopted more explicitly in 1980, instead of the holy war metaphor, progress might have been a little quicker.
So now what other issues are construed as holy warfare when they might better be seen as conflicts over cultural biases reflecting plural, but not endless, rationalities?
And what other, perhaps more pressing, issues can learn from the resolution and non-resolution of the Endian holy war that lasted almost 30 years without most of us noticing a thing?
References:
Bertrand Blanc and Bob Maaraoui (2005) Endianness or Where is Byte 0? White paper. 3b Consulting Accessed at http://3bc.bertrand-blanc.com/endianness05.pdf
Michael Barr 2001 Introduction to Endedness. Embedded.com Accessed at http://www.embedded.com/columns/beginerscorner/9900450?_requestid=65634
Benger, Werner (2009) On Safari in the File Format Jungle – Why Can’t you Visualise my Data? Computing in Science and Engineering (November/December), 98-102. Accessed at https://www.computer.org/cms/Computer.org/ComputingNow/homepage/2009/1109/rW_CS_SafariFileFormatJungle.pdf
Chen, Hui-l, Zhou, Zi-Li Shiyanshi Yanjiu Yu Tansuo (2008) Study on the Method of Transforming Between Big-endian and Little-endian Based on the Experiment of Embed System. Research and Exploration in Laboratory. Vol. 27, no. 5 (May), pp. 66-67.
DAV’s Endian FAQ (nd) Accessed at http://www.rdrop.com/~cary/html/endian_faq.html
Laplante Jr., Phillip A. and Stanley Mazor, (2006) Anecdotes, IEEE Annals of the History of Computing, vol. 28, no. 2, pp. 70-76, Apr.-June.
Thelin, Johan (2009) How to be cute on all desktops with Qt. Linux Journal Issue 185 (September). Accessed at http://m.linuxjournal.com/article/10519
Verts, William T (1996) An Essay on Endian Order. Accessed at http://www.cs.umass.edu/~verts/cs32/endian.html
Disclaimer:
I actually know little indeed about computer architecture, as anyone who does would no doubt have noticed by now. That being the case, I’d appreciate it if you could let me know in the comments what howling errors you can see in this post and I’ll try to improve it. Many thanks!
Four Cultures 
Very interesting. I found your blog on bing and will bookmark it now. Keep up the nice work.