Written by Shaun Foley
How do open source and open standards fit into the larger goal of open, interoperable systems? How do principles of free software apply in a distributed, integrated world? How can we overcome increasing complexity to continue the technological progress that has so advanced our world? Shaun Foley ponders such questions, trying to think differently without falling off of giant's shoulders. He is an opinionated technologist with twelve years’ experience with aerospace, transportation, industrial, financial, and medical systems across North America and EMEA.
Open Source
I love Free Software. A world without emacs and gcc is a scary one to imagine, and I value the right as a user to run, change, and redistribute software. I first realized these rights because of a computer game... I wanted to customize the game’s messaging commands and tactical display, and I was able to do so because I had the program's source code. In the commercial realm, using open source code guarantees this availability.
For example, suppose we want to ensure that we can continue to run a program on future operating systems or processors. At one extreme we could do nothing and rely on the platform for backward compatibility. At the other extreme, we could strike a blow for software self-determinism by maintaining the program from source. Debugging, porting, supporting, and testing it won't be attractive for most people, but such is the price of absolute freedom.
Open source promotes the reusability of a program in isolation, but it does not address the ripple effects that one change will have on related components. Stable APIs and modular design help, but we must look beyond source code, beyond the syntax of function calls or data structures, to manage this complexity.
Open Standards
I love standards. While some kids wanted to play football or fly planes, I wanted to write RFCs. The idea of working on foundational infrastructure was (and is!) exciting. At a low level, standards like TCP, DNS, and BGP let us have one internet instead of a collection of incompatible networks. At a higher level, standards like POP3 and IMAP enable email to work across different service providers and user clients.
Standards act as an abstraction above source code. Instead of needing to inspect and understand the implementation of a component, I can trust (and perhaps verify) that it follows a standard interface. Using my computer game as an example, I could write my own client or run my own server because they followed a defined protocol. It wasn't an IEEE or ISO standard, but it was open enough to be useful for the community.
Source code can define a standard. It may be poorly documented, it may have version incompatibilities, it may rely on platform-specific behavior like endianness or word size, but it exists, which is far better than being given a binary and needing to deduce its behavior from a wire protocol. An implementation can also define a standard, as when a common library is used on both sides of a connection. This moves the point of standardization from the network to the API.
This works when there is a dedicated infrastructure team for a project but it can couple users to the implementation. What happens a decade later when we want to connect to that project but no one is maintaining the library? Such is the risk of custom protocols. One way to mitigate this risk is to standardize at both the network and API levels as DDS does.
Open Systems
Systems are now far more complex than a simple client-server game. An application may act as a client that needs authorization tokens or user preferences from another server, while at the same time being a server that provides processed data to other consumers. Standards ease the implementation of many application-to-application functions—think OAuth or Active Directory for user authentication, or RSS for online content updates—however, they largely ignore how to integrate data and functionality within an application.
Sometimes the problem is only syntactic, like when one protocol uses text and another uses a binary format, but the bigger problems tend to be semantic. Is your "altitude" field the same as my "height" field? A domain expert who knows both systems can answer this, but what happens ten years later when a third system uses "elevation"? Just like I don't want to go back to the program sources, I don't want to re-examine the precise meaning of all the data I'm using. This is expensive, and mistakes are dire; the Mars Climate Orbiter shows what happens when there are mix ups.
Components that document their interfaces, either individually or by following some standard, may be called ‘open’ but a collection of open modules does not yield a practically open system if it is cost-prohibitive to integrate them. Similarly, open source may make such integration possible, but not necessarily feasible. One of my favorite quotes about object-oriented design is "inherit, not to reuse, but to be reused." [1] Systems must design for component reuse and extensibility or they will atrophy.
One example is USENET, a decentralized forum with standard server-to-server message distribution and replication. Newsreaders allow access to groups, similar to email clients. Unfortunately, it is mostly dead and we have taken several steps backward by using web-based forums. Most such forums have no offline access, are single points of failure, are harder to archive, and offer extremely limited user interface customization.
There are several theories for its demise, like the lack of user authentication, limited moderation tools, and spam. None are insurmountable, but only with the integration of new capabilities like PGP or message filtering services, particularly on the client side. It's a shame, because a lot of good work has effectively been lost. I don't blame the designers. The 80's were a drastically different environment than today, there was little or no commercial support, and I'm certain I couldn't have done better.
However, we should nevertheless heed its lessons. The NNTP standard protocol and a plethora of open source implementations did not save USENET from new challenges. We can't always predict the functions required of a system in the future, but the right data architecture enables the integration of these unknowns. That is what makes a truly open system.
[1] C++ Coding Standards: 101 Rules, Guidelines, and Best Practices by Andrei Alexandrescu, Herb Sutter
Comments