Universal business computer 3: Data graphs

You don't have to use tables to represent information.

In this series of newsletters I am sharing my personal views on a more standardised and versatile IT architecture. This week I want to share ideas on how information can be represented.

This may seem silly. Over the past 20 years or so, IT has standardised on relational databases and table-based information representation.

Although the relational model is very effective, it has limitations.

• Relational databases require that you define what data you want to store before you store it. This makes it hard to change what data you want to store at runtime, to reflect unknown or changing requirements. Although it is technically possible, it is not feasible to allow users to create new tables "on the fly" from inside applications.
• Although most data can fitted into tables, some structures, such as hierarchies and inheritance relationships, are difficult. There are ways of representing these as tables, but they are not easy and obvious for the novice.
• Because every table is defined differently, it is difficult to implement processing that applies to all data, such as version control or row-based permission management.

To get around these limitations, many applications implement more dynamic data structures on top of a relational database. For example, many systems use key/value pairs for user-defined properties, or present the user with ways of visualising and setting up hierarchies. Although this can add some flexibility, it is still limited by what the database designer has catered for.

Part of my vision of the universal business computer is that it is versatile and capable of meeting any business requirement. It is also an order-of-magnitude cheaper than conventional approaches. In my current thoughts, the best way of achieving this is to move away from the table as the main information representation metaphor. Instead, data is held in a directed graph, what I like to call a data graph.

In a data graph, any information can be represented as a series of objects and values (known as nodes) connected by links (known as edges) that have both direction and data type. For example, here is some information about a person:

You can use this to represent anything. You can use it to represent a table-like structure.

You can use it to represent non-table structures, like this hierarchy of departments.

This is a well-known approach. It is the basis of Resource Description Framework (RDF), which is widely used for describing data on the web.

We use similar structures within Metrici Advisor, with some additions. We have added additional data to reduce the need for small link nodes. We store and automatically recalculate derived data to reduce the traversals through the structure. We support version management, inheritance, auditing change, and permissions.

I imagine you are sceptical about the need and practicality of a new way of representing information. I certainly would be. In the next newsletter, I will share some ideas about how to add meaning to information, which is where the data graph approach is really valuable.

The next newsletter will be published on 10 January 2012.

© Copyright 2011 Minimal IT Ltd. See the Minimal IT website for the original newsletter and copyright information.

Universal business computer 2: Architecture

Systems benefit from a simple external architecture and a consistent internal architecture.

The universal business computer is a pragmatic quest for standardised IT architecture that is responsive, versatile, meets long-term management objectives, and is an order-of-magnitude simpler and cheaper than traditional approaches.

To get close to this ideal, we need to consider a number of different aspects. This week I want to share some ideas on overall architecture. In later weeks, I want to cover information representation, adding meaning to information, and user interaction.

I want to cover two aspects of architecture: what it looks like from the outside, and what it looks like on the inside.

The main requirement for architecture is that it should be easy to manage. To achieve this, each system should be self-contained. It should manage user interactions, store data, and perform its own housekeeping. There may be interactions between systems, but each system must have enough intelligence to cope when other systems are not available. Although a system may be implemented on multiple physical servers, it should be accessed through a single network address.

The outside view of the universal business computer is therefore very simple. It is a peer on a network of similar computers, with no layers and no dependencies.

Externally, all computers are self-contained, equivalent peers.

The universal business computer needs to support all types of interaction. It needs to interact with the user, typically to a web browser, and interact with other systems. It needs to support its own internal processing, connecting different components, initialization, housekeeping, and so on. Rather than thinking of these as separate types of processing, in my vision of the universal business computer, these are all carried out in the same way.

Our Metrici Advisor software provides an example. We encapsulate all processing and data access in services. These are implemented in Java and communicate with each other using XML messages. The services support all types of processing: they support a web-based user interface, provide access to components such as email, and perform initialization and housekeeping. The user interface is not service-based, but all significant processing is carried out by services called from the user interface layer.

The services are self-contained, for example performing all necessary authorisation checks. This means that anything a user can do can be automated. Services can be called from the UI layer, or from another system using a web service, or by pasting XML directly into a UI utility. Internally, services can be called by writing XML files into specific directories, or specified in startup files, or run from the command line, or run from Java programs, or embedded in user-defined scripts. Standardising all internal integration around services provides a common mechanism for building the application, which works for all types of processing.

Internally, common services encapsulate access to data and other components.

To summarise, the key points of my vision of the architecture of the universal business computer are:

• Externally, each computer is a self-contained peer on the network.
• Internally, all processing is implemented in services which can be called from the user interface and other clients.

Next week I will share some ideas on how information can be represented in a universal business computer.

© Copyright 2011 Minimal IT Ltd. See the Minimal IT website for the original newsletter and copyright information.

Universal business computer 1: The quest

The universal business computer is a quest for a better way to do IT.

Most of us have special interests in IT. Some follow the trends of ever-faster hardware. Others specialise in databases, or mobile devices, or graphics. Some are dedicated to service management. And so on.

One area has fascinated me for at least a couple of decades. I am not sure that there is a proper name for it, but I describe it as the quest for the universal business computer.

By my definition, the universal business computer is a standardised IT architecture which fully meets all non-functional requirements and which can deliver any business requirement. It is not a theoretical notion, but a practical quest, searching for and designing ways to get closer to this ideal. It is a "super design" that can be applied to meet many different needs. I want to know how close we can get to this ideal, rather than constantly re-designing solutions.

I can trace this interest back to different parts of my career. I have spent a lot of my time providing technical support for developers, setting up development environments, creating common code, managing standards, and so on. It struck me that most business requirements can be met by a standard set of technical features. There is a lot of unnecessary technical variation in solutions.

I have been influenced by working as an enterprise architect. Different people have different views of enterprise architecture. My view is that enterprise architecture is an extension to standards, setting down in practical terms how IT needs to be arranged in order to meet long-term business and IT management objectives. The specifics of business processes and applications are driven by short-term commercial, financial and political pressures. To me, the role of enterprise architecture is to create an environment in which IT can easily and safely respond to these transient and often conflicting requirements, rather than pretending that IT can somehow influence them or rationalise them away.

Another input is the interest I had thirty years ago in "home computing". What I noticed when I started work as a professional developer is that although professional development has many more resources, it is less responsive and less versatile. Once you have layered technologies, multiple specialists, and development disciplines, you lose the ability to just do what you want with the computer. (Though this is to an extent reversed by agile methods.)

My interest in minimal IT has also been an influence. Over the past few years, I have been looking at how structure and appropriate management can make order-of-magnitude improvements in simplicity and cost.

All of these come together in the universal business computer. It is a pragmatic quest for a standard IT architecture that is responsive, versatile, meets long-term management objectives, and is an order of magnitude simpler and cheaper than traditional approaches.

There are many paths on this quest. Over the next few weeks I want to lead you down the path I have taken, and share with you an architecture which comes tantalisingly close to this vision of the universal business computer.

© Copyright 2011 Minimal IT Ltd. See the Minimal IT website for the original newsletter and copyright information.