Borrowed too much

Management controls in IT overlook many important aspects. To be more effective, we need to do more than just borrow control methods from other areas.

IT is a young subject, and has to draw on experience from other areas. These have made a great contribution, but in some cases they miss what is especially important for IT. This is particularly true of management controls, such as quality management and audit. Because of this, controls are often seen by the IT organisation as irrelevant and unwelcome, rather than helping them do their job.

The predominant quality control model grew up in manufacturing industry. It is based on the idea that quality can not be inspected into the product. The final product was an inevitable outcome of the manufacturing process. To improve quality, you have to manage the quality of the process.

This process view of quality has been taken into IT. Perhaps the best know example is the CMMI framework.

CMMI and similar frameworks are valuable tools but they have limitations. IT systems are not strictly repeatable; if they were why would we ever need more than one of them? IT systems are variable. They change constantly and the environment around them changes constantly. If you only manage the process, and never look at the product, you will miss this variability and change. You will miss the need for ongoing management to prevent decline.

Audit is a well-established part of accountancy. The stuff of accountancy, money, is relatively simple. But it needs very stringent controls to prevent abuse. You need to check that the accounts are correct, and that effective procedures are defined and followed.

These checks passed into IT audit. But IT is not the same as accounting. The stuff of IT is not just a number on a ledger or in a bank account. The stuff of IT, IT systems, is fantastically complicated, almost an organic entity. Even if procedures are perfectly defined and executed, the IT systems can be woefully inadequate.

We need to address this oversimplification of controls in IT. I am not suggesting that we throw away the existing controls, but mix in extra controls that are especially important for IT, such as:

• Usability
• Technology choices
• Performance and scalability
• Flexibility of design
• Documentation and test packs
• Knowledge of systems
• Longevity of systems

We sometimes include these in controls, but usually along the lines of "Has the system design documentation been filed correctly?", rather than the more direct "Is the system design good?". We need to extend IT controls so that they do not just control processes, but also control the important characteristics of IT systems.

This would make IT controls more relevant and welcome to IT organisations. It would be a way of showing the value of their work, not just beating them up because their procedures are not completely defined, followed and controlled.

This is what system governance is all about. It is an IT control method. It does not ensure procedures are defined and followed. Instead, it focuses on the characteristics of the IT. It brings a much needed balance to IT management controls, and makes controls more relevant and more valuable to the work we do in IT.

© Copyright 2007 Minimal IT Ltd. See the Minimal IT website for the original newsletter and copyright information.

The testing time bomb

Testing is an increasingly important part of IT. We face serious problems with the long-term management and support of systems because testing tools are not based on standards.

I once investigated time usage in a systems development department. 12% of time was programming and 40% was testing (the rest was analysis and support).

In more recent work, I have been using test-driven development and automated regression tests. Looking at our source code repository, 9% is documentation, 30% is functional code, 6% is test code, and 55% is test data.

These examples are typical. We produce more tests than programs, and spend longer testing than programming.

Our emphasis on testing is growing. Testing has moved on: from debugging, through demonstrating requirements, to test-driven development. Automated regression testing is becoming more important as our portfolios of systems grow and age.

There are many tools to help us test. JUnit and similar tools help unit testing. There are tools for planning tests, tracing requirements, running high-level system tests, and checking test coverage. There are session capture and replay tools. There are tools to simulate multiple users for performance and stress testing.

These tools make testing more effective and more efficient, but they create a dependency between the system under test and the testing tool. Testing is critical to ongoing support and the long-term viability of systems. Using testing tools means that systems become dependent on the upgrade path and success of the testing tool vendor (or open source project). If the testing tool fails to stay current we have to redevelop the tests, which could easily require twice the effort we put into programming.

Despite its importance, we place relatively little emphasis on the choice of testing tools. We spend much longer arguing about design approaches, application frameworks and programming languages, even though we will spend longer using, and arguably have more dependence on, testing tools.

There are three ways out of this problem.

• As an industry, standardise on a smaller number of testing tools. This has happened in some areas (such as JUnit for testing Java classes), but overall a huge variety remains.
• Create hand-built testing tools for each system and maintain the tools alongside the systems. When we do this, we miss the benefits of using products that other people have created.
• Define standards for the specification of tests and test data, and use tools that conform to these standards.

The third option interests me most. We need a standard, implementation-neutral syntax for tests to remove our dependency on specific tools. This would be a complete specification of the input data, operations, and expected outputs, not just documentation of test requirements.

This could then be used by testing tools:

• As the output from test design (or session capture).
• As the input to test execution. Test execution tools would map the standard to the data structures, functions and comparison methods of the system under test.
• As an input to tools for controlling tests, such as those that map requirements or check coverage.

Testing tools are a time bomb waiting to wreck the long-term management and support of our systems. If we can find a way to standardise, we can reduce our exposure to this risk significantly.

© Copyright 2007 Minimal IT Ltd. See the Minimal IT website for the original newsletter and copyright information.

Creating joined-up solutions

You need a simple, shared view of your IT to create joined-up solutions to IT problems.

Imagine you have just come back from holiday, and are sifting through your emails.

• Your DBA says that your version of Oracle is going out of support next year, and you must upgrade to keep supported.
• The audit of Business Objects has shown that users make unauthorised access to the underlying data.
• Your support manager has threatened to resign because documentation is so bad.
• The ERP implementation has slipped, and you need to run the legacy systems for another 12 months.
• You have a massive bill to upgrade a proprietary Unix server, which has reminded you that you are not on target for migrating to Linux.
• One of the operating divisions is having a bad year, and have asked to delay all non-critical spend on their systems until next financial year.

What do you do? Each problem demands investigation, decision and action.

Often we investigate problems separately, and then try to fit the solutions together into an overall work plan. This is a mistake. If you investigate each problem separately, your solutions will not join up. You might upgrade systems that are being decommissioned, or spend money where there is no budget. You might miss opportunities to combine changes and cut down on testing. You do not know the size of the problems, their interdependencies, the priorities, or the business case.

It is hard to create joined-up solutions. Each problem looks at your IT in a different way: by database, development tool, support group, application, hardware, or user department. There is no common language. To create joined-up solutions, you need a view of IT that everyone can understand.

To do this, start with a list of all your business applications, because these are the points at which IT is used. Do not forget that the IT department is part of the business and its applications are business applications too. Describe each application so that everybody knows what you mean.

Document the software and hardware that supports each application, and the service and support processes around it. Call the applications "systems", to show that you mean all the technology and human processes, not just the application software.

Do not worry that your system definitions do not match your technical architecture. This is a management tool, not a design tool.

The system list creates a common language to navigate different views of IT. Because the system list covers all of your IT, mapping your problems to the list shows the size of the problems, and highlights where you need to find out more. It shows you where you can combine solutions into one piece of work, or where to avoid work because systems are being decommissioned. It shows priority areas with lots of problems, and areas that can be left until later. It gives clarity and traceability to justify your decisions. It lets you create joined-up solutions that tackle the problems efficiently and effectively.

This system list approach is the basis of system governance, which extends it into an ongoing management discipline. It lets you build joined-up solutions to manage today's problems, and proactively avoid tomorrow's.

© Copyright 2007 Minimal IT Ltd. See the Minimal IT website for the original newsletter and copyright information.

A principle of management control

Management control should focus on listening to acting on the recommendations of staff, not checking that they are doing their job properly.

Clients occasionally ask me what level of detail they should go to when implementing system governance. Answering this question has helped me to clarify an important principle of management control.

Control is part of every management job, whether it is day-to-day supervision, or high-level governance. At every level, the question is the same: at what level of detail should controls be applied? If control is too detailed, you drown in information. If control is too high level, it does not provide enough information to make confident, justified decisions.

The principle we need to adopt is that control should be applied at one level above the work that you manage.

To illustrate, what controls should you apply to make sure that performance of your systems is adequate?

The answer depends on who you are.

If you are the systems administrator, you manage the systems themselves. Your controls should be based on the systems' measures of response times, CPU usage, memory, and so on. You need to take this information, interpret it, and understand where performance is an issue.

At a more senior level, you do not manage the systems directly, you manage the systems administrators. You do not need information about response times, CPU and memory. You need the judgements of the systems administrator about where there are performance problems, and why. Your job is not to repeat the work of those below you, but to listen to them, balance their recommendations with other inputs, and act accordingly.

This may seem obvious. But I think we often forget it

Sometimes we forget the purpose of management control. Using our systems administrator example, the primary purpose of higher level management is to provide a channel for addressing performance issues, not to check that the system administrator is doing their job properly. Having worked as a specialist myself, I know how frustrating it is to have managers crawl over the detail of your work, but not listen to and act on its recommendations.

Sometimes we manage at the wrong level of detail. We consider tools that attempt to bypass lower level work, such as "executive dashboards" showing summaries based on code scanning technology. This might be interesting to programmers, but it needs interpretation before it is presented higher up. Sometimes we carry out reviews based on high-level subjective opinions, without any reference to hard facts or specialist insights. Sometimes we simply repeat specialist work, re-asking the same questions that existing staff ask daily.

This has helped me to clarify the answer to the question about the level of detail for system governance. In each aspect of your IT, system governance should be applied at one level above current management activities. System governance takes the outputs of specialist management, consolidates and interprets them, and identifies and justifies what higher level actions are required. It is not a way of checking that specialists are doing their job properly; it is a way of making sure that they are heard.

© Copyright 2007 Minimal IT Ltd. See the Minimal IT website for the original newsletter and copyright information.