Buy or build? Neither!

Web-based development and deployment could get around the perennial buy-or-build argument.

I was discussing a client's situation with a colleague. The client has a lot of bespoke systems, and my colleague was saying how they would be better off moving to packaged systems.

I was going to argue back, about how bespoke systems can offer better value, when I realised I had had this discussion so many times before. There is a general bias against bespoke systems, but some people (like me) constantly argue in favour of them. Why?

A lot of it stems from career paths and previous experience. Some of us start out as programmers, others in other roles. Some developers move on, but some of us (me included) are still actively developing decades later.

Those of us who are steeped in development are not scared of bespoke systems. We understand how they work, and how to get them to do our business' business. I have seen many ill-fitting but expensive packaged solutions, which seem such a poor substitute for well-tailored, good-value bespoke systems. Unlike a package vendor, the developers have the best interests of the business at heart.

Those not intimately involved in development see it differently. Bespoke systems are expensive and take time. You have to work with awkward and uncommunicative developers (like me), who don't understand the business and just confuse you with technical mumbo-jumbo. There is little control over the process, and nobody takes proper responsibility. Unlike developers, the vendor understands your business.

There is some truth in both positions.

When you first develop a system, it can meet requirements well, and, provided it is not over-ambitious, it can be cheaper than implementing a large package system. In this sense, the developer mindset is reasonable.

But through time, bespoke systems turn to mush. Layer upon layer of modification plays havoc with structure. It is impossible to justify reworking the system or upgrading the technology. The system is neither cheap nor responsive. The real advantage of packaged systems is that this is someone else's problem.

In IT, we have been having this buy-or-build argument for years. But now there is a way round it.

It is becoming increasingly viable to develop and deploy using web-based platforms, like SalesForce.com, or even our Metrici Advisor service. There are three advantages to this.

First, the service will provide the vast majority of the system for you - data management, user interface, user management, and so on. You only need to develop the parts that are specific to your requirements, which is only a fraction of the code required for a bespoke system. As well as speeding development, this hugely reduces the tendency of the system to turn to mush.

Secondly, the provider of the environment will take care of all the infrastructure for you - from how backups work, to making sure that the system continues to run on current technology.

Thirdly, if you want a packaged solution, you may well be able to subscribe to one delivered through the same platform.

This approach can give the best of both worlds. The fit and value of a simple bespoke development, with the management and control of a packaged solution. Could this become the norm?

© Copyright 2012 Minimal IT Ltd. See the Minimal IT website for the original newsletter and copyright information.

Who are your competitors?

You will not succeed until you know who your competitors are.

We starting developing tools and materials for assessing IT systems because we saw value in it and did not see other people doing it. That might sound like a great strategy, but there is a problem.

If you can not find other people doing what you are trying to do, then either there is no market for it, or there is a market but you are ignorant of it. Either way, you are going to have an uphill battle to sell your products. You need to find your competitors to succeed.

We found that there was no existing market for our assessments. As one prospect put it, "It's not our priority right now." People could understand the value proposition, but did not feel compelled to act. We tried to make the tool general purpose, but it was hard to apply to areas other than assessing IT systems.

Since then, we have moved on. We are better at combining our IT system assessments with services that people do recognise, such as information security or service management. We redeveloped our software so that it can cope with any type of survey, assessment, analysis or consultancy engagement.

This gives us lots of potential competitors. We can apply our ideas to any IT-related service or consultancy. The tool overlaps with a great variety of tools: survey tools; governance, risk and control tools, electronic forms, content management systems, e-learning and assessment software, and so on. But this gives us another problem. Are we wandering into the territory of established products against which we can not hope to compete?

The more I think about it, the less I worry.

Our core competence is combining IT system assessments with other types of IT work. Although there are competitors for some standard assessments, there is room in the market for more specialist or customised services.

Our tool is very strong. We can deliver solutions in a fraction of the time of a traditional application development tool, and we can deliver professional, multiuser, online systems that tools like Excel can not deliver. There is a niche for us just as tool provider.

The overlap with other tools is a by-product of our versatility. You can't use your staff appraisal system to write and host a website; the fact that our system can do both doesn't make it a me-too product.

Finally, the businesses I worry about are not our competitors. They are subject matter experts with market presence. They could just as well be customers, basing their specialist tools and services on our Advisor platform.

We have to keep our nerve. We are winning business and are very busy. We should use this success to consolidate our core proposition and invest in our software.

Our competitors are not other software companies with specialist assessment tools. They help us understand where the market is, and may become our customers. Our competitors are labour-intensive manual assessments and analyses, and under-designed solutions in Excel, which is a vast market over which we have compelling competitive advantages. Now that we know our competitors, we can succeed.

© Copyright 2012 Minimal IT Ltd. See the Minimal IT website for the original newsletter and copyright information.

A change of heart and a bright idea

Is there demand for a standard library for server-side JavaScript?

Many applications embed programming languages to let "power users" add extra functionality.

In version 1 of Metrici Advisor, we embedded the XML processing language XSLT for user-defined analysis and reporting. This made a lot of sense. XSLT works well in our XML-based architecture. XSLT is very coherent, simply transforming an XML input to an XML output. It is non-procedural, which means you can do a lot with a small amount of code.

XSLT does have drawbacks. Its main problem is its weirdness. It is very different from other languages, and relatively hard to learn. We therefore had a change of heart, and for our current version of Metrici Advisor we decided to go for simplicity and familiarity rather than architectural fit and purity. Instead of using XSLT, we embedded JavaScript, using the Rhino JavaScript engine. (This is JavaScript running on the server, not the more familiar use of JavaScript running on a browser.)

JavaScript works well as an embedded language. It is easy to write simple scripts, but also supports more advanced programming. We use it for general scripting, validation, calculating derived data, creating new UI features, analysis, reporting, data triggers, and creating reusable services.

JavaScript does have drawbacks. Because it is a very permissive language it is easy to write unstructured code. There is a simple API into Metrici Advisor's data structures, but navigating the data requires a good knowledge of the system's internals. We have created analysis and reporting libraries using JavaScript, but it is hard to keep use of these completely consistent. JavaScript can easily get a bit messy.

This had been in the back of my mind for some months, then last week I had one of those lightbulb-over-the-head moments.

For browser-based scripting, which is more complicated, various JavaScript libraries have been created to simplify programming, of which jQuery is the best known. In jQuery, the difficult parts of programming have been simplified around a small number of concepts. The core concept is selecting a set of elements from the HTML document, which can then be used as the basis for further processing. jQuery has a well-defined plug-in architecture, which allows it to be extended to cover all manner of high-level features.

The bright idea, obvious in hindsight, is to create the equivalent for our server-side use of JavaScript. We might call this "nQuery", where the "n" stands for "node", which is our basic data unit. Using similar semantics to jQuery, nQuery could be used to select and process server-side data. We could deliver analysis and reporting components as nQuery plug-ins. nQuery would make the environment more familiar for programmers experienced in browser-based JavaScript. nQuery would bring coherence and power to our JavaScript, without the weirdness of XSLT.

Creating "nQuery" also opens up an intriguing possibility.

jQuery has become a de facto standard for browser-based programming. Something like nQuery, mapped to a generic data API, could be used much more broadly than as a library for embedded JavaScript in Metrici Advisor. Is there demand for a standard library for server-side JavaScript?

© Copyright 2012 Minimal IT Ltd. See the Minimal IT website for the original newsletter and copyright information.

Technical analysis

The technical content of a piece of work can hide a more complicated requirement that needs significant analysis.

We all understand files on the web. Web-based email allows you to attach files to messages. You can upload photos in Facebook. You can download software from SourceForge. Although it might require some technical programming to implement, we all know what we mean by file uploads, attachments and downloads.

If only it were that simple. Our experience of implementing file support in Metrici Advisor is a good example of how a supposedly technical requirement can hide a much more complicated piece of analysis.

First, we needed to understand what files were to be used for. Are the files going to be served as part of the website, for example as images and downloadable documents? Or do we just need a web-based file system for uploads and downloads?

We want files for three things: for web content, such as images; to present additional reference material, such as PDFs; and to allow users to attach files when performing assessments.

Using files for content means we might need to modify the files to make sure that they are usable, for example downsizing images so they fit on the page, or filtering uploaded web pages that might contain JavaScript. If you present uploaded files as part of your website, you can not provide byte-for-byte equality between what is uploaded and what is presented.

Filtering web content is technically easy (we use AntiSamy), but understanding what to filter is harder. As well as filtering HTML, we need to identify and cope with other files types, like XML, that can not be filtered but which still might be processed by the browser.

We needed to think how file storage is presented to the user. Is there a file upload area, or are files attached to other content?

In our design, each file is represented by an independent object, called a node. The nodes can be anywhere in the website structure, or grouped into upload areas.

We needed to decide what information to hold with each file. In a simple system, a file is just a file. But representing the file as a node lets us give it a title, a description, and other information.

We needed to design file storage. In a simple system, you might just store files in a directory on the web server, matching their file location with their web address. But if you have large number of files, you need to manage them differently, possibly using external storage such as Amazon S3, and then map between their storage locations and their web address. Our solution maps files on the server to web addresses, in a way that will allow us to use external storage in the future.

None of these aspects is particularly hard to program, and there are well-established methods for each part of the implementation. What makes file uploads and attachments hard, like so much technical programming, is analysing the requirements and designing a solution that meets them.

We sometimes split work (and people) into "analysis" and "technical". But we should remember that technical work can involve a great deal of analysis too.

© Copyright 2012 Minimal IT Ltd. See the Minimal IT website for the original newsletter and copyright information.