Over the past few years, the role of the business in technology projects has been the topic of many discussions within the CIO community, with much of the focus centred on how IT departments can better work with the overall business. And there is a very good reason for this concern.

That’s because a survey conducted by IDC of CIOs from across the world found that, on average, around 45 per cent of technology spending undertaken by organisations is now drawn from budgets outside the IT department. This typically represents a mix of business-funded technology projects where the relevant line of business works hand in hand with the IT department (28.5 per cent) and those that the IT department is aware of but plays no role in (16.8 per cent). However, there is also a third type that is much more difficult to quantify and potentially much more disruptive to the work of the CIO — shadow IT.

The issue of shadow IT is a growing one in the technology industry, although the term is often misused due to confusion around exactly what it means. Simply put, shadow IT is spending on technology that is unknown to the IT department. This differs from the ‘sanctioned’ expenditure outlined above because these projects are not funded by the IT budget and, crucially, the IT department is not even aware of their existence, let alone involved in implementing or supporting them.

There is a general belief that sanctioned IT spending is increasing, with the business becoming a larger technology buying centre of technology. And there is little doubt that shadow IT spending by the business is also growing. But what are the major drivers of this growth in shadow IT spending, and what are the key problems it is causing?

My discussions with CIOs both here in the Middle East and elsewhere around the world consistently throw up two theories behind the surge in shadow IT: Firstly, IT suppliers are increasingly bypassing the IT department and directly targeting business users with their products and services; secondly, business owners are demanding that IT projects are implemented at a much faster rate than their own internal IT departments can possibly deliver.

The biggest problem shadow IT presents to the organisation is increased risk of corporate information being compromised, as the data sets sit outside of corporate security systems. Shadow IT also creates information silos; as lines of business procure their own technologies and do not integrate them into the enterprise systems, the information becomes an island unto itself.

It is also more difficult to fully demonstrate the value of the technology if it cannot be integrated with other systems. Social software, for example, may sit solely within the marketing department with the express purpose of collecting social information from consumers to improve marketing campaigns. But that social software and the insights it generates would be far more valuable to the organisation as a whole if it was also connected to customer service, product development, and sales processes.

Another major issue is that the majority of IT professionals believe the mere presence of shadow IT creates unnecessary redundancies, in turn causing excess costs and reducing the organisation’s ability to effectively negotiate with IT suppliers. It is for these reasons that I do not expect shadow IT spending to continue growing indefinitely. But the upshot of that will be no less painful for the CIO.

At some point, organisations will recognise the untapped potential of their collective technology investments and realise the excess costs they have incurred. Unfortunately, at that point, it will be the IT department’s job to put it all back together again. As such, I would urge all IT professionals to get ahead of the game.

To this end, I recommend two paths for minimising shadow IT. The first path is one that tries to address the root cause of why shadow IT developed in the first place. This requires IT executives to change the culture of their departments to make them more effective partners across the wider organisation. Going hand in hand with this is the need to identify the shadow IT projects that already exist.

This shouldn’t be done with the intention of shutting them down, as this will only cause resentment; instead the focus should be on partnering with the relevant departments on these projects to make them more inclusive. I also encourage IT executives to change the way they develop projects with lines of business in order to produce a faster turnaround time, with a particular focus on introducing more small and agile projects.

The second path relates to organisation-wide policy. For example, many IT heads are now working closely with their finance departments to track down shadow IT projects and officially bring them back into line. And another tactic that is increasingly being employed in the battle against shadow IT is for organisations to issue mandates from senior management that no cloud purchases (or purchases of any other specific technology) can be made without the IT department’s direct involvement.

With the various lines of business now funding close to half of all technology projects in an organisation, it is critical that CIOs figure out a way for their departments to become more involved in these projects and be viewed as a valued business partner. And with businesses increasingly looking for rapid turnaround times, CIOs must work hard to change the overall IT culture, build relationships across the business, and increase their project velocity. Until this is achieved, they should remain very wary of what is lurking in the shadows.

The columnist is group vice president and regional managing director for the Middle East, Africa, and Turkey at global ICT market intelligence and advisory firm International Data Corporation (IDC).