Wayne Strelioff, CA
Auditor General of British Columbia

Management of the Information Technology Portfolio
in the Ministry of Attorney General

Notes for a presentation to Information Systems Auditing and Control Association

March 20, 2002

Good afternoon—thank you for the warm introduction and this opportunity to speak to you about our recent audit report on the "Management of the Information Technology Portfolio."

I know your association contributes significantly to the capacity of the public sector to manage public programs and services as effectively as possible. I thank you and encourage you to continue to do so.

I also know that lunches such as these provide all of us the valuable opportunity to discuss issues and build working relationships.

Before I deal with our report, I want to tell you a little about the Office of the Auditor General and me.

As was said, I was appointed Auditor General of British Columbia a little less than 2 years ago. Prior to that, I was the Provincial Auditor of Saskatchewan for nearly ten years. I am a chartered accountant.

As Auditor General, I am an officer of the Legislative Assembly. My reports—which are public—are automatically referred to the Public Accounts Committee.

This eleven person Committee of legislators is chaired by a member of the opposition and has the responsibility to deal with my reports in meetings open to the public.

In general, the role of an Auditor General is to help all legislators and, thus, all citizens, hold the government accountable for its use of public resources and for its management of the considerable responsibilities entrusted to government.

Rigorous public scrutiny is particularly important in this province right now. Significant change to what government does is taking place at a time of difficult financial pressures and economic uncertainty, and at a time of unusual political party representation within the Legislative Assembly. During the next few years, legislators will find it especially challenging to earn and hold public confidence.

We have a relatively small Office of about 90 people with a wide range of professional credentials and experience including a growing number of people who have or are working towards their CISA credentials.

We have access to all parts of government, to all types of information, and we decide what to examine and when, and what to report. And, as you no doubt know—our reports are public.

One of my priorities as explained in our public service plan is to examine governance, accountability and risk management for large capital projects and complex mission-critical information systems. We have done work on this in the past and we continue to build our capacity to do so in the future.

In general, we have four lines of business:

• First, we examine financial statements to ensure they present reliable measures of such key items as surplus or deficit.
  
  
  • In February, we reported on the finances of government focusing on the last five years to March 31, 2001.
    
    
• Second, we review the quality of performance information published by the government
  
  
  • Also in February, we reported on the quality of government performance information—much is changing on that front—good changes—for example, for the first time, the performance plans or service plans of many government agencies were tabled at the same time as the government’s budget on February 19^th.
    
    
  • I look forward to the day when all provincial public sector organizations publish quality service plans in a timely manner—particularly those organizations delivering health and education services.
    
    
• Third, we help legislators make use of performance information in their scrutiny of government
  
  
• Our fourth line of business is to assess how well government manages the key risks it faces in achieving its planned results. Our recent risk work focused on subject matters such as:
  
  
  • The extent to which we are prepared for earthquakes,
    
    
  • The extent to which we provide children a safe learning environment,
    
    
  • The governance and financial arrangements in place at TransLink,
    
    
  • The extent to which we monitor the financial integrity of credit unions and trust companies,
    
    
  • The extent to which we safeguard our drinking water sources.
    
    
• The extent to which we are prepared for major interface fires
  

And, of course, the subject of this luncheon—management of the information technology portfolio

In the next month, we will report on the extent to which the ministry of health uses appropriate information to support its resource allocation decisions, and the extent to which the government’s work environment is congruent with the delivery of quality service to the public.

As you know far better than me, information technology offers government unprecedented opportunity to expand services, improve performance and reduce costs.

Yet, many IT projects are never completed on time or on budget, and many end up unable to do all they were intended.

Managing IT well is currently one of the hardest jobs facing both public and private sector managers.

Yet, IT has become critical to how well and how cost-efficiently our public agencies deliver the services we all depend on as citizens.

Improving the way IT systems are developed and managed means improving the public good.

IT projects represent a significant and growing part of the capital investment of organizations – for example, the Workers Compensation Board has invested about $340 million in capital assets, of that, $140 million relates to its operating systems

IT projects seem even more difficult to manage than bricks and mortar projects

You all know the dismal statistics assembled by the Gartner Group on private sector IT projects--government IT projects may well face even more obstacles.

• First, there is no single agreed-on ‘bottom line’ for government so that success is difficult to determine and measure performance against
  
  
• Second, much of what government does is in the context of conflicting priorities, for example
  
  
  • We often ask public sector managers to acquire goods at the least cost and at the same achieve regional development or equity targets
    
    
  • We ask managers to ensure all Canadians have ready access to extensive ranges of information and at the same time ensure near absolute security or privacy of information
    
    
  • We ask managers to carry out complex multi-year projects—with one year funding certainty
    
    
  • We ask managers to get the job done quickly and, at the same time, ensure a tendering process that is fair, open and unassailable
    
    
  • We ask managers to contract out all systems development and at the same time ensure such contracts are managed with experienced project managers
    
    
  • We are often asked to manage projects involving very experienced, expensive and shrewd private sector contractors—with a shrinking pool of experienced public sector project managers
    
    
  • And on, and on …
    
    

  So, in this challenging environment—How can public sector managers improve the odds of success when making IT investments?

  From our recent work, we concluded that current best practice is an approach called portfolio management.

  Our project team included Bill Gilhooly, Ken Lane, Endre Dolhai and Peter Gregory. We also had an experienced external advisor —Ian Chisholm. We use external advisors for almost all of our large risk management audits.

  Under the portfolio management approach, an organization’s IT resources—its computers, computer systems, and telecommunications equipment—are managed as one would manage other investments such as real estate or stocks.

  Everything related to deriving benefit from IT expenditures is included—that is,

  • every system, proposed, under development, and in use
    
    
  • everything needed to get value from these systems, from training to complete business process re-engineering—which recognizes a key part of success is recognizing and managing the human dimension
    
    

  Each proposed investment is examined in the context of the organization’s current and planned investments as well as in the context of the organization’s business needs.

  When used, the portfolio approach helps senior management take a ‘big picture’ perspective on its IT spending—they can look for total strategic advantage, rather than individual hits and misses

  From what we found out, the portfolio approach can contribute to organization coherence:

  • The IT business of organization is made more understandable as a complete package
    
    
  • It is more likely organizations can get better consistency and synergy
    
    
  • The portfolio approach can help organizations systematically evaluate and manage its IT risk profile
    
    

  The portfolio approach calls for regular scrutiny of all IT assets—accordingly, organizations can be more responsive to changing conditions

  The portfolio approach sees IT as an investment, not an expense—focuses on maximizing payoffs that align with organization’s strategic goals

  Portfolio management is recommended or adopted by many organizations including

  • Treasury Board of Canada
    
    
  • Auditor General of Canada
    
    
  • US Office of Management and Budget
    
    
  • US General Accounting Office
    
    
  • state of Washington [state leader in IT]
    
    
  • consultants like Gartner Group and MEGA
    
    

  There are three key elements to successful portfolio management:

  • clear governance
    
    
  • well-thought-out proposals for change
    
    
  • well-managed project delivery
    
    

  Of course, all these elements are easier to refer to than to bring to life.

  Governance requires a senior decision-making body able to

  • oversee all material IT investments in the organization,
    
    
  • adjust the mix to optimize overall benefits, and
    
    
  • oversee successful delivery of projects
    
    

  Well-thought-out proposals for change are not easy, given the rapid pace of change in IT industry, and limited government resources

  Three parts to choosing best proposals

  • know what’s in present portfolio
    
    
  • know what’s new, what’s possible, in IT
    
    
  • assess possibilities, choose the most worthwhile
    
    

  Choosing the best portfolios has three stages:

  • gathering information on benefits
    
    
  • gathering information on costs
    
    
  • applying analytical tools to find best net value
    
    

  When gathering information on benefits, the proposals:

  • must align with organization’s strategic goals
    
    
  • include costs savings and non-financial benefits
    
    
  • must demonstrate logical links from system capabilities to improved organization or decision making
    
    

  Examples of logical links to better decision making are

  • will the system collect information relevant to executive decisions?
    
    
  • is senior management committed to using that information to shape its decisions?
    
    
  • does senior management have latitude to make decisions based on information, or do other considerations--policy, say, or regional equity--override?
    
    

  Determining costs: more straightforward, but must

  • include financial and non-financial factors (e.g. will project monopolize organization’s scarce supply of project managers?)
    
    
  • reflect entire life cycle (e.g. software upgrades)
    
    
  • recognize time factor (i.e., financial costs should be discounted)
    
    

  Choosing the best proposals

  • think portfolio—look for total best value; consider conflicts, synergies
    
    
  • think strategic—are the promised payoffs tackling the strategic issues?
    
    
  • use objective evaluation methods (e.g. Balanced Scorecard, Information Economics), and use them consistently
    
    
• The third key to successful portfolio management—project delivery
  
  
  • no surprises here: calls for good, classic project management
    
    
  • the gospel according to the Project Management Institute—scope, schedule, budget
    
    
    
  • but, with lots of attention to risk management
    
    

  Risk management

  • isn’t risk avoidance—in IT, big payoffs often entail big risks.
    
    
  • is a kind of lens, focused on obstacles to success

Key risks relate to:

• The degree of change and the number of separate organizations or divisions involved in the proposal
  
  
• The complexity of the system integration required
  
  
• The extent to which proven technology is to be used
  
  
• The capability experience of the organization in successfully carrying out or managing a given level of system development work

Pulling it all together – management maturity

In our report, we also discuss the notion of portfolio management maturity (also known as portfolio management capability) as an integrated way of judging progress in portfolio management.

The core idea is—organizational improvement is not a matter of individual hits and misses, but comes through advances in the overall capability of the organization.

Concept was first elaborated for software development (Carnegie Mellon University for US Department of Defence)

Later extended to other areas of IT, to financial management, to project management

Two years ago, US General Accounting Office developed good maturity model for portfolio management

Model sees organizations advancing from level one –‘ad hoc, chaotic"—to level five—‘mature, disciplined.’

Level 1 – occasional successes are more by good luck than good management

Level 2 — success is planned, efforts focus on strategic outcomes that keep organization far ahead of peers

each increasing level requires investment in e.g. training, staff selection, managerial attention

as a basis of discussion, we suggestion most ministries would get payoffs from reaching level three; beyond that, need a business case

Level three means, e.g.

• well-developed, consistently-used, criteria for selecting new investments
  
  
• projects habitually on time, within budget

Our purpose in introducing this maturity model is that it provides a useful tool for self-rating or self-assessment

So what did we find when we applied these concepts of portfolio management to Ministry of Attorney General?

Why this ministry?

• Carries out important function (administering justice in BC)
  
  
• Its success dependent on IT (140 systems; we focussed on 13 major ones)
  
  
• Had in place several key IT organizational tools—meant that our findings could be useful to other organizations

A reminder: we looked for three key things

• clear governance
  
  
• well-thought-out proposals for change
  
  
• well-managed project delivery

On governance, the ministry did well: it has an appropriate senior decision-making body made up of senior executives from every major branch reporting directly to executive committee with oversight over all significant IT projects in ministry.

On proposals for change—the Ministry:

• had reasonable methods for finding new technological opportunities
  
  
• had reasonable amount of information on current portfolio, but needed organizing
  
  
• we recommended improvements in way benefits were described, and in way proposals were evaluated

On project delivery, we recommended levelling up: kind of project management seen in successful projects like Y2K should be applied to all projects.

In particular, we recommended:

• better risk analysis of proposals
  
  
• better risk management during delivery

Overall, we found that the ministry had a good foundation, particularly in its governance structure, for advancing in portfolio management maturity

I find the work we set out in this report to be a little different than most of our other reports. Although our assurances and advice are relevant to the responsibilities of legislators, I think this report is more focused on helping government managers deal with complexity.

If we view information technology as an investment portfolio instead of as single stand alone projects, we should be more likely to ensure such investments are made and managed in the context of their contribution to the success of our organizations.

If we also surround such investments with strong project management principles, we are also more likely to deliver the initial promises we make when projects are proposed, developed and put in place.

I hope you consider the contents of our report in that context.

I know I am.

One of the benefits of my job that I value dearly is the opportunity to learn from the work we perform, and apply that learning to our own organization.

In this audit, I am provided a sound framework for taking an portfolio management approach to managing our portfolio of audits.

Much of the thinking behind IT portfolio management, project management principles and organizational maturity applies to the work of my Office.

What we do in our Office is project oriented, we must ensure the audit projects we consider, initiate and manage align with achieving our goals and objectives in the most effective manner possible.

I ask you to do the same to your work. Read this report and consider whether our thinking might help you to manage your IT portfolio, or perhaps, another portfolio of projects relevant to the success of your organizations.

Now, if you have any questions I or my colleague Bill Gilhooly will do our best to provide you with answers now or at a later time.

Top of page