Ensure Returns on Investments in Information Technology
Since 1998, Texas has spent an average of $1.4 billion annually on information technology (IT) and telecommunications. Among 21 large IT projects currently under development at state agencies, the average project is over budget and overdue. Such overruns are an indication of serious problems in agency IT planning. To remedy this, the state should implement a return on investment program for IT projects. Funds for IT projects should be pooled and awarded via a competitive process. Such a program would promote interagency cooperation, efficiency and effectiveness; it also would assist agencies in coordinating and consolidating their IT purchasing contracts.
Texas state agencies and institutions of higher education spend more than $450 million every year on information technology (IT) hardware, software and technical support. This amount does not include staffing costs or purchases made with local or federal funds. The State Auditor’s Office (SAO) estimates that total IT spending, including staff costs, will average $1.4 billion over the period of fiscal 1998 through 2003. SAO also estimates that, among the 21 largest IT projects it was monitoring as of October 2002, the average project was $388,000 over budget and 21 months behind schedule. Not included in these averages is the Texas Integrated Eligibility Redesign System (TIERS) project at the Department of Human Services, which is $349 million over budget and will be an estimated 37 months overdue when complete.
In calendar 2001, the projected cost of 44 major IT projects underway at state agencies, including TIERS, exceeded the initial cost estimates by $361 million. Five cancelled projects alone cost more than $28 million. Such losses and overruns, even though paid in part by federal or local tax dollars, equate to fully 86 percent of the state’s $450 million in annual IT spending from general revenue and dedicated state funds. Such costs are an indication of serious problems in Texas’ IT planning.
“Return on investment” (ROI) is a standard business measure of the outcome of a project against its cost. Quantifiable ROIs also can provide valuable information for the success of future projects. Today’s economic climate has prompted a number of states and private companies to create ROI programs to ensure that scarce resources spent on IT deliver the value they promise.
Iowa ROI program
In 2000, Iowa founded a Return on Investment program within its Information Technology Department’s (ITD’s) Enterprise Quality Assurance Office to “quantify, measure, evaluate, and verify technology ‘investment’ benefits to both State government and to Iowa citizens.” The governor wanted the program to evaluate proposed IT projects and expenditures to ensure that they deliver quantifiable returns.
Iowa’s ROI program has been so successful at saving tax dollars for IT projects that it won an award from the National Association of State Chief Information Officers in October 2002 for being the best state IT initiative.
The Iowa program requires an ROI analysis for:
- all non-routine IT expenditures,
- IT expenditures over $100,000, or
- any request for monies from the Pooled Technology Fund, which was established to “support special cost savings or innovative IT projects and/or reengineering using information technology.”
After one year, the state reported savings of $2.4 million from combining projects and $3.2 million from reducing expenses, for a total savings of almost 30 percent of its $19.1 million Pooled Technology Fund. In the second year, Iowa realized $1 million in savings from combined projects and $1.1 million in reduced expenses, a 14 percent savings out of a $15.5 million Pooled Technology Fund.
The ROI program began with meetings among government and private IT executives. This group developed methods for evaluating IT projects and expenditures. Iowa pooled most of the dollars previously appropriated to individual agencies’ IT budgets and then required them to submit competitive applications to the Enterprise Quality Assurance Office for projects costing more than $100,000 annually. Agencies receive reimbursements from the Pooled Technology Fund for the amount approved based on their applications; any cost overruns must be covered by the agencies’ existing funds.
Applications to the Iowa ROI program consist of five sections: a proposal description, a project administration plan, a description of the technology, a financial analysis and evaluation criteria. The criteria include:
- statutory or other requirements mandating the proposed IT project or expenditure (worth 15 points maximum);
- customer service improvements (15-point maximum);
- impact on Iowa’s citizens (10-point maximum);
- tangible and/or intangible benefits (10-point maximum);
- use of IT to reengineer government processes (10-point maximum);
- risks of potential failure, including cost overruns and other concerns (10 point-maximum);
- potential for continued funding requirements (10-point maximum);
- collaboration with other state agencies (10-point maximum); and
- past performance on other IT projects (5-point maximum).
Of all these criteria, only one—continued funding potential—could be considered a “hard dollar” benefit. The others are designed to promote interagency cooperation and make existing agency processes and procedures more efficient. These efficiencies may not provide an immediate dollar savings, but they accrue benefits over time by reducing the need for expensive contract IT personnel or rewriting software.
Before receiving reimbursements from the pooled technology fund, agencies with qualifying projects or expenditures must submit an application to ITD specifying the project’s funding source, detail about the hardware, software and networking used in the project, their compatibility with existing systems, and an evaluation of potential future costs for state government. If approved by ITD, the state agencies themselves manage the IT projects, while ITD prepares project status reports and performs final project outcome audits.
In an interview with e-Texas, Iowa’s ITD manager Paul Carlson said that state agencies initially resisted the concept, believing that their autonomy to create, design and deploy technology for their specific needs would be compromised.
Once in place, however, the same agencies found that they benefited from the competitive model. The process encouraged them to design projects that could be used by other agencies and shifted the burden of implementation to vendors who then had to deliver promised results on time and on budget. After the first year, Carlson found that agencies dropped some projects simply because the ROI process existed, perhaps an indication that the proposed projects were not worth the effort.
Carlson identified three key elements to Iowa’s ROI success:
- Fairness. Agencies appreciate the ‘level playing field’ of standard criteria for measuring the need for and ultimate success of each major project.
- Assistance. By helping agencies develop their applications, ITD gains their trust while guiding them through the evaluation process.
- Executive leadership. The board overseeing the program is a mix of gubernatorially appointed state agency heads and business persons and representatives of ITD’s customer agencies.
Carlson says that the guaranteed outcome audit in the ROI program means that “you get what you inspect, not what you expect.”
The Iowa ROI model is being applied in various forms in Missouri and Kentucky, Canada and several U.S. county governments and universities. According to Iowa’s Carlson, several governments have contracted with an IT strategic advisor to develop an ROI program only to be told to follow Iowa’s example.
The Texas record
For some time, the Legislature has recognized the need to control IT spending by state agencies and institutions of higher education. The 1993 Legislature created the Quality Assurance Team (QAT), with representatives from the Legislative Budget Board (LBB) and State Auditor’s Office (SAO), to establish rules and guidelines for reviewing major information resources projects.
The QAT monitors all IT projects by state agencies and institutions of higher education with development costs of more than $1 million. The projects also must either “require a year or more to reach operational status, involve more than one agency or governmental unit, or materially alter the work methods of agency personnel or the delivery of services to agency clients.”
In a January, 2002 memo to the governor and legislative leaders, the QAT said:
As of December 2001, there were 200 projects subject to quality assurance review with 42 of these projects actively monitored. Of the monitored projects...significant changes in project costs range from an increase of $17.1 million to a decrease of $21.4 million compared to their initial planned costs. These project cost changes represent a cost shift of up to 160 percent, the variability of which impairs planning and service delivery at a statewide level. Five projects expended $28,270,526 prior to cancellation, though some benefits were claimed... Given that only 34 percent of agencies and universities with an application development project have substantially adopted quality assurance guidelines as directed by the Department of Information Resources, these trends will likely continue.
The QAT also found:
- agencies are duplicating the cost to enhance and maintain PeopleSoft™ [a brand of software chiefly used to assist with human resources, budget and other functions] by individually hosting the software and dedicating technical staff to its support;
- enforcement of performance bonds for software development projects has not been successful, resulting in an agency unable to recover the $1.15 million invested in its cancelled project; and
- many projects cross biennia and often are extended every two years when new functionality is added, thereby making project cost and schedule information difficult to evaluate.
The QAT also measures project risks, based on its internal analyses, and estimated project costs. The total lifecycle costs for all projects monitored by QAT exceeded $2 billion as of December 2001. (“Lifecycle” means all costs associated with the project’s inception through its development and implementation.) One project can also be a composite of several smaller, discrete projects designed to accomplish a common objective. Exhibit 1 ranks the number of projects under review by QAT as of December 2001 by risk level and lifecycle costs. QAT determines risk based on its own assessment of the project’s likelihood of cost overrun or failure, among other factors. Projects with a low risk level are waived from further QAT review. Projects with medium risk warrant more reporting to QAT, and projects deemed high risk receive constant QAT supervision.
IT Projects Under QAT Review, December 2001
By Risk and Lifecycle Costs
Source: Quality Assurance Team, Legislative Budget Board and State Auditor’s Office..
Number of Projects Total Project Lifecycle Costs Low Risk 109 $ 959,000,000 Medium Risk 52 $ 337,000,000 High Risk 39 $ 732,000,000 Total 200 $2,028,000,000
Introducing ROI in Texas
The 1993 Legislature began adopting the ROI process when it approved the Information Resource Management Act (IRMA), which outlines broad goals for state IT development:
It is the policy of this state to coordinate and direct the use of information resources technologies by state agencies and to provide as soon as possible the most cost-effective and useful retrieval and exchange of information within and among the various agencies and branches of state government and from the agencies and branches of state government to the residents of this state and their elected representatives. The Department of Information Resources exists for these purposes.
IRMA defined information resources as “the procedures, equipment, and software that are employed, designed, built, operated, and maintained to collect, record, process, store, retrieve, display, and transmit information, and associated personnel including consultants and contractors.” “Information resource technologies” includes “data processing and telecommunications hardware, software, services, supplies, personnel, facility resources, maintenance, and training.”
The executive director of the Department of Information Resources (DIR) is the state’s chief information officer of the state. As such, state law grants the executive director authority over
...all aspects of information technology for state agencies, including:
- the use of technology to support state goals;
- functional support to state agencies;
- technology purchases;
- deployment of new technology;
- delivery of technology services; and
- provision of leadership on technology issues.
The 2001 Legislature created the Electronic Government Program Management Office (PMO) within DIR to direct and facilitate electronic government projects affecting multiple state agencies. The PMO is charged with identifying and approving funding for these projects, creating teams to coordinate multi-agency efforts and eliminating unnecessary duplication. It also is authorized to establish standards that state agencies must follow during such projects.
The PMO’s role, however, has been largely advisory; project funding and implementation have remained with individual agencies. The PMO has not been able to meet its legislative mandates during its first two years of operation, largely because the anticipated funding for projects and operations was not appropriated. This funding would have included approximately $3 million for projects, operations and 10 full-time positions.
The PMO’s success will depend upon the establishment of an adequate funding model that draws revenue from state agencies participating in selected PMO projects to support the office’s operations, as recommended elsewhere in this report. Appropriate funding would allow the PMO to manage projects effectively; create enforceable standards to ensure accountability on major IT projects; and document savings and cost avoidance from its activities.
The PMO has a 15-member advisory committee consisting of state and local government IT experts, one higher education expert and at least two private business IT experts.. It also has a legislative oversight committee of three state senators and three state representatives.
A. State law should be amended to create a return on investment (ROI) program to guarantee fiscal responsibility in state information technology (IT) expenditures.The statute should outline broad goals to measure the success, cost and long-term ROI presented by each major state-funded IT project. It should establish budgetary performance measures for agency projects that quantify actual returns on investment upon completion.
The program should be implemented by the Department of Information Resources (DIR) and overseen by the Quality Assurance Team (QAT). DIR, in turn, should assign responsibility for the program to the Program Management Office (PMO), which should be held accountable for the outcomes, costs and timeliness of all projects. The PMO should identify and report all cost savings and cost avoidance amounts from its activities to DIR, the QAT, and the Legislative Oversight Committee.
B. The Quality Assurance Team should select projects and major IT or telecommunications equipment purchases planned for the 2004-05 biennium for ROI review.Each agency or group of agencies with a project selected for the ROI program should submit implementation plans to DIR demonstrating how their project will:
DIR and the QAT shall review and approve each plan. Projects should not proceed without this approval. The QAT may continue to require periodic reports or audits as needed while implementation progresses.As the ROI program develops, DIR and the QAT may consider the past performance of agencies and vendors in ROI projects before approving any further projects in which they participate.
- benefit individual citizens and the state as a whole;
- use technology owned or adapted by other agencies to the fullest extent;
- employ DIR’s IT standards, including Internet-based technology standards, to the fullest extent;
- be easily expanded to serve other citizens or agencies to the extent practicable;
- seek cooperative and collaborative efforts from other state agencies;
- be developed on time and on budget;
- produce quantifiable returns on investment; and
- meet any other criteria developed by DIR and the QAT.
C. To prevent cost and schedule overruns, state law should be amended to authorize the pooling of funds appropriated for ROI projects in an account under the authority of the executive director of DIR. The director should be required to notify the Comptroller of Public Accounts to release funds from this account to reimburse agencies for their expenditures.The creation and implementation of an ROI program based in part on Iowa’s model would allow both DIR and the PMO to fulfill the Legislature’s intent in creating them. Agencies with IT projects could and should seek the expertise of the PMO in developing plans. This consultation should lead to better planning, improved interagency coordination, effective project management and, ultimately, better services to Texas taxpayers at a reduced cost.The reimbursements should not exceed the total amount appropriated for each project. Reimbursements should occur at regular intervals during project implementation as the project’s sponsors and vendors demonstrate to the director’s satisfaction that the project is proceeding according to the approved ROI plan.The project’s sponsors should be allowed to appeal first to the QAT and finally to the Legislative Budget Board if they disagree with the director’s actions.
D. State law should be amended to require that any expenses of an ROI-approved project over the approved amount must be paid only from existing funds appropriated to the relevant budget strategy of the sponsoring agency.In the case of multiple sponsoring agencies, the cost overrun would be paid in amounts proportionate to each agency’s financial participation in the project.
E. State law should be amended to prohibit agencies from funding rejected ROI projects from other sources.If a project is not deemed sound enough to receive ROI funding, its sponsors should seek alternatives or additional agency sponsors before reapplying, and should not be penalized for doing so. In fact, agencies should be encouraged to reapply with better projects.
Although the goal of the Return on Investment (ROI) program is to bring fiscal responsibility to the implementation of major information technology (IT) and telecommunications projects, its fiscal impact will depend on future legislative funding for such projects and cannot be estimated. Even so, based on Iowa’s experience, Texas should be able to generate annual savings of at least 10 to 15 percent on its IT expenditures.
In Texas, most major IT projects are funded through capital appropriations (those used to purchase or enhance assets, in this case computer equipment and systems, valued at more than $20,000 with an expected life of greater than one year). Of $451.7 million in capital appropriations to all agencies in the 2002-2003 biennium, $277.8 million is directly attributable to IT and telecommunications acquisitions. Of that $277.8 million, $119.5 million came from general revenue, $38.3 million from dedicated funds within general revenue and $120 million from the dedicated Highway Fund.
Assuming that these appropriations remain the same in the 2004-2005 biennium, and that 10 percent savings are achievable with a ROI program, the state could save $11.9 million in general revenue, $3.8 million in dedicated general revenue and $12 million in highway funds, for a total of $27.7 million.
Because the implementation of an ROI program is within the current duties of the Department of Information Resources and the Quality Assurance Team, no additional personnel would be required.
Based on data retained by the Comptroller’s office on funds spent by state agencies and institutions of higher education on information technology equipment, supplies and technical support in fiscal 2001.
Based on data supplied by the State Auditor’s Office on October 17, 2002 concerning statewide aggregate IT spending, both actual and projected, for fiscal 1998 through 2003.
Based on data supplied by the Legislative Budget Board on August 22, 2002.
Legislative Budget Board and the Office of the State Auditor, Annual Report of the Quality Assurance Team (Austin, Texas, January 16, 2002), p. 1, http://www.lbb.state.tx.us/QAT_Report/QualityAssurance_TeamReport2001_1201.pdf. (Last visited October 15, 2002.)
Iowa Information Technology Council, Return on Investment Program Manual for Fiscal Year 2004, p. 2, available in pdf format from http://www2.info.state.ia.us/roi/FY2004/ROI_Program_Manual.pdf. (Last visited October 18, 2002.)
National Association of State Chief Information Officers, “State of Iowa Return on Investment Program Nomination Form for State IT Initiatives Award,” p. 5, http://www.nascio.org/scoring/files/2002Iowa9.doc. (Last visited October 10, 2002.)
National Association of State Chief Information Officers, “State of Iowa Return on Investment Program Nomination Form for State IT Initiatives Award,” p. 6.
Interview with Paul Carlson, Iowa Information Technology Division, Des Moines, Iowa, September 24, 2002.
Iowa Information Technology Council, Return on Investment Program Manual for Fiscal Year 2004, pp. 8-9.
Interview with Paul Carlson, Iowa Information Technology Division, Des Moines, Iowa, November 9, 2001.
Interview with Paul Carlson, September 24, 2002.
Interview with Paul Carlson, November 9, 2001.
Memorandum from the Quality Assurance Team to legislative leaders, Austin, Texas, January 16, 2002.
Tex. Gov’t Code §2054.001.
Tex. Gov’t Code §2054.001.
Tex. Gov’t Code §2054.0285.
Tex. S.B. 1458, 77th Leg., R.S. (2001).
Tex. S.B. 1, 77th Leg., R.S., Article IX, Part 11.05 (2001).
Tex. Gov’t Code §2055.057; and interview with Shannon Porterfield, director, Program Management Office, Texas Department of Information Resources, August 15, 2002.
Tex. Gov’t Code §2055.055(c) and §2055.057(a); and interview with Pat Keith, Program Management Office Advisory Council member and chief information officer, Texas State Auditor’s Office, Austin, Texas, September 11, 2002.