Office of the Auditor General of CanadaOAG reports published in the past are available through Publications.gc.ca.
2003 April Report of the Auditor General of Canada Chapter 2—Managing the Quality of Financial Information
2003 April Report of the Auditor General of Canada
Chapter 2—Managing the Quality of Financial Information
2.1 High-quality financial information that is relevant and useful is important for departmental and government-wide decision making, for preparing the government's summary financial statements, and for reporting to Parliament and to the public the full cost of delivering government programs. As part of its Financial Information Strategy (FIS), the government has invested over $600 million in new financial systems, policies, and training. Despite being in place for two years, the new FIS infrastructure is still not being used to its full potential.
2.2 Managers find it difficult to access the complete financial information they need on-line. Therefore, they continue to maintain their own spreadsheets or local systems, integrating financial data from several systems. These "black books" provide them with the more complete, timely, and accurate picture of their financial operations that they need for making appropriate decisions, projections, and future plans. While generally confident about the basic cash expenditure information, managers expressed some concerns—for example, about delays in personnel changes being reflected in their salary costs, usually the largest portion of their budget. If managers do not use or challenge the data in the departmental systems, there is a risk that the quality of the data may suffer.
2.3 Departments have implemented, or are planning to implement, a number of financial data quality assurance practices, but these practices vary by department and within departments. In addition, there is no formal process to gather and share departmental best practices for ensuring the quality of financial data either inside the departments or among them.
2.4 The Treasury Board Secretariat provides limited guidance on the quality of financial data, and its Framework for the Management of Information is still in the early stages of development. More guidance is needed to help ensure that departmental financial data are prepared to a consistently high standard, primarily for use by departments, but also for high-quality government-wide reporting. We found that some planning for this is underway.
2.5 The Office's assessment of internal controls for the new financial systems in a selection of large departments found weaknesses, such as in managing access rights to the system. Departments are making progress in correcting these weaknesses, but we are continuing to press for quicker action. Managers need to be educated on the importance of different types of controls and how they can be applied to help ensure high-quality financial data.
Background and other observations
2.6 The government is implementing its Comptrollership Modernization and Results for Canadians initiatives. To carry out this process, it needs good-quality information on the full cost of departmental programs. However, departments are not making full use of activity or program costing, including the full allocation of corporate overheads. The systems, policies, and practices for capturing and reporting full cost information need to be improved.
2.7 The current government environment encourages managers to focus on the amount of their original allotment of money left to spend—their "free balance." As a result, full cost information is not as important to managers as cash expenditure information, so they do not focus on full cost information and its quality. Data quality is usually defined in terms of "fitness for use" (usefulness). The best technique for ensuring high-quality information is to encourage its use in day-to-day management. This is not happening. More work is needed to provide high-quality financial information on costs of programs and to encourage managers to use that information.
2.8 While some errors still occur in summary financial data submitted by departments to the Receiver General, the quality of the data has improved over the past year. The Receiver General's quality assurance unit has worked with departments to help them identify and deal with errors uncovered. However, without information on the nature of the detailed transactions behind the summary data, the Receiver General can identify only some obvious coding errors. In addition, the Receiver General represents the end of the information chain; errors noted at that level indicate a lack of quality assurance practices closer to the source (data entry), where prevention and correction are much more cost-effective.
The Treasury Board Secretariat and the Receiver General have responded. They agree with the recommendation (2.39) and describe the work they currently carry out or plan to carry out regarding financial data quality.
Public Works and Government Services Canada, Transport Canada, the Department of Foreign Affairs and International Trade, and Human Resources Development Canada have responded. They agree with the recommendations and indicate the actions they currently take or plan to take to deal with financial data quality.
The Royal Canadian Mounted Police agrees with the chapter and is looking forward to working with the Office of the Auditor General to make the required improvements.
Evolution of the government's financial management environment
2.9 Having access to reliable information and using it appropriately is a fundamental part of management decision making as well as informing Parliament and the public. Regardless of whether the information concerns financial, human resources, operational, or other performance matters, the quality of the data it contains is important to its value and, ultimately, its use. Data quality is therefore a critical factor in the successful implementation of all information systems.
2.10 The government recently implemented new financial information systems in all departments and agencies as part of its Financial Information Strategy (FIS). In 2001-02, it also made further progress in introducing accrual accounting in departments. In this new financial environment, managers will have access to new forms of financial information processed through new systems and will be expected to use that information.
2.11 FIS is a key component of the government's Comptrollership Modernization Initiative. The pilot stage of the initiative was recently completed and it has begun to roll out government-wide. Modern comptrollership aims to provide managers with integrated financial and non-financial performance information, a sound approach to risk management, appropriate control systems, and a shared set of values and ethics. The Treasury Board policy on the responsibility of departments to monitor comptrollership calls for ongoing reviews of financial management accountability. These reviews are intended to ensure that managers make decisions with timely, relevant, and reliable information.
2.12 We reported on the government's implementation of the comptrollership initiative in our October 2000 and April 2002 reports. In the 2000 Report, we noted that program managers expressed concerns about the timeliness and accuracy of the financial data from the departmental systems. To compensate, many managers maintained separate or supplementary books and records (commonly referred to as "black books"). In the 2002 Report on comptrollership strategies, we said that many managers were not yet aware of the comptrollership initiative and its role in their day-to-day management.
2.13 FIS and comptrollership are prerequisites for the broader Results for Canadians initiative. This initiative requires that departments tell Parliament and Canadians what they are achieving with the significant resources at their disposal. The more recent Government On-Line and eGovernment initiatives add a new dimension to these requirements since they have the potential of expanding the public's access to information. The common thread in these new initiatives is that they all point to a need for high-quality management information that is complete, accurate, relevant, and timely.
2.14 When making decisions using this new financial information, managers must have access to the type of information they need and must have confidence in the quality and timeliness of that information; otherwise, they may pay limited attention to it.
2.15 According to the Treasury Board Secretariat, the investment in new financial systems for FIS amounted to over $600 million. This total did not include costs prior to 1995 or the potentially significant maintenance and upgrade costs that will be incurred over the lifetime of the systems. In addition, in our December 2001 chapter on FIS, we said that because the Secretariat had not set up a cost-capturing framework at the beginning of the project, we could not reach a conclusion on the accuracy of this figure.
Information is important to the government
2.16 The government is in the information business. As the Chief Information Officer Branch of the Treasury Board Secretariat says in the preamble to its Framework for the Management of Information in the Government of Canada, "All of the services that we [government] provide to citizens, to businesses, and to internal clients are about information in one way or another. The provision of information is often the service itself...Fundamentally, most government activities are about managing information."
2.17 Financial information plays a large part in the government's business. In 2001-02, the Government of Canada received over $170 billion in revenue and spent over $160 billion. Knowing where that money comes from and where it goes is important to government decision makers, Parliament, and Canadians. The financial information in the Public Accounts of Canada provides readers with a high-level view of government finances, including whether departments have remained within their parliamentary spending authorities. Parliamentarians and the public are also interested in the cost of government programs, particularly where costs are to be recovered or fees are to be set. In the four pillars of modern comptrollership, outlined in the Comptrollership Modernization Initiative, the Treasury Board Secretariat says that managers also need to know what it may cost to produce a result because there are alternative ways of doing so, each with a different cost. Being able to associate resources with results makes for more informed choices.
2.18 Departments need high-quality information to make effective use of all of their funding, as indicated by the following examples:
- In addition to its role as the federal police force, the RCMP provides police services to most provinces and many municipalities on a contract basis. The RCMP needs high-quality information on the costs of the services it provides to its clients, to support the recovery of those costs in accordance with the contracts.
- The Department of Foreign Affairs and International Trade provides services overseas on behalf of other government departments and needs to know the cost of providing those administrative services so that other departments can pay their share of those costs.
- The Real Property Services Branch of Public Works and Government Services Canada provides accommodation services for government departments. While these services are provided without charge to most departments, the departments are nevertheless notified of the annual cost of accommodation services to better understand the full cost of departmental programs.
- Transport Canada is required to capture and report full costs for its portion of the government's efforts to put new security measures in place.
- The senior management committee headed by the Deputy Minister of Human Resources Development Canada needs accurate cost information to compare one benefit program with another so that savings, best practices, and lessons learned can be found and shared, thereby optimizing the cost to Canadians.
Alternative service delivery, outsourcing, and the creation of agencies highlight additional reasons for knowing total costs and for having good-quality information on the assets and liabilities managed by departments.
What is data quality?
2.19 There are many ways to define data quality (Exhibit 2.1). The common thread in these definitions is that data quality depends on the usefulness of the data—that is, their "fitness for use."
2.20 In addition to the definitions of data quality, there are a number of data and information maturity models that can help an organization assess its level of data quality management. One example is the Information Quality Management Maturity Grid (Exhibit 2.2). The goal is to arrive at the point where information quality is an essential part of the organization and improving information quality is a normal and continuing activity. The National Archives of Canada has also developed the Information Management Capacity Check, which departments can use to establish a baseline for information management practices, including information quality.
Focus of the audit
2.21 We carried out the audit in the context of our previous work on quality—for example, our work on information quality at Statistics Canada (April 1999, Chapter 3), on departmental performance reporting (April 2002, Chapter 6), on the implementation of the Financial Information Strategy (December 2001, Chapter 1), and on the quality of health statistics (December 2002, Chapter 6).
2.22 The objective of this audit was to assess the government's systems, policies, and practices to manage the quality of financial information. This consisted of two parts. The first part was to determine whether departments have put in place financial systems, policies, and practices to provide managers with appropriate and reliable financial information. The second part was to determine whether the central agencies have put in place systems, policies, and practices to provide overall direction and guidance to departments and to manage the overall quality of government financial information.
2.23 Our audit included the following federal organizations:
- Department of Foreign Affairs and International Trade;
- Human Resources Development Canada;
- Public Works and Government Services Canada (PWGSC), primarily the Real Property Services Branch;
- Royal Canadian Mounted Police (RCMP); and
- Transport Canada.
2.24 We also report the results of our controls assessments in eight departments as part of our new approach to auditing the Public Accounts of Canada that will rely on controls of financial systems. More information is available in About the Audit at the end of the chapter.
Observations and Recommendations
Clearly stated policies, guidance, and practices
Limited central guidance on financial data quality
2.25 Under the Financial Information Strategy, the government has moved to a decentralized model of accounting whereby departments are responsible for maintaining the detailed financial information, and only summary data are sent to the Receiver General for reporting in the Public Accounts of Canada. Given the importance of the departmental data for government-wide decision making and for preparing the government's summary financial statements, we expected that the Treasury Board Secretariat would provide departments with a wide variety of guidance and best practices on data quality. This would help ensure that the departmental data were prepared to a consistently high standard for both departmental use and government-wide reporting. We also expected the Receiver General to have data quality assurance processes in place.
2.26 Internal controls, audit, guidance, and best practices, all part of modern management, have a significant role to play in ensuring data quality. We noted that other jurisdictions (for example, the United States and the United Kingdom) have provided central guidance (either by central agencies directly or by national audit offices in consultation with central agencies) to their departments to help ensure consistent data quality standards and practices across the government (Exhibit 2.3).
2.27 We found that the Treasury Board Secretariat provided limited guidance on financial data quality. In May 2002, the Secretariat issued an information bulletin to senior financial officers telling departments that the quality of financial information is important and reminding departments of the need for effective internal controls and adequate oversight by internal audit. According to its officials, the Secretariat had also communicated the importance of financial data quality in a number of presentations to the financial community, such as to the senior full-time financial officers. In addition, the FIS Accounting Manual helps departments determine the appropriate accounting treatment for a number of situations.
2.28 The Chief Information Officer Branch of the Treasury Board Secretariat created its Information Management Division in the fall of 2000, with a goal to improve and enhance the management of government information in the digital age. The Division recently began work on the Framework for the Management of Information in the Government of Canada, which promotes a life cycle approach to information management. One aspect of this framework covers data quality. The development of the framework is still in its early stages, and the data quality section and guidance have yet to be developed.
The Treasury Board Secretariat has made efforts to help departments improve financial data quality
2.29 The Treasury Board Secretariat recently implemented a number of steps that could help departments produce high-quality financial data. These steps include the sign-off of departmental trial balances by the senior full-time financial officer or accounting delegate and the review of departmental data before the end of a fiscal year.
2.30 The Secretariat is developing a high-level control framework to help departments develop control systems. When completed, the framework could be a useful tool for monitoring the efforts of departments to develop and implement controls.
2.31 The Secretariat reviews the departmental financial statements and provides feedback to the departments. It has also created a working group to discuss reporting issues related to financial statements. Publication of departmental financial statements would help promote financial data quality in departments because of the statements' visibility to the public. However, this has not yet been done. As we previously noted in our Observations on the Financial Statements of the Government of Canada and in our chapters on the Financial Information Strategy, the current model for departmental financial statements does not include some major costs and liabilities—for example, liabilities maintained centrally by the Treasury Board Secretariat and services provided without charge by other government departments. As we reiterated in Chapter 5 of our December 2002 Report, we believe that without this information, the departmental financial statements do not provide complete information on departments' program costs and financial positions. The Secretariat indicated that the departmental financial statements will not be published until they contain complete information.
The Receiver General has put in place quality assurance practices
2.32 The Receiver General created a quality assurance unit in 2001 to monitor the quality of the summary departmental data being input into the Central Financial Management and Reporting System (CFMRS). Each month, departments provide the central system with summary information for each combination of government-wide codes. This amounts to over 70,000 records monthly. With the exception of tests of control account balances, the CFMRS has no built-in error checking. To carry out its work, the quality assurance unit at the Receiver General developed over 60 quality assurance reports. These reports include tests for inappropriate coding combinations or zeros entered where a code is expected. The Receiver General provides departments with copies of the quality assurance reports and discusses the contents with them regularly.
2.33 Statistics maintained by the Receiver General show improvement in the quality of the summary data in the CFMRS. The total number of errors reported in October 2002 was 639, or about one percent of total monthly records. This compares with 1,358 errors reported in April 2002 and 2,285 in October 2001. The main errors were the entering of zeros where activity codes were required and invalid coding of transactions internal to government; together these accounted for over one third of the errors.
2.34 The Receiver General's quality assurance process is able to detect only some obvious coding errors. Without information on the nature of the detailed transactions behind the summary data, the Receiver General cannot certify the accuracy of the original coding. Only the departments have the necessary information. In addition, the CFMRS represents the end of the information chain, which begins with data entry in departments. The identification of errors at the Receiver General level indicates the lack of quality assurance practices closer to the source, where prevention and correction are much more cost-effective. The onus is on the departments to ensure that their data are complete and accurate, and particularly that there are no obvious coding errors, before the summary data are sent to the CFMRS for inclusion in the accounts of Canada. Errors reduce the usefulness of this government-wide information.
2.35 The Office of the Auditor General also carries out an annual audit of the government's summary financial statements, which are published in the Public Accounts of Canada. In the Observations of the Auditor General on the Financial Statements of the Government of Canada and in our December 2002 chapter on financial management and control, we commented on the issues and problems found during the 2001-02 audit. These included a lack of timely reconciliation of control and suspense accounts; errors in completing Public Accounts submissions; unreconciled interdepartmental transactions; and corrections to departmental financial information during the audit, including several significant corrections to account balances at the end of the audit. Although these issues were resolved at the government-wide level to allow us to issue an opinion without reservation on the 2001-02 financial statements of the Government of Canada, their resolutions may not be reflected at the departmental level; this reduces the usefulness of the information to managers. Also important to consider is that our audits of the summary financial statements are carried out to an audit level of materiality for the government as a whole. Management's level of materiality is much lower because of the need for high-quality, detailed information to support the decisions made by managers at a program or departmental level. (More information is available in the Observations of the Auditor General on the Financial Statements of the Government of Canada, published in the Public Accounts of Canada.)
Difficulties in capturing and reporting the full cost of activities
2.36 The financial systems need to be designed to permit managers to view the data in a variety of ways—accounting, authorities, and full costs. When talking about the quality of financial information, all of these views need to be considered. For a view of the full costs, we noted that departmental coding structures normally are able to capture the costs of activities, but there needs to be some way of allocating the costs of internal activities to determine and report the full costs of a department's services to Canadians. A number of costing methods are available to give managers information on the full cost of their programs, including the cost of corporate overhead supporting the programs. We found that costing methods are not widely used or standardized in departments, although some departments are exploring particular methods—for example, Public Works and Government Services Canada and the Department of Foreign Affairs and International Trade are considering introducing activity-based costing.
2.37 Departments use an activity code structure to assist in providing information to the central agencies for government-wide reporting. Departments and agencies identify their business lines that are known as government-wide activity codes in the chart of accounts. These business lines form the basis for funding and reporting. The generally accepted view of business lines is that they should represent the external services provided to Canadians. However, our review of the government-wide chart of accounts showed that almost 90 percent of the 44 entities that have more than one business line also have separate corporate administration business lines. Because the costs of those business lines are not allocated to the business lines serving Canadians and other clients, the full cost of departmental activities is not being presented to Parliament and the public in an easily understood and usable manner. Supporting schedules could show how the full costs are derived, including identifying corporate overhead.
2.38 An important aspect of the government's Results for Canadians initiative is that "departments and agencies need to produce information on program costs and results required for sound decision making. They must bring together financial and non-financial performance information to link costs with actual or expected results." One of the first steps in ensuring high-quality cost information for programs is to have the policies, systems, and practices in place to properly capture it. In addition, central direction is needed to ensure consistent application among departments.
2.39 Recommendation. The Treasury Board Secretariat and the Receiver General should provide departments with guidelines on financial data quality and monitor their application in departments. This monitoring should include identifying and sharing best practices for ensuring the quality of financial data.
Treasury Board Secretariat's and Receiver General's response. As noted in the chapter, the Treasury Board Secretariat and the Receiver General have already provided guidance on data quality. The Receiver General has provided guidance in the form of the Receiver General Manual and communications and explanations regarding CFMRS exception reports. The Chief Information Officer Branch and the Comptrollership Branch of the Treasury Board Secretariat have developed or promoted the development of various types of products and best practices related to data quality, some of which are mentioned in the chapter. The following Secretariat Web site on modern comptrollership practices is one such venue: http://www.tbs-sct.gc.ca/cmo_mfc/practices_e.asp
The Secretariat will continue to co-ordinate internal efforts and work with other organizations such as the Receiver General and the Office of the Auditor General to develop and communicate guidance on data quality. Specifically related to financial data quality, the Secretariat has established a working group on financial data quality that will encourage departments to share best practices. The group, which includes participants from various departments, the Treasury Board Secretariat, the Receiver General, and the Office of the Auditor General, commenced meeting in February 2003. The subject of financial data quality will also continue to be discussed at meetings of the Interdepartmental Financial Statement Working Group chaired by the Secretariat. In addition, an interdepartmental working group has been created to guide the development of a generic control framework, focussed on financial controls, that all departments will be able to tailor to their specific needs.
Departmental quality assurance practices vary
2.40 None of the departments we visited had a comprehensive financial data quality assurance plan in place. However, departments have implemented, or are planning to implement, a number of financial data quality assurance practices. These include the following:
- post-payment verification;
- sign-off on financial information;
- goals in manager performance contracts to improve data quality;
- system edits;
- reconciliation of accounts with Receiver General central systems and other departmental systems; and
- ad hoc exercises for cleansing data.
2.41 Post-payment verification. Post-payment verification is a process whereby transactions, primarily expenditures, are reviewed by the finance group for errors. The extent and timing of the review is generally risk-based, with high-risk transactions often pre-audited prior to payment. Other transactions are normally sampled after payment. The criteria used in these reviews generally look at whether the payment meets the requirements for payment, has been properly approved, went to the right payee, and was in the right amount. Accounting coding (for example, the proper use of an activity code) is usually a secondary, non-critical criterion. However, given the importance of coding to providing good financial information, we believe that it needs to be seen as critical. The results of the post-payment verification process are discussed with the managers involved, critical errors are corrected, and additional training is provided if necessary. PWGSC, Human Resources Development Canada, the Department of Foreign Affairs and International Trade, and Transport Canada have established post-payment verification processes. However, these processes vary by department and, in some cases, within departments. Exhibit 2.4 outlines PWGSC's process.
2.42 To test the quality of departmental financial data, we selected a small statistical sample of 25 to 30 transactions from each of the five departments we audited. The sample was drawn from transactions processed in June and July 2002. The sampling plan was not designed to test for the completeness of data or for the salary allocation issue that was raised by managers during the audit. We obtained supporting documentation for the sample transactions and examined the appropriateness of the departmental and government-wide coding as well as the accuracy and timeliness of the amounts recorded. We can conclude from the results of the sample testing that the overall error rate is less than five percent.
2.43 Review and sign-off. Managers know their operations and are therefore the best people to review their financial results for unusual items. Four of the five departments we audited therefore ask managers to sign off on their expenditures to date and their forecast spending to the end of the year. This practice varies by department and ranges from monthly sign-off of key reports at Transport Canada to periodic sign-off of forecasts at Human Resources Development Canada, PWGSC, and the RCMP.
2.44 The review and sign-off process helps ensure that a department does not overspend its appropriation—that is, the amount of spending authorized by Parliament. The focus of the review is on the manager's forecast of "free balance." The manager is usually supported by a financial officer who reviews the information and makes the necessary adjustments and corrections before the manager reviews and approves the forecast.
2.45 We found that departmental managers keep a close eye on their cash expenditures. Forecasting future expenditures is an ongoing requirement to ensure that they do not overspend their discretionary funds. All of the managers we interviewed said that they were generally confident about the cash expenditure information in their financial systems because of the focus on an accurate and up-to-date free balance. They were less confident about the salary data because of limited accessibility, a lack of good reports, and delays in the processing of human resources transactions.
2.46 In all of the departments we audited, a central human resources unit processed human resources transactions. Salary costs are the largest part of most managers' budgets; therefore, allocating and forecasting those costs is critical to managing their free balances. Centralized processing of human resources transactions adds to the delays in processing transactions related to new employees, departures, transfers, acting positions, and all other staffing matters. As a result, most managers maintain a separate salary management system or spreadsheet and/or a list of open positions to monitor and forecast their salary costs.
2.47 Goal in performance contracts to improve data quality. In the Real Property Services Branch of PWGSC, manager performance contracts now include a goal to improve data quality by 10 percent. While this goal relates primarily to operational data, it is an example of a practice that could be extended to financial data. The RCMP plans to put at risk a part of its managers' performance pay, based on the accuracy of their last quarterly financial forecast. The importance of financial data quality in the eyes of managers would increase if appropriate and measurable goals were included in performance contracts.
2.48 System edits. We found that the departmental systems include a variety of system edits (error checking) of data input. These include editing coding and code combinations and limiting input to particular responsibility centre codes. System edits are an important tool to assist in improving the quality of financial information early in the process.
2.49 Reconciliation of accounts. Departments are required to reconcile their departmental accounts with the Receiver General's control accounts on a monthly basis prior to submitting their trial balances to the CFMRS. The CFMRS does not accept departmental trial balances unless the control accounts agree with those of the Receiver General. These control accounts simply ensure that the cash payments and receipts processed through the Receiver General's payments, deposits, and payroll systems have been accounted for in departmental systems. The accounting distribution of those items is carried out at the departments, and the Receiver General must rely on the departments to ensure that the information is correct. For example, the Department of Foreign Affairs and International Trade (DFAIT) has made significant improvements in its CFMRS reconciliation process by creating a reconciliation unit whose staff have developed a sense of ownership over the data and their accuracy. Data from the new departmental financial systems and older existing systems are also reconciled.
2.50 Cleaning up data. Departments have carried out a number of ad hoc exercises to clean up the data in their systems. Although these exercises are needed to ensure good-quality data, they are often time-consuming. For example, DFAIT recently verified the human resources data of its European missions as at 30 September 2002 because the data had become outdated. When the work was completed, the Department would have had good information up to 30 September. However, new data entered in the intervening period may not have been validated as thoroughly, particularly if the business processes leading to the original problem had not been reviewed and improved. Having a well-functioning practice in place to ensure that good-quality data are entered in the system at source virtually eliminates the need for ad hoc cleanup afterwards.
Regional differences in practices
2.51 The departments audited had considerable regional operations. We found that the quality assurance practices were not consistently applied across departments and that regional differences in approaches existed. While some regional differences in practices may be appropriate, for example, based on risk, departmental financial information needs to be seen as a corporate resource with quality assurance standards applied across the department. In addition, managers reported a lack of formal processes to capture and share best practices within the departments, for example, on quality assurance.
Internal audit involvement is limited
2.52 In most of the departments, internal audit had limited involvement in reviewing departmental quality assurance processes. As part of their assurance role under the new policy on internal audit, all departmental audit groups have undertaken, or are planning to undertake, some focussed audit work on the quality of financial information. For example, Internal Audit and Risk Management Services of Human Resources Development Canada plans to carry out ongoing testing of the Department's financial control framework. The focus will be on examining controls for the departmental financial system and monitoring the results of post-audit activities and the work carried out by internal control groups in the finance area.
No formal process for gathering and sharing best practices
2.53 Although the Treasury Board Secretariat has some working groups where best practices could be discussed, there is no formal process to gather and share departmental best practices for assuring the quality of financial data. Sharing best practices could help to ensure consistency in quality assurance standards across the government and minimize "reinventing the wheel." For example, as previously described, departments have created post-payment verification processes to ensure the accuracy of their expenditure data. But according to the financial managers we interviewed, there is no central co-ordinator at the Secretariat to facilitate the sharing of these data quality practices. Departments need to use their own contacts and networking methods to find out what is being done at other organizations.
2.54 Recommendation. Departments should develop appropriate quality assurance plans and practices that are consistently applied across the department. Managers should sign off regularly on the quality of data.
Public Works and Government Services Canada's response. The monitoring of data quality was a critical element in the 1999 development of PWGSC's account verification framework to support the devolution of data entry and operational management reports of the centralized financial system to its line organizations. The department-wide implementation of the framework, including regions and special operating agencies, will be completed in 2003-04. To standardize accounting information, the Department is developing an accounting manual, establishing a new reconciliation framework, and implementing an active monitoring framework to review data quality. Each month, management signs off on the quality of data in the Department's trial balance. In addition, a Quality Assurance Program is being developed to integrate and support the 2002-03 Privy Council Office's corporate priorities for the Public Service of Canada into our departmental executives' performance agreements.
Transport Canada's response. Transport Canada agrees with the recommendation and has implemented various system edit checks, including extensive on-line coding edit rules to ensure that data are captured accurately. In addition, the Department has extensive monthly and year-end policies and procedures that provide the framework for ensuring that data are reviewed and reconciled by accounting offices consistently across the Department. Senior accounting staff sign off to indicate that the procedures are completed on a timely basis. As part of the monthly forecasting process, the Department's managers are responsible for reviewing their reports, including salary reports, for completeness and accuracy. Transport Canada actively monitors data quality on a continuous basis, and makes changes to quality assurance practices as required.
Department of Foreign Affairs and International Trade's response. The Department is currently implementing a project entitled The Security and Authorizations Project, which will regulate access to our financial system, improve control, and ensure more uniformity in both headquarters and missions. In line with the Financial Information Strategy (FIS), we are currently improving the corporate reports for use by the Executive Committee on a monthly basis. Combined with the Deputy Minister's approval of the Letter of Representation and the Annual Public Accounts (for which the Department recently received an "A" rating from the Treasury Board), this will ensure senior management's involvement in the quality of our financial data. Regarding our Human Resource Management System, we have recently completed a significant clean up of data and our Human Resource organization is committed to the business process review to ensure that high data quality is maintained.
Human Resources Development Canada's response. The Department concurs with the recommendation. It has quality assurance practices in place and continues to assess and improve the practices, such as our National Post Audit Process. Managers are required to sign off on forecast spending reports.
Departmental controls and control frameworks need improvement
2.55 Chapter 1 of this Report, Integrated Risk Management, outlines how integrated risk management entails managing risk and control activities across an organization. Controls are management's actions to mitigate risks and achieve established objectives. As a minimum, basic controls are needed to reduce the risk that financial data may be incorrect or corrupt and to reduce the potential consequences of errors in financial data. When controls are applied systematically, this is usually referred to as a control framework (Exhibit 2.5).
2.56 An effective framework of internal controls can help reduce the risks of data error and its impact on decision making. These controls are as basic as preventing unauthorized or improper changes to key financial data or requiring a senior official to certify that the financial records are complete, up-to-date, and accurate. A key control is the monitoring of financial information by managers. In our audit we interviewed over 80 managers in five departments and two central agencies. Most relied on their financial assistants to perform this task and to present the information to them for review. Only a few of them said that they carried out this key control by viewing the financial information on-line themselves; in our opinion, doing so is a stronger control because managers see the actual raw data.
2.57 In Chapter 5 of our December 2002 Report, we reported on our work over the past two years at eight large departments to examine the controls covering nine financial systems (see About the Audit at the end of this chapter). The results indicated that controls for the financial systems were weak, particularly in managing access rights to the system.
2.58 Since our December 2002 Report, the Office has carried out or is carrying out additional controls assessments. We also worked with internal audit at one agency to do a controls assessment that the agency sponsored. We have seen similar weaknesses in almost all of the systems that we examined.
2.59 In our view, the rights to access and update financial information were not carefully managed in the computer systems and in the business applications, and managerial checks and sign-offs were not well executed. As a consequence, there is a risk that the organization's manual and managerial controls were not reliable. For example, members of the systems development team were able to modify information in the organization's financial system. This privilege had been granted in the early days of development so that system errors and failures could be swiftly addressed. However, after many months or years of reasonably stable operations, these privileges still remained. A properly controlled financial system would not permit any ad hoc changes to data already entered in the system. Any changes to financial information should be properly logged, validated, and signed off using clear and established procedures. This is necessary to protect the organization and the individuals involved.
2.60 Another example of a common control weakness concerns the movement and transfer of people within an organization. In many cases, people change responsibilities in an organization and perform different functions. Employees might carry the privileges of previous jobs to new jobs without justification, and the appropriate financial officers might not be notified of changes in responsibilities. This security loophole needs to be closed to protect the organization and the individuals. Controlling the number of people who are authorized to perform certain business transactions has a positive effect on the reliability and quality of data.
2.61 Considering the similar problems across all of the large systems and departments that we assessed, we believe that guidance from the Treasury Board Secretariat could play a role in improving the situation quickly. A key aspect is educating managers on the importance of different types of controls—for example, preventive controls that are largely based on systems or detective controls that are largely based on management review. Once established and effective, system-based controls are applied consistently, usually at the source (data entry). The consistent application of controls based on management review depends on the efforts of individual managers; this review is usually carried out near the end of the information chain. Some departments believed that their management controls and reviews of reports were sufficient and that system-based controls were therefore not necessary.
2.62 In Chapter 5 of our December 2002 Report, we recommended that the Treasury Board Secretariat ensure that departments deal with the internal control challenges they face. As we discussed earlier, the Treasury Board Secretariat told us that it plans to issue high-level guidance on control frameworks by the end of the fiscal year. In addition, the SAP financial system cluster group (user groups that share expertise in support of the financial system) has created a tool for generating user profiles to assist departments in managing user privileges.
2.63 We carried out our first controls assessments over two years ago. At the time, departments indicated a willingness to make the necessary changes. A follow-up review showed that Agriculture and Agri-Food Canada has made significant improvements. A similar follow-up at Canadian Heritage found some improvement, but progress to date has been slow. At December 2002, we had not yet started our planned follow-up review of the third organization, the Canada Customs and Revenue Agency. The Agency has reported to us that it has made progress. We continue to press for these problems to be addressed more quickly. Effective controls will be beneficial for the departments and their management as well as for their internal and external auditors.
2.64 There was one fully successful controls assessment last year. Based on our review of controls in the PWGSC Regional Pay System, including the pay-related controls in departments, we found sufficient systems controls and managerial controls in place to change our approach in auditing government-wide payroll costs for the 2001-02 Public Accounts. By relying on the controls, we were able to reduce the amount of detailed testing.
Making full use of financial systems
The new financial systems are not being used to their potential
2.65 Over $600 million has been spent to implement new financial systems across the government. At the time of acquisition, only the financial and material management modules from large Enterprise Resource Planning (ERP) systems such as SAP and Oracle were licensed by the government. Today, these ERP systems are capable of handling a wide variety of functions—for example, financial management, human resources management, and materiel management—in an integrated fashion.
2.66 These financial systems, designed for on-line use, were selected and implemented in part to provide managers with direct access to their management information. However, we found that about 90 percent of the managers we interviewed do not access the systems directly. They rely on their financial and administrative staff to extract and provide them with the information in the format they need.
2.67 This information often takes the form of locally created spreadsheets and other analyses. These "black books" help managers track their spending, forecasts, and free balances. Managers claimed that the quality of data in these locally maintained records was better than the quality of centrally supplied data, with the exception of basic expenditure information. The use of these black books underscores the difficulty of readily obtaining information needed to manage and the lack of confidence in the quality of data in departments' corporate systems.
2.68 Another concern is that departments are not taking advantage of the potential power of their financial systems. The systems are used mainly to process financial transactions. There is a lack of forecasting support, and other modules such as project management and planning are either not used or are not available.
2.69 The first step in creating an environment that encourages high-quality financial information is to reduce the need for black books by ensuring that the central systems are complete, accurate, and up-to-date and present the information in a form that is easy to use. The second step is to ensure that the departmental financial systems provide managers with the functionality and support tools they need. Some departments have plans in place to upgrade their financial systems.
Managers need to integrate financial information from a number of systems
2.70 In general, the government hires managers to run programs and create policy, and thereby support their department's activities and produce results for Canadians. We found that in every department we visited, managers were forced to deal with several internal systems in order to get timely, complete, authoritative, and accurate information. This process is counterproductive to the managers' primary objectives. They find that this additional administrative burden adds to the already high pressure on the time they have to deliver their programs. In addition, quality may suffer when managers have less time to review and challenge the information.
2.71 In the early 1990s, the Treasury Board Secretariat licensed PeopleSoft government-wide as an approved human resources information system. Since then, the PeopleSoft product has evolved into a full Enterprise Resource Planning system competing with a range of products including SAP and Oracle. All of these systems today provide much the same functionality.
2.72 Many departments use PeopleSoft for human resources information and SAP or Oracle for their financial information needs. Since these systems are not built to integrate with other systems easily, managers need to integrate data sources themselves or acquire additional software packages that will perform the integration. For example, in most of the departments we audited, human resource and salary information is maintained in at least two different systems. The lack of integration limits management's access to the complete picture and leads to the development of black books to bring the information together.
2.73 Reporting continues to be a major concern with these systems. The systems were designed to be used online. However, since managers do not use the systems directly or on-line, they require reports. Managers told us that none of the systems provides reporting that is easy to use. The departments recognize this, and in some cases they have used commercial, add-on reporting tools to compensate for difficult reporting features. New tools mean additional costs to departments, often in the millions of dollars, and this is on top of the significant original investment. These reporting tools are sold on their ability to integrate data from a variety of sources. However, the quality of the reporting tools' data is totally dependent on the quality of the systems that feed the data. The purchase of the reporting software provides a focal point for a major exercise to clean up data in a department. However, integrating and reporting information from a variety of systems can easily become a very expensive process.
2.74 It is now possible for organizations, both large and small, to obtain one integrated system for all administrative information. Modern systems do not require multiple or redundant data entry; nor do they force managers to manually integrate the key administrative information that they need to manage their programs. Savings may be gained by reducing the number of software packages that a department must support. The Treasury Board Secretariat has initiated a project to look into setting up shared financial services; however, broader shared corporate services might be more appropriate.
2.75 Making full use of Enterprise Resource Planning systems, integrating them fully with other systems, providing easy-to-use interfaces, and providing managers with full cost information are some of the methods that departments could use to help ensure payback from the significant investment in new financial systems.
Reporting of information to managers needs improvement
2.76 In the departments we audited, the finance/accounting area was usually responsible for creating and maintaining accrual-based information. The information was not yet being provided to operational managers. It was largely considered necessary to satisfy the requirements of the Receiver General and the Treasury Board Secretariat.
2.77 Financial managers said that the accrual-based information is of limited use to operational managers due to the lack of comparability in the presentation of numbers in financial information, budgets, and appropriations. They complained of the need to maintain two sets of books, one to manage and one to report. In their opinion, operational managers would always focus on the use of cash as long as they were accountable to Parliament on that basis. As reported in our December 2002 chapter on financial management and control, the government is currently reviewing how it could incorporate accrual accounting into its budgeting and appropriations process. We recognize that regardless of the outcome of this review, cash management will always be important to the government and its managers. However, since managers are not yet receiving or using accrual accounting and full cost information, they pay little attention to this information or to its quality.
2.78 It is important to identify managers' needs for financial reports on a continuing basis. Surveys, interviews, and working groups are some of the methods a department could use to acquire such knowledge. Local systems or spreadsheets could be another valuable source of information on managers' needs. For example, Human Resources Development Canada has carried out a survey of its managers' needs for financial reports. Similarly, Transport Canada is making an effort to identify and develop specific reports for its managers.
2.79 To improve financial reports for managers, departments could use a business case approach to developing the reports needed, make data more accessible for ad hoc analysis, and educate managers on the availability and use of the new financial information and reports.
2.80 Recommendation. Departments should consolidate or integrate key management information from the various departmental systems that provide financial information and present this information in a useful form to managers. Departments should develop financial reports that are easy to use and should provide appropriate training on the use of the systems and the information they produce.
Public Works and Government Services Canada's response. PWGSC is consolidating its financial information into one system, the Common Departmental Financial System (CDFS). Access to this information is provided through a range of standard and ad hoc reporting tools that include: direct access to CDFS for financial officers; indirect access through the Management Reporting Module (MRM) for responsibility centre managers to review their revenues, expenditures, and budget; and access to a reporting database for management reports. As the CDFS is refreshed daily, it provides accurate and current financial information that supports operational managers. To date, the Department has trained, in the National Capital Area, over 190 financial officers on access to CDFS and over 398 line managers on the use and benefit of MRM. As the system is migrated to the regions and special operating agencies, training will be provided to over 170 additional financial training coaches. The availability of accrual-based information in CDFS will commence in 2003-04. The Department is actively exploring options for improving the integration of performance information and developing management reports that enhance the use of graphics in order to improve their effectiveness as a communication tool.
Transport Canada's response. Transport Canada agrees with the recommendation. In support of modern comptrollership, the Department is currently migrating to a new Web-enabled version of the Oracle Enterprise Resource Planning (ERP) suite and implementing new business intelligence tools. These systems changes, along with the redesign of business processes, will improve administrative efficiency within the Department, provide improved management information, and expand the reach of the Department's internal and external clients using Web-based methods for service delivery. In particular, the project includes a June 2004 implementation of new intuitive financial reporting and budgeting/planning tools. The introduction of these tools, and improved business processes, will provide more accessible, useful financial information for decision making. As part of the implementation process, there will be extensive processes to define user requirements, including ongoing working groups and consultations, cross-country managerial workshops, tool demonstrations, and interviews, to ensure that the financial information needs of managers are addressed. Training plans are also being developed to ensure that managers will be adequately trained. The new business intelligence tools also support data warehousing and will provide the necessary foundation for integrated reporting of financial and non-financial information. Transport Canada's Modern Comptrollership/Management Office will be overseeing the integration of financial and non-financial information within the Department, along with the implementation of other modern comptrollership principles.
Department of Foreign Affairs and International Trade's response. The Department is developing its Business Intelligence solution, which currently integrates financial and non-financial information from a number of source systems. The plans are to also integrate activity business lines and other program information into the solution, which will provide managers with useful and timely reports in their areas of operations. Training currently exists for the systems outlined above and will be enhanced in line with the systems' evolution.
Human Resources Development Canada's response. The Department concurs with the recommendation. It continues to develop and improve reporting tools that integrate financial and non-financial information while at the same time providing the flexibility to meet managers' needs.
2.81 In this audit we examined how departments and central agencies were managing the quality of financial information.
2.82 Departments did not have regular, ongoing processes to determine and respond to the information needs of managers. Practices were ad hoc in two of the departments we audited.
2.83 In general, current quality assurance practices in departments, such as reconciliation of data transferred to the central systems and verification of management information on free balances, are designed to support the financial reporting processes. However, these practices vary by department and within departments.
2.84 The Treasury Board Secretariat has made some effort to provide guidance to departments on managing financial data quality, but it needs to do more.
2.85 The Receiver General has established quality assurance practices for the government-wide data it collects from departments. Its work with departments has helped to reduce the number of errors in those data.
2.86 Data quality needs to be looked at in the context of overall financial management, which is continuing to evolve in the Government of Canada. To get high-quality financial information that will support good management, the government and departments need to create the right environment and infrastructure—one that encourages managers to demand appropriate, good-quality financial information at the right time.
2.87 Quality assurance is a department-wide responsibility. All employees must understand the purpose of data and take ownership to ensure data quality. While quality control practices applied at the end of the business process are useful, these detective practices are not as cost-effective as those that ensure proper entry of the data at source. Staff need to be educated so that they can apply quality control procedures when data are created and/or updated.
2.88 The government has invested a lot in people, accounting policies, and financial systems in its move to a modern financial management environment. It has been two years since the government put in place the new financial systems for its Financial Information Strategy. It is now time to follow through with a concerted effort, led by the Treasury Board Secretariat and senior management, to begin reaping the benefits of this significant investment. This effort will involve encouraging the active participation of all managers in ensuring high-quality financial information, its appropriateness to their needs, and its use in day-to-day decision making.
About the Audit
The objective of the audit was to determine whether the government has put in place appropriate systems, policies, and practices to manage the quality of financial information for managers. This consisted of the following two parts:
- determining whether departments have put in place financial systems, policies, and practices to provide managers with appropriate and reliable financial information; and
- determining whether central agencies have put in place systems, policies, and practices to provide guidance to departments and to manage the overall quality of government financial information.
Scope and approach
We carried out our audit at the Department of Foreign Affairs and International Trade; Human Resources Development Canada; Public Works and Government Services Canada, primarily the Real Property Services Branch; the Royal Canadian Mounted Police; and Transport Canada.
We also reviewed the results of controls assessments carried out by our Office over the past two years in eight federal organizations as part of our planned move to a controls-reliant approach to auditing the Public Accounts of Canada. The organizations were Agriculture and Agri-Food Canada, Canada Customs and Revenue Agency, Canadian Heritage, Public Works and Government Services Canada, Indian and Northern Affairs Canada, the Department of Foreign Affairs and International Trade, and Fisheries and Oceans Canada, as well as the government-wide Regional Pay System managed by Public Works and Government Services Canada.
We also assessed the roles of the Treasury Board Secretariat and the Receiver General in providing overall direction and guidance on managing the quality of financial information.
The audit team interviewed about 80 financial and operational managers in the departments and about 10 managers in central agencies. The team reviewed relevant documentation, including departmental surveys of managers that examined issues related to corporate information systems and reporting. To test the accuracy of departmental data, we reviewed a statistical sample of transactions in each of the departments.
For the departments, the summary audit criteria were the following:
- Senior management and the department have appropriate policies and practices to determine the information and data needs of departmental management.
- Senior management and the department have a sound understanding of the nature of the information and data and their fitness for use. Departmental management has in place a data quality assurance policy and practice to ensure that data used by management are accurate, consistent, complete (in entirety, breadth, and depth), timely (and up-do-date), usable, relevant, and secure.
For the central agencies, the summary audit criteria were the following:
- Central agencies demonstrate information management leadership by providing appropriate guidance to departmental and agency management.
- Central agencies have a sound understanding of the nature of the government-wide information and its fitness for use. Senior management in central agencies have in place information management policies and practices to ensure that government-wide information used by central agency management is available, accurate, consistent, complete (in entirety, breadth, and depth), timely (and up-to-date), usable, relevant, and secure.
Assistant Auditor General: Douglas Timmins
Principal: Eric Anttila
Director: David Willey
For information, please contact Communications at (613) 995-3708 or 1-888-761-5953 (toll free).
Materiality—The level of error in financial statements that would be tolerated before a reader's judgment based on those statements would be affected. (Back)
Free balance—The amount of an original allotment of money that a manager has left to spend. (Back)