2012 Spring Report of the Auditor General of Canada Chapter 2—Replacing Canada’s Fighter Jets
2012 Spring Report of the Auditor General of Canada
Chapter 2—Replacing Canada’s Fighter Jets
Risks associated with participating in the Joint Strike Fighter Program were communicated to decision makers
2.1—There have been three changes from 2001 to 2010 in estimated delivery dates for the F-35 (conventional takeoff and landing variant)
2.2—Estimates for the full production period of unit recurring flyaway costs increased from 2001 to 2009
2.4—Projected benefits (in $billions) to Canadian industry, presented at key points to decision makers and ministers, have fluctuated widely
Performance audit reports
This report presents the results of a performance audit conducted by the Office of the Auditor General of Canada under the authority of the Auditor General Act.
A performance audit is an independent, objective, and systematic assessment of how well government is managing its activities, responsibilities, and resources. Audit topics are selected based on their significance. While the Office may comment on policy implementation in a performance audit, it does not comment on the merits of a policy.
Performance audits are planned, performed, and reported in accordance with professional auditing standards and Office policies. They are conducted by qualified auditors who
- establish audit objectives and criteria for the assessment of performance;
- gather the evidence necessary to assess performance against the criteria;
- report both positive and negative findings;
- conclude against the established audit objectives; and
- make recommendations for improvement when there are significant differences between criteria and assessed performance.
Performance audits contribute to a public service that is ethical and effective and a government that is accountable to Parliament and Canadians.
What we examined
Canada currently operates a fleet of CF-18 Hornet fighter jets purchased in the 1980s with an original life expectancy until 2003, since extended to between 2017 and 2020 after a modernization program in early 2000. National Defence has determined that a suitable replacement for the CF-18 is required, or it will lose its ability to carry out domestic and international missions mandated by the Government of Canada.
National Defence has been a partner in the Joint Strike Fighter (JSF) Program since 1997. Led by the United States, and with eight other country partners, the Program is undertaking concurrent design, development, and manufacturing of the F-35 Lightning II aircraft. It will eventually include a regime for long-term collaborative sustainment. Canada’s participation has been formalized by signing international memoranda of understanding—in 1997, 2002, and 2006—for each of the three major phases of the JSF Program. As of September 2011, the government had disbursed about CAN$335 million toward participation in the JSF Program and related support to Canadian industry. The government has committed a total of US$710 million to the Program.
In May 2008, through the Canada First Defence Strategy, the federal government announced its intent to replace the CF-18 fleet with 65 “next generation” fighter aircraft. Then, in July 2010, the government announced its decision to buy the F-35 Lightning II, without following a competitive process, as the CF-18 replacement.
We examined whether National Defence, Industry Canada, and Public Works and Government Services Canada exercised due diligence in managing Canada’s participation in the JSF Program and in managing the federal decision-making process to acquire the F-35 as a replacement for the CF-18.
Our conclusions relate only to the management practices and actions of public servants. We did not audit private sector contractors and, consequently, our conclusions do not pertain to the contractors’ practices or to their performance. We did not audit the merits of the F-35 aircraft.
Audit work for this chapter was substantially completed on 30 September 2011. Further details on the conduct of the audit are in About the Audit at the end of this chapter.
Why it’s important
Buying major defence equipment is subject to decision-making and project management processes whose aim is to ensure that decisions are well founded, projects are managed effectively, and goods and services are acquired in a way that enhances supplier access, competition, and fairness. National Defence, Industry Canada, Public Works and Government Services Canada (PWGSC), and central agencies are involved.
Buying and maintaining the F-35, or any other fighter jet, will require a significant long-term financial commitment. The F-35 is still being developed and tested, and projections of purchase price and sustainment costs are still being refined. Decisions taken to date as well as those yet to come will have impacts for the next 40 years.
What we found
- National Defence took the appropriate steps in managing Canada’s participation in the Joint Strike Fighter (JSF) Program to develop the F-35. National Defence engaged Industry Canada early, and together they managed industrial participation well. Early efforts to secure contract opportunities for Canadian companies were successful.
- There were significant weaknesses in the decision-making process used by National Defence in acquiring the F-35 to replace the CF-18. By the end of 2006, the Department was actively involved in developing the F-35, and a number of activities had put in motion its eventual procurement. In the lead-up to the government’s 2010 announcement, required documents were prepared and key steps were taken out of sequence. Key decisions were made without required approvals or supporting documentation.
- PWGSC did not fully carry out its role as the government’s procurement authority. Although it was not engaged by National Defence until late in the decision-making process, PWGSC endorsed the key decision to sole source the acquisition of the F-35 in the absence of required documentation and completed analyses. By that time, practically speaking, Canada was too involved with the aircraft and the JSF Program to run a fair competition.
- National Defence did not provide complete information in a timely manner. For example, briefing materials prepared for decision makers did not explain the basis for and limitations of projections of industrial benefits to Canadian companies, and the risks of relying on the projections for decision making. In addition, briefing materials did not inform senior decision makers, central agencies, and the Minister of the problems and associated risks of relying on the F-35 to replace the CF-18. Nor did National Defence provide complete cost information to parliamentarians.
- National Defence likely underestimated the full life-cycle costs of the F-35. The budgets for the F-35 acquisition (CAN$9 billion) and sustainment (CAN$16 billion) were initially established in 2008 without the aid of complete cost and other information. Some of that information will not be available until years from now. If the budgets prove insufficient to cover total costs, the Department will have to find ways to cover additional costs that may be incurred. Alternatively, it may have to seek additional funds from the government or use funds from other parts of its capital or operating budgets.
The departments have responded. National Defence agrees with our recommendation. Its response follows the recommendation.
National Defence, Industry Canada, and Public Works and Government Services Canada have accepted the facts presented in the chapter. Both National Defence and Public Works and Government Services Canada disagree with the conclusions set out in paragraphs 2.80 and 2.81.
2.1 Canada purchased its current fleet of CF-18 Hornet fighter jets in the 1980s, with an original estimated life expectancy to 2003. A major modernization program carried out in the early 2000s extended the life of the aircraft until between 2017 and 2020. According to National Defence, when the CF-18 is retired at that time, unless there is a suitable replacement, Canada will lose its fighter jet capability and consequently its ability to carry out domestic and international missions mandated under the 2008 Canada First Defence Strategy.
Photo: Department of National Defence
2.2 In July 2010, the Government of Canada announced its decision to buy 65 F-35 Lightning II jets to replace Canada’s CF-18 fleet. The announcement was the culmination of nearly 13 years of Canada’s participation in the United States-led Joint Strike Fighter Program. Buying and maintaining the F-35, or any other fighter jet, will require a significant long-term financial commitment. It will have far-reaching economic and operational impacts on Canadians and the Canadian Forces. Decisions taken to date as well as those yet to come will have impacts for the next 40 years.
F-35 Lightning II
Photo: Lockheed Martin Aeronautics
The United States-led Joint Strike Fighter Program
2.3 The Joint Strike Fighter (JSF) Program started in the late 1990s. It is the United States Department of Defense’s largest-ever development and procurement program, aiming to produce an affordable, multi-role aircraft able to perform in combat operations. The JSF Program is unique. Led by the United States, and with eight international partners (Australia, Canada, Denmark, Italy, Netherlands, Norway, Turkey, and United Kingdom), the JSF Program is undertaking concurrent design, development, and manufacturing of the F-35. It will eventually include a regime for long-term sustainment.
2.4 Supporters of the F-35 expect it to be the most technologically advanced strike fighter jet in the world. Three variants of the aircraft will be produced:
- conventional takeoff and landing,
- carrier, and
- short takeoff/vertical landing.
2.5 These variants are intended to replace existing fighter jets operated by the United States Air Force, Navy, and Marine Corps and other militaries around the world. Together, the nine partners are anticipated to buy about 3,100 aircraft, with the United States acquiring 2,443. Canada plans to purchase 65 jets, of the conventional takeoff and landing variant.
2.6 The JSF Program is divided into three major phases. At the beginning of each phase, country partners formalized their participation and financial contribution by signing a government-to-government memorandum of understanding (MOU). The following are the three phases:
- concept demonstration (1997 to 2001);
- system development and demonstration (2001 to 2018); and
- production, sustainment, and follow-on development (2006 to 2051).
2.7 The JSF Program was conceived to be a new model for development and acquisition of military equipment. It has the following features, some of which are unique:
- Industrial participation. Companies from partner countries are eligible to participate in the manufacturing and sustainment over the lifetime of all jets produced. There is no guaranteed work for companies from partner countries.
- Acquisition methods. There are two ways to buy the F-35: through the third-phase 2006 MOU as a partner or through foreign military sales as a non-partner (at a higher price). In both cases, the purchase occurs through the US government. There is no contract directly between partner countries and the manufacturers. The 2006 MOU recognizes that procurement is subject to partners’ national laws and policies.
- Purchase costs. The purchase price paid for the basic aircraft in any given year will be the same for all partners, including the US military. The purchase price does not include the substantial development costs incurred by the United States government. Partners will have to pay the cost of designing and manufacturing any requested modifications to their basic aircraft.
- Input and access. Partner countries have input on the design of the aircraft and the JSF Program through participation on decision-making and technical committees. Canada’s contribution gives National Defence access to the newest technologies and some highly technical information granted only to partners.
- Sustainment. For maintenance and repair, provision of spare parts, technological upgrades, training, and other operating activities, partner countries and industry will be part of a global network. The sustainment strategy is still being developed.
2.8 The Joint Strike Fighter Program Office plays a critical role in managing the JSF Program. It provides a forum for decision making, information sharing for all partners, information technology support, and management of the project on behalf of participants. National Defence has representatives on the JSF Program senior leadership, financial, and technical committees. Notwithstanding its international nature, the JSF Program is US-led and is subject to US law, rules, and Congressional oversight.
2.9 The JSF Program has experienced cost increases, schedule delays, and technological problems. These issues are not uncommon for a major development program. As of April 2011, the US Government Accountability Office estimated total development funding to be US$56.4 billion to complete the project by 2018, which is a 64 percent increase in estimated costs since the system development and demonstration phase started.
2.10 The number of F-35 jets being produced and delivered is expected to increase over time. Initially, small numbers are being produced while the aircraft design is being tested and modified, and operational capabilities are added. These are referred to as first operational aircraft. Once the design is stable, and it has been demonstrated that the aircraft is capable of being deployed into a hostile situation, it will be subject to further operational testing and evaluation by US authorities. Once approved, the manufacturing process will enter its full-rate production. At the time of this audit, the aircraft was still in development and many technical challenges were yet to be overcome.
2.11 The JSF program has been subject to several major technical, cost, and manufacturing reviews and has been significantly adjusted three times. Exhibit 2.1 shows changes in delivery schedule estimates of the F-35 conventional takeoff and landing variant, from the start of the system development and demonstration phase in October 2001 to April 2010. Exhibit 2.2 shows the increases in the estimated unit recurring flyaway cost for the conventional takeoff and landing variant over the full production period until 2035.
Exhibit 2.1—There have been three changes from 2001 to 2010 in estimated delivery dates for the F-35 (conventional takeoff and landing variant)
(system development and demonstration phase starts)
|Estimated delivery dates|
|First operational aircraft delivery||2008||2009||2010||2010|
This exhibit does not reflect the outcome of the United States (US) review of the Joint Strike Fighter Program.
Source: US Government Accountability Office (April 2011)
Exhibit 2.2—Estimates for the full production period of unit recurring flyaway costs increased from 2001 to 2009
* Figure is not available—awaiting outcome of the United States (US) review of the Joint Strike Fighter Program.
Amounts are in US millions of “then year” dollars, the equivalent of Canadian budget-year dollars.
Source: Joint Strike Fighter Program Office
2.12 In July 2010, US authorities began another comprehensive review of the JSF Program. This review is expected to affect the unit recurring flyaway cost, sustainment costs, as well as the development, production, and delivery schedule. At the time of this audit, partner countries were awaiting the outcome of this review. Exhibits 2.1 and 2.2 do not reflect the outcome of this review.
Canada’s fighter jet acquisition process
2.13 The federal process to procure major defence equipment is governed by legislation, policy, and departmental guidance. Together, these specify the respective roles of federal departments and agencies, the key steps to be followed, the types of information and mandatory documents required, and the approvals that must be obtained as a procurement proceeds.
2.14 In May 2008, the federal government released the Canada First Defence Strategy, which signalled its intent to replace the CF-18 fleet with 65 “next generation” fighter aircraft. In July 2010, the government announced its decision to buy 65 F-35 aircraft at a cost of CAN$9 billion, without following a competitive process. National Defence expects the first aircraft to arrive in Canada in 2019.
Focus of the audit
2.15 This audit examined whether the three main departments involved in acquiring military equipment—National Defence, Industry Canada, and Public Works and Government Services Canada (the government’s contracting authority)—applied due diligence in managing Canada’s participation in the Joint Strike Fighter Program and managing the federal decision-making process to replace the CF-18 fighter jets. We define “due diligence” to mean that the departments have, in support of key decisions and related management activities,
- performed and documented analyses (of benefits and risks, operational requirements, options, and costs);
- clarified rules, roles, and responsibilities;
- consulted with other entities; and
- obtained approvals and provided oversight.
2.16 Our criteria reflect basic principles of good management practices and due diligence, and are based on relevant Treasury Board policies that support decision making and sound stewardship, and that contribute to transparency, accountability, and value for money.
2.17 More details about the audit objectives, scope, approach, and criteria are in About the Audit at the end of this chapter.
2.18 Our recommendation is in paragraph 2.77.
Participating in the Joint Strike Fighter Program
2.19 To date, Canada has been involved in the Joint Strike Fighter (JSF) Program for almost 15 years. Officials from National Defence have contributed to all three phases, including participating in its senior decision-making and technical committees. In addition, Industry Canada, together with National Defence, made efforts to ensure Canadian companies had (and continue to have) opportunities to bid on work in connection with the aircraft development as well as eventual production and sustainment. There is no single set of federal policies or rules that govern participation in an international initiative such as the JSF Program. Exhibit 2.3 provides a timeline of key events in the JSF Program and Canada’s F-35 procurement process.
Exhibit 2.3—Timeline of key events for the Joint Strike Fighter (JSF) Program and the F-35 procurement
Source: Adapted from various National Defence, Industry Canada, Public Works and Government Services Canada, and JSF Program Office documents
2.20 National Defence signed the memorandum of understanding (MOU) for the first phase of the JSF Program, concept demonstration, in December 1997. Its US$10.6 million (budget-year dollars) contribution was made within the expenditure authority of the Minister of National Defence and did not require approval by the Treasury Board. Faced with the eventual need to replace the CF-18 fighter jets, National Defence felt its participation provided the chance to work with allies in developing a new fighter jet as well as opportunities for Canadian companies to be part of the design and eventual production of an estimated 3,100 aircraft (if Canada decided to purchase the aircraft).
2.21 In October 2001, the United States Department of Defense selected Lockheed Martin as the winner of a design competition held between two major industry consortia. National Defence signed the MOU for the second phase, system development and demonstration, in February 2002. In total, Canada committed to contributing US$150 million (budget-year dollars) for this phase; Treasury Board approved US$100 million in December 2001, which National Defence provided directly to the JSF Program. A further US$50 million was distributed to Canadian industry through pre-existing Industry Canada programs that were mandated to provide funding support to strategic research and development projects. According to National Defence, securing industrial benefits for Canadian companies continued to be a driving motivation for participation.
2.22 In 2006, the JSF Program entered its third phase: production, sustainment, and follow-on development. This marked an important transition to a regime for producing, procuring, and maintaining the F-35. In our view, 2006 also represented the most critical period concerning Canada’s participation in the JSF Program and future acquisition of the F-35.
2.23 In November 2006, Industry Canada signed industrial participation memoranda of understanding with the JSF Program’s prime contractors, the US-based companies that will manufacture the jets and engines: Lockheed Martin, Pratt & Whitney, and GE Rolls-Royce. These MOUs formalized a framework for offering opportunities to Canadian companies to be part of the supply chain to produce the jets and engines as subcontractors. Most importantly, National Defence signed the third-phase MOU in December 2006. This signing represented a significant financial commitment to the Program—up to US$551 million over 40 years, provided Canada remains a partner—to be used for, among other things, manufacturing test and tooling equipment. It also represented Canada’s acceptance of the procurement regime for the F-35. The Treasury Board approved a portion of Canada’s contribution to the JSF Program (CAN$182 million to 2013) in November 2006. It also approved CAN$10 million to support National Defence’s internal Program Office costs. As of September 2011, the government had disbursed a total of CAN$335 million toward participation in the JSF Program and support to Canadian industry. National Defence plans to seek expenditure authority for the remainder of the contribution at a later date.
2.24 According to National Defence, ongoing partnership in the JSF Program, and the eventual purchase of the F-35 through the provisions of the 2006 MOU, offered significant benefits. In addition to industrial benefits, these included
- unprecedented access to data about next generation fighter aircraft,
- cost avoidance (not having to pay for research and development costs or the fees associated with foreign military sales),
- savings on long-term sustainment costs through a collaborative approach with other partners,
- military “interoperability” (ability to work with allies and their equipment), and
- potential royalties on sales of the F-35 to non-partner countries.
2.25 In determining whether the federal government applied due diligence with respect to Canada’s participation in the JSF Program, we examined the extent to which key decisions, especially those taken in 2006, were supported by appropriate information and analyses, consultation, departmental oversight, and government approvals.
Early efforts to promote industrial participation were successful
2.26 For most defence contracts signed by the Government of Canada, the government’s Industrial Regional Benefits Policy applies. This policy seeks to ensure that defence purchases generate high-value business for Canadian industry and, in effect, guarantees work for Canadian companies, usually equivalent to the value of the acquisition contracts awarded.
2.27 For the F-35, under the provisions of the JSF Program, partners cannot require prime contractors to provide work to companies in their countries. For Canada, this fact meant that the Industrial Regional Benefits Policy could not be applied. Industrial benefits were not guaranteed; rather, companies from partner countries were, and still are, eligible to obtain contracts, provided that the companies meet the “best value” criteria (including cost, schedule, and quality) and that the country buys the F-35 as a partner in the JSF Program.
2.28 We examined how National Defence and Industry Canada carried out their respective activities and whether ministers and senior departmental decision makers were duly informed of the conditions on industrial participation established by the JSF Program.
2.29 In 2000, National Defence gained Industry Canada’s assistance in supporting the JSF Program, and together they approached the potential prime contractors to understand their requirements and to seek opportunities for Canadian industry. In early 2001, an interdepartmental JSF team was established to coordinate the federal promotion of economic opportunities for Canadian industry, trying to match Canadian capabilities with the needs of the prime contractors. A memorandum of understanding was signed by National Defence, Industry Canada, Foreign Affairs and International Trade Canada, and the Canadian Commercial Corporation; this memorandum defined roles and responsibilities for each organization in conducting outreach missions with US contractors.
2.30 We found that the collaborative arrangement among federal departments was effective. We also found early efforts to secure industrial opportunities through the system development and demonstration phase were significant and the results were successful. By 2006, National Defence estimated that Canadian companies had received 150 contracts valued at approximately CAN$157 million. We also observed that, prior to signing the various MOUs in 2002 and 2006, the ministers of Industry Canada and National Defence, and senior decision makers in both departments, were informed that industrial benefits could not be guaranteed under the JSF Program.
Projections of industrial benefits fluctuated and uncertainties were not communicated
2.31 The forecasted value of industrial benefits is significant. It has been used extensively as a basis for key decisions related to Canada’s participation in the JSF Program and the purchase of the F-35 as well as for communications to Canadians. Making forecasts is challenging and depends on, among other things, such factors as the total number of aircraft that will be produced eventually (which was unknown at the time of this audit) as well as the ability of Canadian companies to compete in a global market.
2.32 We have two concerns regarding the forecasting of industrial benefits. First, there are significant fluctuations in the estimates of contract opportunities presented by National Defence and Industry Canada to senior decision makers and ministers (Exhibit 2.4). For example, prior to signing the 2002 MOU for system development and demonstration, National Defence estimated between CAN$8 and 10 billion in projected benefits. In 2006, prior to signing the MOU for the production, sustainment, and follow-on development phase, National Defence communicated estimates to ministers that ranged from US$5.2 to 14.7 billion. In 2009, it estimated these benefits to be as high as US$16.6 billion; this estimate was reduced to US$15.4 billion leading up to the government’s 2010 decision to purchase the F-35 jets.
Exhibit 2.4—Projected benefits (in $billions) to Canadian industry, presented at key points to decision makers and ministers, have fluctuated widely
|Projected benefits for industrial participation||2001||2002||2006||2008||2009||2010|
|National Defence briefing documents||CAN$8–10||CAN$7||US$5.2
|Industry Canada briefing documents||CAN$8–10||–||US$6||US$10||–||US$12|
Source: National Defence and Industry Canada documents
2.33 Second, we have concerns about the basis of the projections of industrial benefits for Canadian companies that National Defence and Industry Canada provided to ministers. Prior to 2006, National Defence took the lead in monitoring and forecasting industrial benefits, using a database that it designed and filled with data. In 2006, responsibility for monitoring and reporting was taken over by the prime contractors, in accordance with the industrial participation MOUs they signed with Industry Canada. Projections made by the prime contractors were (and continue to be) extrapolated over the entire production period, and were based on a combination of opportunities related to
- contracts that were already awarded in the system development and demonstration phase,
- potential contract opportunities that are offered exclusively to Canadian companies, and
- potential contract opportunities that are available through competition to companies from partner countries.
This third category represents the majority of the value of available opportunities, but these are the least certain, since Canadian companies must compete against companies from other partner countries.
2.34 Since 2006, Industry Canada and National Defence used these projections as the basis of information provided to decision makers and ministers. These projections were not independently validated by federal departments, and in fact, this validation was difficult to do, since the data is largely proprietary. In 2010, the combination of estimated contract opportunities totalled US$12 billion. National Defence added to these projections its own estimates of benefits that may accrue to Canadian companies as a result of assumed expenditures on local sustainment over 20 years. These estimates were based on assumptions about the type of work that may be performed in Canada and how much of it National Defence may direct to Canadian companies. These estimates are not related to the industrial participation opportunities provided by the prime contractors. In 2010, the benefits related to sustainment were estimated to be US$3.4 billion.
2.35 We found that briefing materials prepared by the departments for decision makers and ministers did not explain the basis for the projections, or the consequent limitations involved in relying on those projections for decision making. Moreover, in the majority of cases, only the most optimistic scenario was put forward, rather than a range of potential benefits that reflected the inherent uncertainties in the projections. We are concerned, because these projections were used to support key decisions related to Canada’s participation in the JSF Program and the purchase of the F-35 aircraft.
Preliminary analysis was conducted to consider various fighter jet options
2.36 The 2006 MOU for the third phase of the JSF Program (production, sustainment, and follow-on development) does not commit partner countries to purchase the F-35 aircraft. Nonetheless, in our view, sound management practices would have required that National Defence assure itself that the F-35 was a suitable, if not preferable, aircraft before further committing Canada and Canadian industry to the JSF Program. We therefore examined departmental activities in relation to identifying operational requirements and assessing how available options would meet those requirements.
2.37 These activities took place in 2005, when National Defence began an analysis of potential aircraft to replace the CF-18s. It defined preliminary operational requirements for a replacement and assessed five candidate aircraft against them, based in part on information obtained from site visits to various aircraft manufacturers. Four of these candidates were existing aircraft; the fifth, the F-35, was still being developed. In June 2006, National Defence summarized this options analysis in the Operational Requirements Concept Document (ORCD). It concluded with a strong preference for the F-35, stating, “It is not only the aircraft that best meets Canadian Forces’ requirements, with the longest life expectancy, but also is the most affordable.”
2.38 The ORCD did not assess the number of aircraft required to meet mission expectations and commitments. National Defence provided the JSF Program Office with an estimated procurement number of 80 aircraft, which was included in the 2006 MOU and was the basis for calculating Canada’s financial contributions. This number of aircraft was an estimate based on the approximate number of modernized CF-18s existing at the time and was not based on an assessment of future operational needs.
2.39 The ORCD conclusions were presented to senior officials within National Defence, including the Chief of the Air Staff and the Assistant Deputy Minister (Materiel). The conclusions were approved by the Chief of the Air Staff. These conclusions formed part of the basis upon which National Defence recommended to the Minister that the 2006 MOU be signed. In our opinion, this was an important activity that helped to support the government’s decision to sign the MOU.
Risks associated with participating in the Joint Strike Fighter Program were communicated to decision makers
2.40 Treasury Board policies recognize that risk is an important element in decision making and needs to be identified, assessed, and communicated at key decision points. The policies also require measures to mitigate risks. We examined whether National Defence identified, assessed, and communicated risks related to Canada’s participation in the JSF Program before seeking expenditure authority from the Treasury Board. The key decision points were when National Defence sought expenditure authority for Canada’s financial contributions to the second and third phases of the JSF Program. We also examined the strategies that National Defence developed to mitigate identified risks.
2.41 We found that National Defence identified and communicated risks and developed mitigation strategies for participating in the JSF Program. In 2001, prior to signing the MOU for the second phase of the JSF Program (system development and demonstration), risks such as the JSF Program being cancelled or running over budget were presented to senior decision makers in National Defence. The Department identified strategies to mitigate these risks, including imposing a ceiling for Canada’s financial contributions and negotiating provisions in that MOU that would allow Canada to withdraw from the JSF Program. National Defence identified and communicated similar risks and risk mitigation strategies prior to signing the 2006 MOU for the production, sustainment, and follow-on development phase.
2.42 However, there is no documentation indicating how the risks were determined and analyzed. Consequently, we have no basis to determine if the risk assessments are complete and appropriate under the circumstances.
There was a lack of consultation and communication on the procurement regime
2.43 The 2006 MOU for the third phase of the JSF Program (production, sustainment, and follow-on development) prescribes, for partners, the regime for procuring the F-35. This regime has implications for whether and how Public Works and Government Services Canada (PWGSC) carries out its role as the contracting authority for large defence procurements. Therefore, we believe National Defence should have engaged PWGSC and central agencies, before and after signing the 2006 MOU, to map out a strategy and process for obtaining government approvals and managing applicable procurement rules.
2.44 This did not happen. While negotiating and finalizing the 2006 MOU, National Defence did not consult PWGSC to determine whether and how the procurement process outlined in the MOU mechanism could be managed in accordance with Canadian laws and policies. The reason is that National Defence did not consider this situation to be a procurement; rather, it was an initiative to gain industrial benefits and obtain technological transfer. PWGSC was not given a copy of the 2006 MOU until December 2009.
2.45 We examined what ministers were told in briefing materials and submissions to government related to the decision whether to sign the 2006 MOU. We found that the ministers of National Defence and Industry Canada and those ministers on the Treasury Board were not fully informed in the materials about the procurement implications of the 2006 MOU. Briefing materials we have reviewed describe the features and benefits of Canada’s participation in the JSF Program, with particular emphasis on industrial benefits. While ministers were told, correctly, that signing the 2006 MOU did not commit Canada to buy the F-35, we did not see evidence they were told that retaining industrial benefits depended on buying the F-35 as a partner in the JSF Program. Also, while ministers were told that the 2006 MOU did not prevent Canada from having a competition in the future, they were not told of the practical limitations of doing so. For example, as a partner in the development of the F-35, National Defence’s long-standing relationship with and access to proprietary data from one of the prime contractors, coupled with the unique benefits offered only to partners, meant that other potential aircraft manufacturers would be disadvantaged from competing fairly.
2.46 In summary, we found that National Defence, as the lead department, took the appropriate steps in managing Canada’s participation in the Joint Strike Fighter (JSF) Program. The Department managed industrial participation well (together with Industry Canada), identified and communicated risks and mitigation strategies related to participating in the Program, and assessed options before signing the 2006 MOU. However, National Defence did not fully inform decision makers of the implications of participation in the JSF Program for the acquisition process. In some cases, documented analysis did not exist to support decisions.
Procuring new fighter jets
2.47 As noted in the introduction, the federal process to procure defence equipment and to support decision making reflects legislation, policy, and departmental guidance. National Defence’s formal project management process, in place during the period of this audit, is depicted in Exhibit 2.5. (This process was revised in September 2011.) National Defence is responsible for leading project management activities. Public Works and Government Services Canada (PWGSC) plays a critical role in contracting activities. We examined whether the departments applied due diligence in managing the replacement of the CF-18 fighter jets, called the Next Generation Fighter Capability project.
Exhibit 2.5—Key phases and steps in project management
Source: Adapted from the project management process used within National Defence
2.48 Within National Defence, a capital project officially starts after it is formally designated as such, at the end of the Project Identification Phase. For the fighter jets, this occurred in January 2010, when the Next Generation Fighter Capability project was approved internally and the formal Project Options Analysis Phase was started. That said, many of the types of activities required under the formal process—such as identifying operational requirements and analyzing options, assessing risks, and estimating costs—had already been under way for years. At the end of the Project Options Analysis Phase, National Defence typically seeks formal project approval from the Treasury Board. At the time of this audit, National Defence expected to do so in 2012.
Key decision-making steps and documents were of little consequence
2.49 By the end of 2006, officials from National Defence and the Canadian Forces were actively involved in the development of the F-35. It was clearly the fighter jet of choice, and, in our view, a number of activities and decisions had put in motion its eventual procurement. By that time, National Defence had
- completed a preliminary options analysis of contender aircraft and concluded in favour of the F-35,
- signed three memoranda of understanding (MOUs) with its military allies,
- assigned personnel to work full-time in the Joint Strike Fighter (JSF) Program Office and participated in the decision-making structure,
- helped negotiate industrial participation memoranda with the F-35’s prime contractors (which were signed by Industry Canada),
- successfully supported Canadian industry’s involvement in the initial manufacturing (knowing that retaining industrial benefits was contingent on buying the aircraft), and
- committed about US$710 million to the JSF Program (based on a projected purchase of 80 aircraft) and disbursed CAN$176 million through contributions and support to industry.
2.50 Despite the above, a decision by the Government of Canada to procure new fighter jets to replace the CF-18 was still required. This eventually happened in July 2010, when the government announced its decision to buy 65 F-35 aircraft. As described in the following paragraphs, we observed that in the lead-up to this announcement, required documents were prepared and key steps were taken out of sequence. Key decisions were made without required approvals or supporting documentation. We believe that National Defence ought to have treated procurement of the F-35 as a project in 2006, or shortly thereafter, and established an appropriate management plan. Doing this would have provided a framework for planning subsequent activities, supporting key decisions, and obtaining approvals in a timely manner.
2.51 Government policy aims to ensure that goods and services are acquired in a manner that enhances access, competition, and fairness. It requires competition but permits a non-competitive (sole-source) purchase if one of four conditions exists, as set out in the Government Contracts Regulations. An important step in the procurement process is the selection of a procurement strategy, essentially a decision whether to run a competition or not. The use of an exception to competition must be fully justified and is typically supported by a formal statement of operational requirement and options analysis. It is the responsibility of Public Works and Government Services Canada (PWGSC) to approve a procurement strategy and, more broadly, to uphold the integrity of the procurement process by ensuring that it adheres to all relevant legislation and policy. We examined how National Defence and PWGSC managed activities supporting the government’s 2010 decision to buy the F-35.
2.52 In May 2008, following the release of the Canada First Defence Strategy, National Defence identified 14 draft high-level mandatory capabilities for the replacement of the CF-18s. These were very broad and qualitative in nature and were a precursor to the formal statement of operational requirement. In 2008, two departmental boards—National Defence’s Capability Development Board and the Joint Capability Requirement Board—endorsed these draft high-level mandatory capabilities.
2.53 Also in 2008, National Defence undertook an options analysis of three contender aircraft against the high-level mandatory capabilities. These included the F-35, which was still under development at the time. This analysis concluded that, while all three aircraft could meet the high-level mandatory capabilities, the F-35 offered the “best value” because it provided “exceptional capability at the lowest cost and unparalleled benefits for the Canadian aerospace industry.” This analysis became pivotal to the decision-making process. The conclusions were presented to senior decision makers. There was no documentation supporting the analysis and conclusions.
2.54 Between late 2008 and mid-2009, National Defence led a process to get a government decision to buy the F-35, partly in response to pressure from industry to commit to buy the F-35 in order to keep and increase industrial benefits. Senior departmental decision makers and ministers were advised that a delay in this decision would lead to possible losses of billions of dollars in potential contracts for Canadian companies. Based on this, and on the conclusions of the 2008 options analysis, National Defence recommended to central agencies, other government departments, and its Minister that Canada commit to buying the F-35 through the provisions of the 2006 MOU for the third phase of the JSF Program, without competition. National Defence recognized at the time that holding a balanced competition among contender aircraft would be “exceedingly difficult,” in part because of Canada’s long-standing partnership in the development of the F-35. Ultimately, this decision process was put on hold, but for reasons unrelated to holding a competition.
2.55 In early 2010, National Defence restarted the process to obtain a government decision to buy the F-35. By this time, the Next Generation Fighter Capability Project had officially started and the Project Options Analysis Phase (Exhibit 2.5) was under way. It was at this time that PWGSC became actively involved in the decision-making process.
2.56 Since National Defence intended to recommend the purchase of the F-35 without competition, it had to identify and justify an appropriate exception to competitive tendering set out in the Government Contracts Regulations. One of the permitted exceptions is that “the nature of the work is such that it would not be in the public interest to solicit bids.” Up until late May 2010, National Defence planned to use this exception, based in part on the conclusions from the 2008 options analysis and on its argument that it needed a fighter jet with fifth generation capability. While this exception was supported by PWGSC, ultimately the use of this exception was not supported inter-departmentally. National Defence then decided to use another of the permitted exceptions, that “only one person [that is, contractor(s)] is capable of performing the contract,” and the decision-making documents were reworded accordingly.
2.57 To support the use of this exception, National Defence was required to identify its operational requirements and to provide a full justification to PWGSC. Neither was provided to PWGSC in a timely manner, despite several requests from PWGSC. PWGSC was not given a copy of the statement of operational requirement until August 2010, well after the government had announced its decision to purchase the F-35 in July 2010. In fact, the statement was approved by the Chief of the Air Staff after decision-making documents related to the government’s July 2010 announcement had been signed and submitted for consideration by ministers. Then, according to National Defence, the formal options analysis was completed (as required as part of the Project Options Analysis Phase as shown in Exhibit 2.5), which concluded that the F-35 was the only available aircraft that could meet the mandatory requirements of the Canadian Forces. The conclusion was cited as the basis for the government’s decision to purchase the F-35 without competition.
2.58 While decision-making documents were being finalized in late May 2010, PWGSC questioned National Defence’s assertion that no other aircraft met its mandatory requirements. Senior decision makers in PWGSC were informed that PWGSC had not been provided sufficient justification to support National Defence’s proposed procurement strategy and assertion that only one company was capable of performing the contract. In lieu of a finalized statement of operational requirement or a completed options analysis, PWGSC informed National Defence on 1 June 2010 that it would endorse the sole-source justification if National Defence provided a letter confirming National Defence’s requirement for a fifth generation fighter and confirming that the F-35 is the only such aircraft available. The same day, National Defence provided such a letter. There were no other supporting documents. By this time, decision-making documents had already been signed in both National Defence and PWGSC. It is important to note that the term “fifth generation” is not a description of an operational requirement.
2.59 We found that National Defence engaged PWGSC late in the decision-making process and hampered PWGSC’s ability to carry out its responsibilities as contracting authority to ensure the integrity of the procurement process. At the same time, PWGSC relied almost exclusively on assertions by National Defence and endorsed the procurement strategy in the absence of required documentation and completed analysis. We believe this has compromised an important control in the procurement process: independent validation of a proposed procurement strategy.
2.60 In our view, many of the steps and documents used to support the government’s 2010 decision were of little consequence, because the key questions of whether to procure the F-35 and whether to run a competition were effectively determined by decisions taken much earlier, calling into question the integrity of the process. Not only were they of little consequence, they might also have been unnecessary if National Defence had sought government approval at an earlier stage to be completely exempt from the requirement to fit the procurement into one of the specified exceptions to competitive tendering. Practically speaking, by 2010, Canada was too involved in the JSF Program and the F-35 to run a fair competition.
Risks and mitigating strategy did not reflect cost increases and delays
2.61 Treasury Board policies require risks to be identified and communicated to decision makers at key decision points and to be reassessed periodically throughout the life of the project. Our earlier findings focused on risks associated with Canada’s participation in the JSF Program. This section addresses how National Defence identified and communicated the risks associated with Canada’s reliance on the JSF Program and the F-35 to replace the CF-18. Such risks relate to potential schedule slippage, cost increases, and technical challenges that may affect fulfillment of the mandatory operational requirements. We reviewed the briefing materials that were provided to various senior departmental committees and the Minister of National Defence between 2006 and 2010. We also examined the strategies developed to mitigate identified risks. It is important to note that the management of cost, schedule, and technical risks associated with the development and production of the F-35 is the responsibility of the US government. National Defence relies on the JSF Program Office to obtain information about such risks.
2.62 We have concerns with the manner in which National Defence identified and communicated risks. As noted in the introduction to this chapter, the JSF Program has experienced cost increases, schedule delays, and technological difficulties, and has been subject to several major reviews. Officials from National Defence who participated in the senior decision-making committees of the JSF Program were regularly informed of these problems. Yet in briefing materials from 2006 through 2010 that we have reviewed, neither the Minister nor decision makers in National Defence and central agencies were kept informed of these problems and the associated risks of relying on the F-35 to replace the CF-18. In 2009, in presentations to central agencies and the interdepartmental community, National Defence identified the risk of potential cost increases but asserted that costs had stabilized and that all major outstanding cost issues had been resolved. In 2010, when the JSF Program was undergoing a comprehensive review of cost, schedule, and technology development due to ongoing problems, neither the Senior Review Board nor the Minister was informed of these problems in briefing materials. According to National Defence, until 2010, when the decision was made to procure the F-35, none of the cost, schedule, or technical risks associated with the JSF Program involved a significant impact to Canada.
2.63 When communicating risk information to departmental decision makers, National Defence provided an overall risk rating for replacing the CF-18 with the F-35. The Senior Review Board assessed this rating as “medium” prior to the July 2010 announcement. This rating is lower than previous ratings provided to decision makers in 2008 and was, in our opinion, inconsistent with the problems the JSF Program was experiencing at the time. There was no documented analysis to determine how the risks were assessed and rated.
2.64 With respect to mitigating risks associated with potential cost increases, schedule delays, and technical challenges, since 2006, National Defence has consistently taken the position that it would make its purchase at the ideal time of the production cycle. That is, it would do so when economies of scale are at their highest, prices are at their lowest, and all technical difficulties have been resolved. This mitigation strategy has not changed since 2006, despite the fact that the system development and demonstration phase was extended to 2018 and full-rate production is now expected to start in 2018. In February 2010, National Defence received formal communication from the US Under Secretary of Defense for Acquisition, Technology and Logistics that developing the F-35 would cost more and take longer to finish than planned and that the US Department of Defense was reassessing its cost projections.
2.65 In our opinion, National Defence has been overly confident about potential mitigation strategies, especially given that the F-35 is still under development and that the CF-18 is expected to reach the end of its life between 2017 and 2020. The transition to the F-35 is expected to start in 2017 and be completed in 2020, at which time the CF-18 fleet would be retired and no longer supported.
Full life-cycle costs were not presented and were likely underestimated
2.66 Treasury Board policies require consideration of all relevant costs over the useful life of equipment, not just the initial acquisition or basic contract cost. Careful planning and full costing are needed to ensure that all of the elements required to provide the needed capability come together in a timely and predictable way and that adequate funds are available to support the equipment over the long term. We examined whether National Defence conducted full life-cycle costing related to its Next Generation Fighter Capability project and whether cost estimates were complete, supported, and validated, using the best information available at the time. Estimating future full life-cycle costs for military equipment, especially the F-35, is challenging.
2.67 To determine the unit price of the F-35, Canada relies in part on data generated by the US government. A Selected Acquisition Report is presented annually to the US Congress. Once approved, the JSF Program Office produces for Canada a “bilateral cost breakdown.” This breakdown contains the predicted unit recurring flyaway cost, based on the number of aircraft Canada plans to purchase and when it plans to do so. Based on the 2009 bilateral cost breakdown, the predicted average unit recurring flyaway cost for Canada, for 80 conventional takeoff and landing variant aircraft, was US$75 million. To this cost, National Defence must add the predicted cost of Canadian modifications to the basic variant, which include adding drag chutes for landing on short runways and adding a different system for refuelling in the air. The JSF Program Office also provides National Defence with the predicted sustainment cost of the F-35 conventional takeoff and landing variant. National Defence reviews the cost breakdown for consistency with its planning assumptions. Following the government’s 2010 announcement, National Defence informed the US Joint Strike Fighter Program Office of its plan to reduce the purchase to 65 aircraft.
2.68 Exhibit 2.6 illustrates two cost estimates developed by National Defence: the first was used for decision making in June 2010; the second was presented to Parliament in response to the March 2011 report of the Parliamentary Budget Officer. It is important to note that some of these costs are specific to the F-35 (such as capital acquisition and sustainment), whereas other costs are more broadly associated with fighter jet capability (such as personnel).
Exhibit 2.6—National Defence’s estimated 20-year costs for the F-35 have varied
|Elements related to the purchase of the F-35 (in millions of Canadian budget-year dollars)||National Defence’s estimates used for decision making
|National Defence’s public response to Parliamentary Budget Officer’s report
|Capital acquisition costs—aircraft||Aircraft||5,580||–|
|Total capital costs for 65 F-35s||6,000||6,000|
|Additional capital acquisition costs||Initial logistics and training (including simulators)||1,320||1,300|
|Project management (initial)||160||200|
|Weapons (initial buy)||270||300|
|Total additional capital acquisition costs||2,980||3,000|
|Total capital acquisition costs||8,980||9,000|
|Personnel, operating, and maintenance costs||Contracted sustainment||5,710||5,700|
|National Defence personnel||4,740||–|
|Total personnel, operating, and maintenance costs||16,140||5,700|
|Total 20-year costs||25,120||14,700|
Source: National Defence
2.69 To date, there have been two key announcements regarding the budget for replacing the CF-18s: the May 2008 Canada First Defence Strategy and the July 2010 announcement of the F-35 purchase. The Canada First Defence Strategy established a budget of CAN$9 billion to acquire 65 next generation fighter aircraft. This budget figure was subsequently included in National Defence’s 2009 Investment Plan. It was also carried forward into decision documents to support the July 2010 announcement. Also in 2008, a budget of CAN$16 billion was established to operate and sustain the F-35 over 20 years. These budgets were based on estimates provided by National Defence, yet there is no documented analysis to show how they were developed. In 2010, National Defence informed decision makers that the costs of operating and sustaining the F-35 would be covered by existing funds.
2.70 These budgets have since been treated as a maximum by National Defence, yet many decisions that could affect the ultimate costs are still to be taken. Moreover, as noted in paragraphs 2.11 and 2.12, United States’ estimates of the future purchase price of the F-35 are in flux. Estimates for sustainment costs are not fully developed. Although the budgets established for the acquisition (CAN$9 billion) and for operations and sustainment (CAN$16 billion) include provision for contingency, there is a risk that these budgets may not be sufficient.
2.71 We have a number of observations regarding the life-cycle costing for the F-35. First, costs have not been fully presented in relation to the life of the aircraft. The estimated life expectancy of the F-35 is about 8,000 flying hours, or about 36 years based on predicted usage. National Defence plans to operate the fleet for at least that long. It is able to estimate costs over 36 years. We recognize that long-term estimates are highly sensitive to assumptions about future costs as well as to currency exchange rates. However, in presenting costs to government decision makers and to Parliament, National Defence estimated life-cycle costs over 20 years. This practice understates operating, personnel, and sustainment costs, as well as some capital costs, because the time period is shorter than the aircraft’s estimated life expectancy. The JSF Program Office provided National Defence with projected sustainment costs over 36 years.
2.72 Second, the following expected costs were not accounted for:
- Replacement aircraft. National Defence considers 65 aircraft the minimum number needed to meet its training and operational requirements. Based on past experience, National Defence expects to lose aircraft in the course of normal usage. Based on National Defence’s assumed attrition rate, in order to maintain the fleet of 65 aircraft, Canada may need to purchase up to 14 additional aircraft over the next 36 years. National Defence did inform the government of the need to consider the requirement for attrition aircraft at a later date. The cost of replacement aircraft is not included in the life-cycle estimate for this project and will be treated as a separate project in the future.
- Upgrades. It is expected that over the life of the aircraft, Canada will need to invest in various upgrades to the F-35 fleet, both in software and hardware. These costs were not known when the 2008 and 2010 budgets were established, but have since been estimated to be more than CAN$1.2 billion over 20 years.
- Weapons. National Defence has currently allocated enough weapons for an initial stockpile to last for 45 days of conflict operations. National Defence assumes that it will use existing weapons from the CF-18s to fulfill some of its needs. It will absorb future weapons purchases from its operating budget, and the purchase of new weapons during the fleet’s life cycle will be addressed through future acquisition projects.
2.73 Third, many costs are not yet reliably known or cannot yet be estimated. These include the basic unit recurring flyaway cost of the aircraft, the cost of Canadian required modifications, and the cost of sustainment. In addition, National Defence is still developing its planning assumptions for operating the fleet. This involves hundreds of interrelated decisions about such matters as how pilot and technician training will be delivered, what physical infrastructure is required and what portion is directly attributable to the F-35, how maintenance and repair activities will be supported, and what they will cost. National Defence currently assumes that average annual maintenance and repair costs for the F-35 fleet will be the same as for the CF-18 fleet. At the time of this audit, the project had not yet entered the formal Project Definition Phase, when many of these decisions will be taken and the cost implications will be better known. Consequently, estimates of the full life-cycle costs for the F-35 will likely change as the basis for the estimates becomes firmer.
2.74 Also unknown are the cost implications if further delays in the delivery of the F-35 require National Defence to make significant investments to extend the operation and life of the CF-18 fleet.
2.75 We believe that establishing a budgetary cap without the aid of more complete and reliable cost estimates creates significant risks. National Defence may have to abandon or reduce some of the Canadian Force’s capabilities, for example, by reducing the number of aircraft purchased or their flying hours. Alternatively, it may need to seek additional funds from the government or use funds from other parts of its capital or operating budgets. This could make it difficult to determine the complete life-cycle costs associated with the F-35 fighter jet capability.
2.76 We also have significant concerns about the completeness of cost information provided to parliamentarians. In March 2011, National Defence responded publicly to the Parliamentary Budget Officer’s report. This response did not include estimated operating, personnel, or ongoing training costs (Exhibit 2.6). Also, we observed that National Defence told parliamentarians that cost data provided by US authorities had been validated by US experts and partner countries, which was not accurate at the time. At the time of its response, National Defence knew the costs were likely to increase but did not so inform parliamentarians.
2.77 Recommendation. National Defence should refine its estimates for complete costs related to the full life cycle of the F-35 capability, and provide complete estimated costs and the supporting assumptions as soon as possible. Furthermore, National Defence should regularly provide the actual complete costs incurred throughout the full life cycle of the F-35 capability.
The Department’s response. Agreed. National Defence will continue to refine its full life-cycle cost estimates for the F-35 capability and commits to making the estimates and actual costs of the F-35 available to the public.
2.78 The Joint Strike Fighter (JSF) Program is unique. In this context, National Defence, as the lead department, exercised due diligence in managing Canada’s participation in the Program. National Defence managed industrial participation well (together with Industry Canada), identified and communicated risks and mitigation strategies related to JSF Program participation, and assessed options before signing the 2006 memorandum of understanding (MOU), committing Canada to the third phase of the JSF Program (production, sustainment, and follow-on development.) However, National Defence did not fully inform decision makers of the implications of participation in the JSF Program for the acquisition process. In some cases, documented analysis did not exist to support decisions.
2.79 Industry Canada exercised due diligence in managing Canada’s industrial participation in the JSF Program. In partnership with National Defence, Industry Canada worked to secure industrial participation.
2.80 National Defence did not exercise due diligence in managing the process to replace the CF-18 jets. National Defence did not appropriately consult Public Works and Government Services Canada (PWGSC) on the procurement implications of the 2006 MOU for the third phase of the JSF Program or develop an appropriate plan for managing the unique aspects of the acquisition. Problems relating to development of the F-35 were not fully communicated to decision makers, and risks presented to decision makers did not reflect the problems the JSF Program was experiencing at the time. Full life-cycle costs were understated in the estimates provided to support the government’s 2010 decision to buy the F-35. Some costs were not fully provided to parliamentarians. There was a lack of timely and complete documentation to support the procurement strategy decision.
2.81 PWGSC did not demonstrate due diligence in its role as the government’s procurement authority. Although it was not engaged by National Defence until late in the decision-making process, PWGSC relied almost exclusively on assertions by National Defence and endorsed the sole-source procurement strategy in the absence of required documentation and completed analysis.
2.82 Both National Defence and Public Works and Government Services Canada disagree with our conclusion that they did not demonstrate due diligence in their respective roles in the replacement of the CF-18 jets. The departments believe that the level of due diligence was appropriate within the time frame covered by this audit.
2.83 Procuring developmental equipment can bring unique risks and challenges. In our opinion, the experience with the acquisition of the F-35 has potential lessons for development and acquisition of other military equipment. In this context, while National Defence did several things well, we have described several concerns in the chapter. We do not believe a recommendation based on these concerns is required, given that best practices and policies governing these areas are sufficient.
About the Audit
All of the audit work in this chapter was conducted in accordance with the standards for assurance engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these standards as the minimum requirement for our audits, we also draw upon the standards and practices of other disciplines.
The audit objectives were to determine whether
- National Defence and Industry Canada applied due diligence in managing Canada’s participation in the United States (US)-led Joint Strike Fighter (JSF) Program, and
- National Defence and Public Works and Government Services Canada (PWGSC) applied due diligence in managing the Canadian Next Generation Fighter Capability project for the replacement of the CF-18 fighter jets.
Scope and approach
The audit was divided into three lines of enquiry: one for National Defence and one for Industry Canada on Canada’s participation in the US-led JSF Program; and the third line of enquiry for National Defence’s and PWGSC’s management of the Canadian Next Generation Fighter Capability project. Each line of enquiry examined the extent to which the departments applied due diligence in managing their respective responsibilities. For National Defence, these responsibilities relate largely to the JSF Program and project management, and for Industry Canada, to industrial participation. We also examined the role of PWGSC in relation to its responsibilities for procurement.
In addition to our audit examination work, we obtained background information to better understand the history of the JSF Program and to provide context for Canada’s participation in it.
The audit examined documents and correspondence contained in National Defence’s program, project, and payment files; Industry Canada’s industrial participation and contribution files; and PWGSC’s procurement files. We conducted interviews with individuals who are currently or were formerly involved in the management of the JSF Program. We interviewed officials of the JSF Program Office, the Cost Assessment and Program Evaluation directorate at the US Department of Defense, the US Government Accountability Office, and Lockheed Martin. We visited the Canadian Forces’ Wing in Cold Lake, Alberta, one location of the CF-18 fleet.
We have chosen the phrase “applied due diligence” to describe our expectation and have drawn on a number of sources (acts and regulations and departmental policies and guides) that address requirements for effective decision making, sound stewardship, and value for money.
We did not examine certain decisions that were made, because they are policy or are beyond our technical capability. We did not audit the merits of the F-35 aircraft.
We also did not audit the JSF Program or the activities of the international partners.
It should be noted that our conclusions about management practices and actions apply only to those of public servants in the federal government. The rules and regulations we refer to apply to public servants; they do not apply to contractors. We did not audit the records of the private sector contractors. Consequently, our conclusions cannot and do not pertain to the contractors’ practices or their performance.
Our criteria reflect basic principles of good management practices and due diligence, and are based on relevant Treasury Board policies that support decision making and sound stewardship and contribute to transparency, accountability, and value for money.
|To determine whether National Defence and Industry Canada applied due diligence in managing Canada’s participation in the United States-led Joint Strike Fighter Program and whether National Defence and Public Works and Government Services Canada applied due diligence in managing the Next Generation Fighter Capability project for the replacement of the CF-18 fighter jets, we used the following criteria:|
National Defence has identified, assessed, and managed risks to support the decision making related to the Canadian participation in the Joint Strike Fighter (JSF) Program. (Sources: 10, 16, 17, 23)
National Defence has a governance and management approach to carry out its activities, within its mandate and authorities, and applied it to the Canadian participation in the JSF Program. (Sources: 1, 2, 3, 4, 5, 10, 13, 15, 16, 18, 19, 21, 24, 25)
National Defence has carried out and sought appropriate oversight and approvals at key decision points related to the Canadian participation of the JSF Program. (Sources: 4, 10, 13, 18, 24, 25)
National Defence knows its contributions are being used for the purposes intended, in accordance with the terms and conditions of the 2002 system development and demonstration memorandum of understanding and the 2006 production, sustainment, and follow-on development memorandum of understanding. (Sources: 4, 10, 14, 15)
Industry Canada has identified, assessed, and managed risks and benefits related to the Canadian industrial participation in the JSF Program. (Sources: 8, 10, 16, 17, 19)
Industry Canada has a governance and management approach to carry out its activities, within its mandate and authorities, and applied it to the Canadian industrial participation in the JSF Program. (Sources: 3, 10, 11, 13, 15, 18)
Industry Canada has carried out and sought appropriate oversight and approvals at key decision points related to the Canadian industrial participation in the JSF Program. (Sources: 10, 11, 13)
Industry Canada knows that it has met its contribution commitments in accordance with the terms and conditions of the 2002 system development and demonstration memorandum of understanding and the 2006 production, sustainment, and follow-on development memorandum of understanding. (Sources: 3, 10, 11, 15, 18)
National Defence and Public Works and Government Services Canada have identified, assessed, and managed key activities of project management to support the decision making related to the Next Generation Fighter Capability project for the replacement of the CF-18 fighter jets. (Sources: 2, 5, 6, 7, 8, 9, 10, 12, 13, 17, 19, 20, 23, 24)
National Defence has a governance and management approach to carry out its activities, within its mandate and authorities, and applied it to the Next Generation Fighter Capability project for the replacement of the CF-18 fighter jets. (Sources: 1, 3, 9, 10, 12, 13, 15, 16, 18, 19, 21)
National Defence and Public Works and Government Services Canada have carried out and sought appropriate oversight and approvals at key decision points related to the Next Generation Fighter Capability project for the replacement of the CF-18 fighter jets. (Sources: 8, 10, 13, 24)
Management reviewed and accepted the suitability of the criteria used in the audit.
Period covered by the audit
Although Canada’s participation in the United States-led Joint Strike Fighter Program began in 1997, the activities we audited occurred between January 2001 and July 2010.
Audit work for this chapter was substantially completed on 30 September 2011.
Assistant Auditor General: Jerome Berthelette
Principal: John Reed
Director: Joyce Ku
For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free).
The following recommendation is found in Chapter 2. The number in front of the recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses indicate the paragraphs where the topic is discussed.
|Procuring new fighter jets|
2.77 National Defence should refine its estimates for complete costs related to the full life cycle of the F-35 capability, and provide complete estimated costs and the supporting assumptions as soon as possible. Furthermore, National Defence should regularly provide the actual complete costs incurred throughout the full life cycle of the F-35 capability. (2.47–2.76)
Agreed. National Defence will continue to refine its full life-cycle cost estimates for the F-35 capability and commits to making the estimates and actual costs of the F-35 available to the public.
Sustainment—Aircraft maintenance and repair, provision of spare parts, technological upgrades, training, and other operating activities. (Return)
First operational aircraft—A production jet that does not contain specialized equipment required for operational testing. This is in contrast to a development jet that is equipped with specialized equipment to monitor the jet’s performance during testing. (Return)
Unit recurring flyaway cost—The cost of an aircraft’s airframe, vehicle and mission systems, and propulsion. (Return)
Budget-year dollars—Costs in budget-year dollars reflect the purchasing power of the dollar in the year the cost is incurred. This is in contrast to costs in constant-year dollars, which reflect prior year, current, and future costs at the level of prices of a defined base year. (Return)
Capability Development Board—A board that assists the Chief of Force Development in formulating decisions, direction, and guidance pertaining to central force development. (Return)
Joint Capability Requirement Board—A board that reviews and endorses technical and operational requirements of projects. (Return)
Fifth generation capability—Fighter jets that, according to manufacturers, incorporate the most modern technologies, such as stealth, advanced radar, and integrated avionics. There is no accepted or objective definition of fifth generation capability. (Return)
Senior Review Board—A board that is established for each large project to provide rigorous examination of and challenge to a project as well as ongoing review and oversight. (Return)