​Evaluation of Strategic Outcome: Current Government Information is Managed to Support Government Accountability From 2013–2014 to 2016–2017

Evaluation Function
Corporate Planning and Accountability

October 2018

Table of contents

• Summary
• 1. Introduction
  • 1.1 Purpose of the evaluation
• 2. Library and Archives Canada and Program profiles
  • 2.1 Overview of Library and Archives Canada (LAC)
  • 2.2 Program overview
  • 2.3 Resources
  • 2.4 Program governance
  • 2.5 New approach
  • 2.6 Expected outcomes
  • 2.7 LAC's priorities related to the Program from 2013–2014 to 2016–2017
  • 2.8 Legislative context
• 3. Methodology
  • 3.1 Evaluation period
  • 3.2 Evaluation questions and methods
  • 3.3 Limitations of the evaluation
  • 3.4 Coding of findings
• 4. Findings
  • 4.1 Was the performance measurement strategy implemented?
  • 4.2 To what extent were the immediate and intermediate outcomes achieved?
  • 4.3 Were other ways of doing things considered to improve the achievement of outcomes?
  • 4.4. Was the recommendation of the Auditor General of Canada regarding the disposition authorities successfully met?
• 5. Conclusions and recommendations
  • 5.1 Conclusions
  • 5.2 Recommendations
• Appendix A: Management Response and Action Plan
• Appendix B: Logic Model
• Appendix C: List of Federal Organizations Subject to the LAC Act
• Appendix D: Performance Measurement Strategy
• Appendix E: Bibliography

Summary

Introduction

This report presents the findings and recommendations of the Evaluation of Strategic Outcome: "Current Government Information is Managed to Support Government Accountability." The evaluation was conducted by the Program Evaluation Function of Library and Archives Canada's (LAC) Corporate Planning and Accountability Directorate, and it is in compliance with the directives of the Government of Canada's Policy on Results. ^endnote1 Its primary objective was to review progress made in achieving the outcomes of the Program. The evaluation covered a four-year period, from 2013–2014 to 2016–2017.

Program overview

According to the Program Alignment Architecture (PAA), Strategic Outcome 1.0 – "Current Government Information is Managed to Support Government Accountability" – includes two activities: namely, activity 1 "Development of regulatory instruments and recordkeeping tools" and activity 2 "Collaboration in the management of government records." Through these two activities, the Program ensures that information management and recordkeeping practices in all Government of Canada organizations are consistent and effective and that they comply with recordkeeping directives and other policies.

Evaluation methods

The Program's administrative and financial documentation, performance statistics and other internal documents were consulted. A total of 23 interviews were conducted with managers and staff members involved in the management and delivery of the Program. In addition, 14 interviews were conducted with federal organizations subject to the LAC Act. Finally, an interview was conducted with an LAC partner (Treasury Board Secretariat). The use of multiple evaluation methods and the triangulation of data facilitated the corroboration of findings.

The evaluation answers the following questions:

• Was the Performance Measurement Strategy implemented?
• To what extent were the immediate and intermediate outcomes achieved?
• Were other ways of doing things considered to improve the achievement of outcomes?
• Was the recommendation of the Auditor General of Canada regarding the disposition authorities successfully met?
  • Has the response to the recommendation of the Auditor General of Canada contributed to the improvement of the Program?

Findings and recommendations

In 2014–2015, a new approach was implemented to complete the disposition authorities more rapidly. A three-year plan was also implemented to support the issuance of disposition authorities to the 175 federal organizations subject to the LAC Act. The Program successfully followed the Auditor General of Canada's recommendation.

While the Program offered support and directions to federal organizations in terms of information management, it did not have the authority to implement measures to increase the information management capacity of federal organizations, since this is a responsibility of Treasury Board. The Program is responsible for providing support to federal organizations regarding recordkeeping. In addition, the decrease in financial and human resources between 2013–2014 and 2016–2017 also hindered efforts to properly support federal organizations in matters of recordkeeping.

Aside from some output data, the evaluators noted that certain data had not been collected for the entire four year period covered by this evaluation and that certain performance measurement indicators had been either changed or dropped. There is very little data available to measure the progress made in the achievement of the mid- and long-term outcomes.

Recommendation 1: The Program should complete and implement its Performance Information Profile (PIP) and ensure that data is collected for all performance indicators on an ongoing basis.

Recommendation 2: The Program should implement an action plan and a schedule for the validation of disposition authorizations.

Recommendation 3: The Program should implement a communications action plan for federal organizations in order to encourage them to apply their disposition authorizations.

The management's response to the recommendations and the management action plan can be found in Appendix A.

1 Introduction

The evaluation report presents the findings and recommendations of the evaluation of Strategic Outcome: "Current Government Information is Managed to Support Government Accountability" (hereinafter referred to as the Program) of Library and Archives Canada (LAC). The evaluation was conducted at the level of Strategic Outcome 1.0 of the Program Alignment Architecture (PAA) that was in effect until March 31, 2018. More specifically, the evaluation reviewed the progress made in achieving the outcomes of the Program's two key activities. The methodology used meets the requirements of the Treasury Board's 2016 Policy on Results. ^endnote2

1.1 Purpose of the evaluation

The primary objective of this evaluation was to evaluate the program's performance. Section 3.2 includes the four evaluation questions that were addressed. As per the terms of reference of the evaluation that were approved in 2017, ^endnote3 the purpose of the evaluation was to review:

• the extent to which the key activities have made progress in achieving their outcomes;
• the extent to which the recommendation made by the Auditor General of Canada with respect to the Disposition Authorities Program has been successfully followed and helped improve the Program.

It should be noted that the formative evaluation ^endnote4 conducted in 2012 confirmed the relevance of the Program, and that is why this element was not assessed in this evaluation.

2 Library and Archives Canada and Program profiles

2.1 Overview of Library and Archives Canada (LAC)

Library and Archives Canada is a federal institution mandated to acquire, preserve and make accessible Canada's documentary heritage. LAC was established in 2004 following the merger of the National Archives Canada and the National Library of Canada (founded in 1953). The Library and Archives of Canada Act ^endnote5 came into force in 2004. It states that the institution's mandate is:

• to preserve the documentary heritage of Canada for the benefit of present and future generations;
• to serve as a source of enduring knowledge, accessible to everyone, contributing to the cultural, social, and economic advancement of Canada as a free and democratic society;
• to facilitate in Canada co-operation among the communities involved in the acquisition, preservation and diffusion of knowledge;
• to serve as the continuing memory of the Government of Canada and its institutions.

With regards to government records, LAC must apply the standards and practices stipulated in federal legislation, policies, regulations and internal procedures that govern the work of the Program, such as:

• Library and Archives of Canada Act;
• Policy on Information Management;
• Directive on Recordkeeping;
• Directive on Open Government;
• Directive on Disposition Authorizations;
• Evaluation and Acquisition Policy Framework;
• Guidelines on Appraisal of Government of Canada Records for the Issuance of Disposition Authorizations;
• Procedures for Approving and Issuing Disposition Authorizations
• Validation for Government of Canada Records: Frequently Asked Questions.

2.2 Program overview

The Strategic Outcome "Current Government Information is Managed to Support Government Accountability" is comprised of two activities as illustrated by the logic model in Appendix B. The first activity is "Development of regulatory and recordkeeping instruments," while the second activity is "Collaboration in the management of government records."

According to the program's data there are 175 federal organizations that are subject to the LAC Act. The full list of federal organizations that fall under the LAC Act is provided in Appendix C.

The Program plays a role in the development of standards, tools and best practices regarding information management and recordkeeping. This work is done in collaboration with Treasury Board, federal departments and other Government of Canada organizations. The Program issues disposition authorizations to federal organizations and provides directives and assistance regarding the storage, preservation and transfer of government records to LAC. The Program offers advice, support, services and training to federal institutions so that they can manage their information more effectively and comply with the Directive on Recordkeeping. The Program also offers information to support the work of decision makers in the community of federal libraries. However, LAC's activities related to the community of federal libraries are not covered by this evaluation.

Key activity 1: Development of regulatory instruments and recordkeeping tools

LAC issues disposition authorizations to federal organizations in order to support effective recordkeeping. These authorities specify which records must be transferred to LAC at the end of their lifecycle, because of their historical or archival value. The organization that created them can dispose of all its other documents at the end of their retention period. At LAC, the Government Archives Division is the entity responsible for developing, issuing and managing disposition authorities. The Division is also responsible for answering all questions from federal organizations regarding disposition authorities, record transfers to LAC and transfer procedures. In addition, the archivists are responsible for answering questions and requests regarding Access to Information Requests related to government records that LAC has in its custody. This Division is comprised of four sections:

1. Culture, Specialized Media and Description
2. Governance, Economy, Environment and Sciences
3. Security, Military, International Affairs, and Transportation
4. Disposition Tools, Indigenous Affairs, and Society

Each section is responsible for disposition authorities according to the portfolio of federal organizations which they have been assigned.

Following the issuance of disposition authorities, the sections initiate an analysis of the federal organizations' records in order to prepare validation reports. Thus federal organizations have all the tools they need in order to dispose of or transfer records, at the end of their retention period, to LAC.

Key activity 2: Collaboration in the management of government records

The Government Records Initiatives Division is responsible for this activity. It is comprised of two sections, each of which has a distinct role and mandate:

1. Government Records Services
2. Recordkeeping Strategies

The Government Records Services (GRS) section is responsible for monitoring existing disposition authorities as well as for following up on record transfers. The section's staff is also responsible for the translation of the disposition authorities and for entering information on the disposition authorities into the "RDACS" ^endnote6, as well as into the Collaboration Portal, which is LAC's records sharing system. The staff is also responsible for compiling and updating the list of contacts in federal organizations. Essentially, this section ensures that the procedures for the approval and issuance of disposition authorities are properly followed and that no steps are forgotten.

The Recordkeeping Strategies (RS) section is responsible for providing federal organizations with support and advice on recordkeeping. These basic services are the core of the services offered to federal organizations. More specifically, this section has developed and updated the "Generic Valuation Tools" and has helped federal organizations determine their retention periods. In addition, the section also collaborates with Treasury Board, Canada School of Public Service, Shared Services Canada and Public Services and Procurement Canada (e.g., symposium on recordkeeping, training in recordkeeping for federal public servants) on the development of various initiatives. In conjunction with Treasury Board, the section also helps federal organizations manage information by:

• offering advice on recordkeeping and  records management to federal organizations and to intergovernmental committees;
• giving information and awareness sessions to federal public servants at symposiums and recordkeeping forums;
• creating networks within the government information management community.

It is also involved in the Committee on the Establishment of ISO Standards for Recordkeeping and it chairs the National, Provincial and Territorial Recordkeeping Council. The section's staff participates in and represents LAC at international conferences on information management and recordkeeping.

2.3 Resources

In spite of a few variations, the Program's financial and human resources declined between 2013–2014 and 2016–2017 ^endnote7. The total financial resources of the Program went from $11.2 million to $7.5 million, a drop of $3.7 million, as presented in Table 1.2 below.

Table 1.1: LAC's Financial Resources

Fiscal Years	2013–2014	2014–2015	2015–2016	2016–2017
LAC actual spending	$100,803,692	$102,593,650	$91,451,612	$114,500,638endnote8

Table 1.2: Government Records Program (PAA 1.0) Financial Resources

Fiscal Years	2013–2014	2014–2015	2015–2016	2016–2017
Program 1.1: Development of regulatory instruments and recordkeeping tools	$2,694,577	$3,423,217	$2,636,780	$3,698,370
Program 1.2: Collaboration in the management of government records	$8,506,781	$9,392,789	$4,797,140	$3,788,193
Total financial resources (Salaries and operating expenses)	$11,201,358	$12,816,006	$7,433,920	$7,486,563
Proportion of LAC actual spending	11%	12.5%	8%	6.5%

Similarly, Table 2.2 indicates that the Program's total human resources, which were 111 FTEs in 2013–2014, declined to 85 FTEs in 2016–2017, a drop of 26 FTEs.

Table 2.1: LAC's Human Resources

Fiscal Years	2013–2014	2014–2015	2015–2016	2016–2017
LAC's human resources (FTEs)	885	951	913	903

Table 2.2: Government Records Program (PAA 1.0) Human Resources

Fiscal Years	2013–2014	2014–2015	2015–2016	2016–2017
Program 1.1: Development of regulatory instruments and recordkeeping tools	26	36	28	41
Program 1.2: Collaboration in the management of government records	85	77	59	44
Human resources (FTEs)	111	113	87	85
Commensurate with LAC's human resources (FTEs)	12.5%	12%	9.5%	9%

2.4 Program governance

Strategic Outcome "Current Government Information is Managed to Support Government Accountability" is under the responsibility of the Archives Branch, which itself is under the responsibility of the Operations Sector.

The Archives Branch includes the following two divisions:

• Government Archives Division;
• Government Records Initiatives Division.

2.5 New approach

In 2014 a new approach was introduced and was intended to provide complete and up-to-date disposition authorities coverage regardless of the state of information management and recordkeeping in federal organizations. Prior to the implementation of this approach, the granting of disposition authorities largely depended on the state of information management and recordkeeping in federal organizations, factors over which LAC has no control.

Subsequent to the tabling of the Report of the Auditor General of Canada in 2014, a special team was tasked with refocusing the disposition authorities on the identification of documents of historic and archival value forming the documentary heritage of the Government of Canada. To respond to the recommendation of the Auditor General of Canada, the resources normally tasked with the provision of advice to federal organizations were reallocated to Activity 1, the issuance of disposition authorities (see Table 2, Section 2.3).

To issue disposition authorities more rapidly, the Program proceeds in two stages. The first stage consists of issuing a disposition authority and allowing federal organizations to dispose of certain records at any time, provided they meet the destruction and disposition criteria set out in the disposition authority. Thus, the issuing of a disposition authorization is no longer dependent on a comprehensive knowledge of the functioning of federal organizations and on a detailed understanding of their information management practices.

In the second stage, i.e., the validation, the Program undertakes an in-depth analysis of the records of potential archival value produced by federal organizations. To do so, federal organizations must provide detailed information only regarding targeted sectors of their operations. The goal of the validation stage is therefore to identify concretely what records will be transferred to LAC by federal organizations based on their retention periods.

2.6 Expected outcomes

According to the logic model, the expected outcomes of the Program are the following:

Immediate outcomes:

• Issuance of disposition authorities in a timely manner.
• Increase in the capacity and level of preparedness to effectively manage Government of Canada information.

Intermediate outcome:

• A regulatory regime is established within the Government of Canada so that government information is managed and disposed of appropriately.

Final outcome:

• Current government information is managed so as to support accountability.

2.7 LAC's priorities related to the Program from 2013–2014 to 2016–2017

Between 2013–2014 and 2016–2017, LAC identified several priority activities in its Reports on Plans and Priorities (RPPs) that are related to the Program. These priorities are quite diverse, while their purpose is to improve the Program. In addition, they were significantly altered during that period. An examination of the priorities was carried out as part of the evaluation to determine whether these commitments have been met. The findings section links the departmental priorities and the outcomes achieved by the Program. The following can be found among the numerous Program commitments made in the Plans and Priorities:

• Issuance of disposition authorities: This commitment appears throughout the four years covered by this evaluation. The Program made a commitment to simplify the process and issue a disposition authority for all federal organizations.
• Development of standards, tools and guides: From 2013–2014 to 2015–2016, the Program made a commitment to play a major role in the development of standards, tools and guides for information management and recordkeeping.
• Support offered to federal organizations: From 2013–2014 to 2016–2017, the Program made a commitment to offer advice and guidance on information management, recordkeeping and the management of disposition authorities.
• Committees: From 2013–2014 to 2016–2017, the Program made a commitment to share diverse information through its participation in various committees.
• Recordkeeping Portal: In 2013–2014 and in 2014–2015, the Program made a commitment to disseminate advice and guidance, as well as tools and guides, through a portal.
• Network:^endnote9 In 2015–2016 and in 2016–2017, the Program made a commitment to implement a collaboration network.

2.8 Legislative context

The following section identifies the various internal and external policies and directives that provide a consistent intervention framework for the management of government records.

2.8.1 LAC Act (Justice Canada)

According to theLAC Act (2004), the Program is responsible for the following:

1. acquiring and preserving the documentary heritage;
   
2. making that heritage known to Canadians and to anyone with an interest in Canada and facilitating access to it;
3. being the permanent repository of publications of the Government of Canada and of government and ministerial records that are of historical or archival value;
4. facilitating the management of information by government institutions.

Destruction and disposal

Subsection 12 (1): "No government or ministerial record, whether or not it is surplus property of a government institution, shall be disposed of, including by being destroyed, without the written consent of the Librarian and Archivist or of a person to whom the Librarian and Archivist has, in writing, delegated the power to give such consents."

Transfer of records

Subsection 13 (1): "The transfer to the care and control of the Librarian and Archivist of government of ministerial records that he or she considers to have historical or archival value shall be effected in accordance with any agreements for the transfer of records that may be made between the Librarian and Archivist and the government institution or person responsible for the records."

2.8.2 Policy on Information Management (Treasury Board)

In addition to the four elements mentioned in the LAC Act, the Policy on Information Management, which came into force on July 1, 2007, states the following regarding the responsibilities of Library and Archives Canada:

Paragraph 8.2.3: Provides direction and assistance on recordkeeping for the Government of Canada.

Paragraph 8.2.4: Identifies, selects, acquires and preserves government records, as defined in the Library and Archives of Canada Act, in all media considered to be of enduring value to Canada as documentary heritage.

Paragraph 8.2.5: Issues records disposition authorities, pursuant to section 12 of the Library and Archives of Canada Act, to enable departments to carry out their records retention and disposition plans.

2.8.3 Directive on Recordkeeping (Treasury Board)

Stemming from the Policy on Information Management, this directive came into effect on June 1, 2009. Recordkeeping is a function through which information resources of business value are created, acquired, captured, managed in departmental repositories and used as a strategic asset to support decision making and facilitate ongoing operations and the delivery of programs and services. Information resources of a business value include published and unpublished documents in any media that are created or acquired to enable decision making, the conduct of operations and the delivery of services. They provide program managers, deputy heads, ministers, and Canadian citizens with reliable evidence of operational decisions, activities and transactions.

2.8.4 Directive on Disposition Authorizations (LAC)

This LAC internal directive took effect on June 26, 2013, and was revised on July 28, 2016. According to this directive, there are three types of disposition authorities:

1. Records Disposition Authority ^endnote10: which is either a Multi-Institutional Disposition Authority or an Institution-Specific Disposition Authority;
2. Disposition Authorization ^endnote11: granted to a federal organization to consent to the disposition of information resources under LAC's Disposition and Recordkeeping Program;
3. Official recommendation to dispose of federal documents preserved under LAC's custody.

Disposition authorities are based on the recommendations from the archivists. They follow LAC's internal approval processes in order to receive the approval of the Librarian and Archivist of Canada or his official delegate, and then they are forwarded to federal organizations.

Under LAC's mandate, in addition to the elements related to the alteration, revocation and monitoring of disposition authorities, there are two other obligations: the identification of federal records at risk of destruction or serious damage, and the obligation of the Librarian and Archivist of Canada to be responsible for the care and control of records of government organizations whose functions have ceased.

3 Methodology

3.1 Evaluation period

The evaluation covers the period of April 2013 to March 2017, that is, four years.

3.2 Evaluation questions and methods

The evaluation questions cover the performance of the Program. The following questions were examined and are presented below along with the evaluation methods used.

• Was the performance measurement strategy implemented?
  • Evaluation methods:
    • Records review
    • Internal interviews

• To what extent were the immediate and intermediate outcomes achieved?
  • Evaluation methods:
    • Documents review
    • Internal interviews
    • External interviews

• Were other ways of doing things considered to improve the achievement of outcomes?
  • Evaluation methods:
    • Documents review
    • Internal interviews

• Was the recommendation of the Auditor General of Canada regarding the disposition authorities successfully met?
  • Has the response to the recommendation of the Auditor General of Canada contributed to the improvement of the Program?
  • Evaluation methods:
    • Documents review
    • Internal interviews

The Program's administrative and financial documents, performance statistics and other internal documents were consulted. A total of 23 interviews were conducted with LAC managers and staff members involved in managing and delivering the Program. In addition, 14 interviews were also conducted with federal organizations subject to the LAC Act. Lastly, an interview was conducted with a LAC partner (the Treasury Board Secretariat). The use of several evaluation methods and data triangulation facilitated the corroboration of the findings.

3.3 Limitations of the evaluation

1. Since the performance data ^endnote12 available to evaluate the Program outcomes was limited, the evaluation team used other data sources, such as interviews and the Program's internal records to mitigate this limitation and better support the analyses.
2. The evaluation period covers the period starting in 2013–2014, which made it impossible to do a comparative analysis of the program's outcomes in previous years. The interviews allowed for the mitigation of this limitation by providing retroactive information.

3.4 Coding of findings

Evaluation findings were colour-coded to emphasize the aspects of the Program that require special attention.

• Green – No improvement needed
• Yellow – Some improvements would be needed
• Red – Improvements needed / recommendations

4. Findings – Relevance

4.1 Program relevance

Finding 1:

The lack of continuous data limits the analysis of the performance of certain Program activities and their progress in achieving the expected outcomes.

To carry out this evaluation, the evaluation team used the Program's logic model ^endnote13 which was reviewed during the pre-evaluation of the Program in 2015–2016, as well as the indicators included in the Performance Measurement Strategy (available in Appendix D).

To meet its performance information needs, the Program collected various data. However, this is mostly output data. The evaluators also noted that certain data had not been collected for the entire four-year period covered by the evaluation and that the indicators had been either altered or dropped. There is little data available to measure the progress made in achieving the intermediate and long-term outcomes. Consequently, the review of the available data did not yield a sufficient analysis to develop solid findings on the performance of certain program activities.

The interviews with managers and staff indicated that they were not aware of the program's logic model or of the performance indicators. They indicated that this was the responsibility of the planning officer. They also were not aware whether or not the data collected had been used.

4.2 To what extent were the immediate and intermediate outcomes achieved?

Immediate outcomes:

4.2.1 The disposition authorities are issued in a timely manner.

Finding 2:

The issuance of new disposition authorities is faster, taking a few months to under a year, instead of three to five years, as had previously been the case.

The new approach implemented in 2014–2015 was designed to speed up the completion of disposition authorities. The approach also supported the completion of disposition authorities in a more autonomous manner so that the need to interact with federal organizations was minimal. To achieve this goal, a three-year plan was implemented to support the issuance of disposition authorities for the 175 federal organizations subject to the LAC Act.

The interviews conducted with Program managers and staff confirmed that the issuance of new disposition authorities takes much less time. Mainly, it involves completing a standardized disposition ^endnote14 authorization and a customized disposition framework for each federal organization.

As can be seen in Table 3 below, the RDACS^endnote15 data show that the Program issued 98 disposition authorities to 169 federal organizations in the last four years. However, it should be noted that certain disposition authorities that were issued prior to 2014-2015 are still valid. There are 21 federal organizations that have such authorities. In addition, new federal organizations have been created, while others have been abolished, divided or amalgamated. That being said, the most significant finding (according to interview data) is that 100% ^endnote16 of federal organizations were covered by a disposition authority as of March 31, 2018.

Table 3: Number of Federal Organizations Covered by a new Disposition Authority

Year	2014–2015	2015–2016	2016–2017	2017–2018	Total
Number of signed disposition authorities	19	32	26	21	98*
Number of federal organizations	38	61	29	41	169

According to the review of internal documentation and the interviews with managers and staff, the Program has made significant efforts to simplify and improve the issuance of disposition authorities. The interviews with managers showed that the teams worked at full capacity and that the workload had to be reorganized in order to maximize productivity. Since there were changes made to the Program in 2014–2015, special teams (for the processing of the archival records backlog and the issuance of disposition authorities approach) were put in place, which affected the level of available resources over the years (see Table 2, Section 2.3).

The evaluation has determined that the procedures and templates that were implemented for the approval and issuing of disposition authorities allow for decision making to be properly documented. In addition, the data collected showed that the quality of the disposition authorities' instruments has improved; to that effect a Working Committee composed of representatives of each section of the Archives branch was put in place. The Committee provides guidance and regularly solicits employee's comments and feedback. Managers and staff are aware of the procedures and have a positive perception of the usefulness of the directives to their work. However, both groups indicated that the rapid frequency with which the procedures are updated makes any changes to the procedures difficult to follow.

Program managers and staff reported that the new approach works well. According to the managers, the approach makes it possible to:

• accelerate the issuance of disposition authorities by separating the issuance from the validation stage thereby increasing the efficiency of the process;
• relieve the pressure on federal organizations regarding records disposition;
• better plan and set priorities;
• re-examine the disposition authorities' instruments and tools and keep those that are still relevant.

The managers referred to the challenge of finding a balance between the various components of the disposition authorities issuing process and ensuring that all parts are moving simultaneously. In addition, they pointed out that there is a need to continue improving the monitoring of disposition authorities. They also highlighted the need to improve the planning and the prioritization of the validation process. The archivists are also solicited by other LAC programs, e.g., Public Services, on matters related to the Access to Information Act and the Privacy Act. In addition, the managers maintain that having good relations with federal organizations is critical to sustaining the willingness of federal organizations to co-operate with LAC throughout the validation period.

With regards to the Liaison Centre, which is under the Government Records Initiatives Division, it has been able to provide support to federal organizations, as indicated in Table 4 below.

Table 4: Number of Annual Requests by Type of Activity at the Liaison Centre
from 2014–2015 to 2016–2017

Fiscal Year	Disposition Authorities	Retention Periods	Transfer	Generic Evaluation Tools	Symposium (RK Day)	Information Management / Recordkeeping	Disposition and Recordkeeping Program
2014–2015	122	74	58	50	83	189	359
2015–2016	65	27	76	30	10	332	239
2016–2017	45	20	62	32	2	291	172

There are variations and inconsistencies in the requests data as demonstrated by Table 4. On one hand, some requests have significantly decreased, e.g., requests related to disposition authorities, which can be attributed to the efforts made since 2014 under this activity. On the other, it could be noted that the requests that still require the most involvement of program staff are the ones related to information management and recordkeeping.

The evaluators asked representatives of federal organizations about the services received from Program staff. The following are a few examples of their appreciation:

• 68% of federal organization respondents said that they appreciate the support that is given;
• 88% of them said that the Program archivists quickly respond to their questions, either on the same day or within two to five business days;
• 43% of respondents said that they needed more support and answers to more concrete questions.

For the archivists, the biggest challenge regarding the disposition authorities' process is their relationships with federal organizations. They pointed out that not all federal organizations have good recordkeeping practices in place, that not all are well organized and that not all have record keeping specialists. In some organizations, the people responsible for the management of disposition authorities do not have the experience and knowledge to understand their disposition authorities and manage their records well. All these factors have an impact on the efficiency of Program employees' work.

Federal organization respondents made similar comments:

• Regarding their role in the process of issuing disposition authorities, 79% of federal organization respondents reported that they were aware of the role they have to play;
• 62% of respondents reported that they have a good understanding of the issuance of disposition authorities process, while 31% do not understand it well or do not understand it at all;
• 46% of respondents reported that they had a good understanding of the document;
• 69% of respondents reported that they had received a new disposition authority as of 2015;
• 31% did not know whether or not they had a new disposition authority or they did not know whether or not they had one;
• 67% reported that they had received directives on the application of their disposition authority; yet, among the latter, 57% said that they had problems in applying their disposition authorities;
• Lastly, 57% reported that they had a previous disposition authority, while 29% said that they did not have one. Another 14% did not know whether or not they had a previous disposition authority.

4.2.2 Validation

Finding 3:

The validation process was initiated in 2017. However, the validation procedures need to be clarified and a monitoring mechanism needs to be put in place to ensure government records are kept up to date and relevant.

The second stage implies that the Program produces one or more validation reports specifying the records that LAC wants to receive from federal organizations. During this stage, federal organizations are required to make an effort to assist the Program's archivists to identify records of archival or historical value. In addition, with respect to the records identified, the archivist is responsible for making a selection based on the information provided by the federal organization in order to complete the validation process. Validation can be carried out in several stages, depending on the complexity or size of the federal organization. In order to enable the archivists to complete the validation reports the following factors are very important: collaboration with federal organizations, availability of their staff and their level of preparedness.

The validation stage has just begun (see Graphic 1 below) as the Program planned to first finalize the issuance of disposition authorities in 2017–2018. In light of what has been discussed above, the evaluators cannot provide a robust opinion on progress made regarding the validation stage since the process will unfold more intensively in 2018.

• Description of Chart 1

  presents the validation status of federal organizations in numbers. There are 16 federal organizations that have full validation, 11 federal organizations have partial validation and 126 federal organizations have not yet been validated.

The evaluation team has also been informed that in order to be able to complete the validation process, federal organizations must have put in place a record classification system. While the following data are not representative of all federal organizations subject to the LAC Act, it appears that the majority of those interviewed during the evaluation have such a system. According to the interviews with federal organizations:

• 71% of respondents said that they had a retention plan or a disposition schedule for their records;
• 57% mentioned that they had a records management system.

The LAC Act required that records of archival or historical value be transferred under the custody and responsibility of LAC. This transfer must be carried out according to the terms and conditions that specify the obligations, transfer criteria and consequences of non-compliance with the terms and conditions. According to the managers who took part in the evaluation, while LAC is responsible for issuing disposition authorities, it does not have the authority to require that federal organizations transfer records.

According to Program data and as illustrated below (see Chart 2), there were 3,865 ^endnote18 transfers of government records to LAC from 2013–2014 to 2016–2017, which represents 86,000 boxes of records. That said, a federal organization can decide to preserve its records for as long as it wishes, even after their retention period has expired. The only exception concerns records that are at risk of damage or destruction. The Program has the authority to require that such records be transferred to LAC in order to preserve them.

Source: Program Data (Liaison Centre).

• Description of Chart 2

  presents the number of records transfers by federal organizations to LAC for the period 2013-2014 to 2016-2017. For 2013-2014 there are 1518 records transfers, for 2014-2015 there are 1284 record transfers, for 2015-2016 there are 589 record transfers, and for 2016-2017 there are 474 record transfers.

4.2.3 Increasing capacity and preparedness allows for effective management of government information

Finding 4:

There are barriers that limit the Program's contribution to increasing federal institutions' recordkeeping capacity and readiness.

Division of responsibilities

Section 8.2.3 of the Policy on Information Management indicates that LAC's program "provides direction and assistance on recordkeeping for the Government of Canada." Section 8.1.2 of the same Policy states that Treasury Board "develops and promotes...standards, procedures, directives, guidelines, tools, and best practices that achieve the goals and expected results of the Policy on Information Management."

Internal program documentation indicates that, "in cooperation with central agencies and other partners, the Program plays a role in developing information management and recordkeeping standards, tools and best practices", for which Treasury Board has primary responsibility.

Although the Policy on Information Management and the Directive on Recordkeeping clearly set out the responsibilities of both LAC and the Treasury Board, these responsibilities have not actually been divided as prescribed by the policy. The Program does not have the authority to implement measures to increase information management capacity in federal organizations, as this is a responsibility that belongs to Treasury Board. The Program is responsible for providing federal organizations with recordkeeping support in preparation for the transfer of records of archival or historical value. However, the Program has provided tools and activities beyond the scope of its responsibility. In 2014, with the new approach, the Program revised its activities to ensure they reflect its incumbent responsibilities.

As a result, for program managers and employees, there seems to be a grey area in terms of the support to be provided to federal organizations. As the Program had handled certain types of activities in the past, the perception that these activities fall under the Program's purview seems to have persisted among some people. This ambiguity is also felt by federal organizations. The interviews with federal organizations showed that:

• 56% of respondents said that the support provided through the Program did not meet their needs; and
• 69% of respondents said that the Program did not consult with them on their information management and recordkeeping needs.

Availability of resources

The decrease in the Program's financial and human resources between 2013–2014 and 2016–2017 (ref.: section 2.3) also hindered efforts to provide federal organizations with adequate recordkeeping support.

In 2013–2014,^endnote19 the Program had supported federal organizations by providing them with advice and work instruments, including four generic valuation tools, on its new Recordkeeping Portal. In the 2014−2015 Departmental Performance Report, the Program indicates that it continued to provide generic valuation tools. Therefore, federal organizations are well equipped to manage their records on common government activities and understand the specifications for their retention. Furthermore, in 2014–2015, 5,394 people visited the Portal, a 14% increase over the previous year. However, as resources declined between 2013–2014 and 2016–2017, these tools have not been updated.

Cooperation

The Program organized various activities over the years to meet the needs of federal organizations:

• two recordkeeping symposiums in 2014–2015 for federal public service employees in cooperation with TBS; and
• two information management learning activities with TBS in 2016–2017. ^endnote20

However, according to interviews conducted with program staff, there has been less cooperation between the Program and TBS in recent years, which has impacted the services provided to federal organizations. Similar statements were made during an interview with a TBS employee. Although TBS is no longer involved in organizing the symposiums, the employee recognized that there was a successful collaboration with LAC, which regrettably has decreased over time. The person recognized LAC's recordkeeping expertise and the usefulness of the generic valuation tools LAC had developed and which TBS continues to share with federal organizations.

According to the interview with TBS, the latter uses the Management Accountability Framework to assess whether federal organizations comply with the Information Management Policy and the Directive on Recordkeeping.

Intermediate

4.2.4 Implementing a regulatory system in all Government of Canada organizations allows for the appropriate management and disposition of government records.

Finding 5:

LAC issues disposition authorizations. However, there are no monitoring mechanisms in place to ensure that the authorities are applied.

Program documentation shows that once the disposition authority is issued and the validation is completed (partially or fully), the Program is responsible for monitoring its application. However, there are no monitoring mechanisms in place allowing the Program to adapt to any significant changes that could arise in federal organizations, such as a mandate change or an information management system transformation. Regular monitoring of the application of disposition authorities and validation would allow the Program to ensure that identified records of archival or historical value are transferred to LAC after their retention periods have expired.

The Program is currently exploring several options for proper follow-up with federal organizations to ensure authorities are applied.

Although the Program can provide advice and guidance on recordkeeping, federal organizations are responsible for developing a records classification plan, for ensuring records management and for establishing records retention periods within their organization. Once the retention periods have ended, federal organizations must dispose of the operational records and transfer the records of archival or historical value to LAC in accordance with their disposition authorities and validation report(s).

4.3. Were other ways of doing things considered to improve the achievement of outcomes?

Finding 6:

The disposition authority issuing approach implemented in 2014 has helped reduce timelines.

Prior to 2014, the issuing of disposition authorities depended largely on the state of information management and recordkeeping at federal organizations, over which LAC has no control. The new approach is intended to allow LAC to provide complete disposition coverage, regardless of the state of information management and recordkeeping at federal organizations.

Following the 2014 release of the Auditor General of Canada's report, a new method was intended to be used to identify historical and archival records in order to issue disposition authorities faster. However, these new authorities required an additional step: the validation. According to archivists, the disposition authority issuing process is simplified, better structured and more targeted. It now takes a few months to a year to issue new disposition authorities. More specifically, they stated that the disposition authority procedures and guidelines put in place have been very useful. The archivists interviewed pointed out the following benefits of the new approach:

• the right heritage materials are received, which alleviates the volume of records that LAC acquires;
• the record creation in federal organizations can be better studied, and thus better understood;
• the focus is on the disposition authority issuing process; and
• federal organizations are provided with greater flexibility, as they can apply their disposition authorities immediately and thus dispose of certain types of government records.

In summary, federal organizations can immediately dispose of a portion of their records as soon as their authorities are issued. However, they must make an effort before they can do the same with records identified in their disposition framework as potentially having archival or historical value.

However, because of the period examined in the evaluation it was not possible to determine whether any other approaches were tested prior to 2014. That being said, the managers who took part in the interviews stated that the changes made to the Program in 2013–2014 are satisfactory and that they saw no need to make any other changes to the approach being used right now.

4.4 Was the recommendation of the Auditor General of Canada regarding the disposition authorities successfully met?

Finding 7:

The recommendation issued by the Auditor General of Canada was successfully met.

In fall 2014, the Auditor General of Canada made the following recommendation:

"Library and Archives Canada should ensure that disposition authorities of the federal government's archival records are kept up to date. To accomplish this, it should develop a plan with achievable timelines for issuing and updating the necessary disposition authorities. It should also continue to engage with institutions and to monitor the adequacy of existing disposition authorities."

The Program provided a management response in which it made a commitment to address the Auditor General of Canada's recommendation. To achieve this, a three-year plan was implemented to support the issuance of disposition authorities to the 175 federal organizations subject to the LAC Act. Therefore, the purpose of the new approach that the Program implemented in 2013–2014 was to complete disposition authorities more quickly to minimize the need to interact with federal organizations. According to program data, the disposition authority issuance reached 100% in late March 2018.

According to the Auditor General's report, ^endnote21 disposition authorities previously took three to five years to issue. These authorities took a great deal of time because the Program had to rely on federal organizations' availability. This is no longer the case with the issuance of the new disposition authorities.

4.4.1 Has the response to the recommendation of the Auditor General of Canada contributed to the improvement of the Program?

Yes, as new disposition authorities are issued much faster; it now takes a few months to a year instead of three to five years, which was the case prior to the implementation of the new approach in 2014.

5 Conclusions and recommendations

5.1 Conclusions

The Program's data collection is incomplete with respect to the performance measurement strategy indicators. Although the Program collects certain data, especially on outputs, some data on immediate and intermediate outcomes are partial or lacking for the four year period covered by the evaluation, or the collection of these data was discontinued. However, the evaluation team was informed that the Program has established a draft Performance Information Profile (PIP) in 2017, which includes a new logic model and new performance indicators.

Finally, the evaluation led to the following findings:

• Implementing a new approach helped reduce the time it takes to issue disposition authorities;
• All federal organizations have been covered by a disposition authority as of March 2018, according to program data;
• The Auditor General of Canada's recommendation regarding disposition authority coverage for all federal organizations has been met;
• The validation phase has begun; however, the acquisition of records of archival and historical value will depend on the Program's capacity to produce validation reports;
• Federal organizations appreciate the support received from program staff, in the form of training, advice and guidance on recordkeeping;
• Smaller organizations have pointed out the need for more concrete information on recordkeeping;
• Although not all staff responsible for information management and recordkeeping in federal organizations have the knowledge and experience needed to understand well the disposition authorities and manage their records, all of them have shown interest in learning more and obtaining further support from the Program and TBS;
• The Program documentation and interviews with managers and staff confirmed that there were no mechanisms in place to monitor the application of disposition authorities between 2013–2014 and 2016–2017.

5.2 Recommendations

The evaluation of strategic outcome, "Current Government Information is Managed to Support Government Accountability," led to the following recommendations.

Appendix A: Management Response and Action Plan

Appendix B: Logic Model

• Description of Chart 3

  Appendix B shows the logic model for the Strategic Outcome: Current government information is managed to support government. To read the logic model, it should start from bottom-up by the activities, then the outputs (products of the Program) and the outcomes (what the Program try to change).

  Activities

  • Development of regulatory instruments
  • Collaboration in the management of government records

  Outputs

  • Disposition instruments
  • Policy instruments and Tools
  • Support and Services

  Immediate Outcome

  • Timely issuance of disposition authorities
  • Increased capacity and readiness to manage GC information effectively

  Intermediate Outcome

  • Regulatory regime is established across GC and government information is managed and disposed of appropriately

  Ultimate Outcome

  • Current government information is managed to support Government accountability

Appendix C: List of Federal Organizations Subject to the LAC Act (as of April 16, 2018)

1. Administrative Tribunals Support Service of Canada
2. Asia Pacific Foundation of Canada
3. Atlantic Canada Opportunities Agency
4. Atlantic Pilotage Authority
5. Atomic Energy of Canada Limited
6. Bank of Canada
7. Belledune Port Authority
8. British Columbia Treaty Commission
9. Business Development Bank of Canada
10. Canada Border Services Agency
11. Canada Council for the Arts
12. Canada Deposit Insurance Corporation
13. Canada Development Investment Corporation
14. Canada Development Investment Corporation
15. Canada Economic Development for Quebec Regions
16. Canada Employment Insurance Commission
17. Canada Foundation for Innovation
18. Canada Foundation for Sustainable Development Technology
19. Canada Lands Company Limited
20. Canada Mortgage and Housing Corporation
21. Canada Post Corporation
22. Canada Revenue Agency
23. Canada School of Public Service
24. Canada-Newfoundland Offshore Petroleum Board
25. Canada-Nova Scotia Offshore Petroleum Board
26. Canadian Air Transport Security Authority
27. Canadian Broadcasting Corporation
28. Canadian Centre for Occupational Health and Safety
29. Canadian Commercial Corporation
30. Canadian Dairy Commission
31. Canadian Environmental Assessment Agency
32. Canadian Food Inspection Agency
33. Canadian Grain Commission
34. Canadian Human Rights Commission
35. Canadian Institutes of Health Research
36. Canadian Museum for Human Rights
37. Canadian Museum of History
38. Canadian Museum of Immigration at Pier 21
39. Canadian Museum of Nature
40. Canadian Northern Economic Development Agency
41. Canadian Nuclear Safety Commission
42. Canadian Race Relations Foundation
43. Canadian Radio-television and Telecommunications Commission
44. Canadian Security Intelligence Service
45. Canadian Space Agency
46. Canadian Tourism Commission
47. Canadian Transportation Accident Investigation and Safety Board
48. Canadian Transportation Agency
49. Civilian Review and Complaints Commission for the Royal Canadian Mounted Police
50. Communications Security Establishment
51. Copyright Board
52. Correctional Service of Canada
53. Defence Construction (1951) Limited
54. Department of Agriculture and Agri-Food
55. Department of Canadian Heritage
56. Department of Citizenship and Immigration
57. Department of Employment and Social Development
58. Department of Finance
59. Department of Fisheries and Oceans
60. Department of Foreign Affairs, Trade and Development
61. Department of Health
62. Department of Indian Affairs and Northern Development
63. Department of Industry
64. Department of Justice
65. Department of National Defence
66. Department of Natural Resources
67. Department of Public Safety and Emergency Preparedness
68. Department of Public Works and Government Services
69. Department of the Environment
70. Department of Veterans Affairs
71. Department of Western Economic Diversification
72. Export Development Canada
73. Farm Credit Canada
74. Federal Bridge Corporation Limited
75. Federal Economic Development Agency for Southern Ontario
76. Federal Public Service Health Care Plan Administration Authority
77. Financial Consumer Agency of Canada
78. Financial Transactions and Reports Analysis Centre of Canada
79. First Nations Financial Management Board
80. First Nations Tax Commission
81. Freshwater Fish Marketing Corporation
82. Great Lakes Pilotage Authority
83. Gwich'in Land and Water Board
84. Gwich'in Land Use Planning Board
85. Halifax Port Authority
86. Hamilton Port Authority
87. Historic Sites and Monuments Board of Canada
88. Immigration and Refugee Board
89. Indian Residential School Truth and Reconciliation Commission
90. International Development Research Centre
91. Inuvialuit Water Board
92. Laurentian Pilotage Authority
93. Library and Archives Canada
94. Mackenzie Valley Environmental Impact Review Board
95. Mackenzie Valley Land and Water Board
96. Marine Atlantic Inc.
97. Military Grievances External Review Committee
98. Military Police Complaints Commission
99. Montreal Port Authority
100. Nanaimo Port Authority
101. National Arts Centre Corporation
102. National Battlefields Commission
103. National Capital Commission Corporation
104. National Defence and Canadian Forces Ombudsman
105. National Energy Board
106. National Farm Products Council
107. National Film Board
108. National Gallery of Canada
109. National Museum of Science and Technology
110. National Parole Board
111. National Research Council of Canada
112. Natural Sciences and Engineering Research Council
113. Northern Pipeline Agency
114. Nunavut Impact Review Board
115. Nunavut Planning Commission
116. Nunavut Surface Rights Tribunal
117. Nunavut Water Board
118. Office of Infrastructure of Canada
119. Office of the Administrator of the Ship-source Oil Pollution Fund
120. Office of the Auditor General
121. Office of the Chief Electoral Officer
122. Office of the Commissioner of Lobbying
123. Office of the Commissioner of Official Languages
124. Office of the Coordinator, Status of Women
125. Office of the Correctional Investigator
126. Office of the Director of Public Prosecutions
127. Office of the Information Commissioner of Canada
128. Office of the Privacy Commissioner of Canada
129. Office of the Public Sector Integrity Commissioner
130. Office of the Superintendent of Financial Institutions
131. Office of the Veterans Ombudsman
132. Oshawa Port Authority
133. Pacific Pilotage Authority
134. Parks Canada Agency
135. Patented Medicine Prices Review Board
136. Pierre Elliott Trudeau Foundation
137. Polar Knowledge Canada
138. Port Alberni Port Authority
139. PPP Canada Inc.
140. Prince Rupert Port Authority
141. Privy Council Office
142. Public Health Agency of Canada
143. Public Sector Pension Investment Board
144. Public Service Commission
145. Quebec Port Authority
146. RCMP External Review Committee
147. Ridley Terminals Inc.
148. Royal Canadian Mint
149. Royal Canadian Mounted Police
150. Saguenay Port Authority
151. Sahtu Land and Water Board
152. Sahtu Land Use Planning Board
153. Saint John Port Authority
154. Seaway International Bridge Corporation, Ltd.
155. Security Intelligence Review Committee
156. Sept-Îles Port Authority
157. Shared Services Canada
158. Social Sciences and Humanities Research Council
159. St. John's Port Authority
160. Standards Council of Canada
161. Statistics Canada
162. Telefilm Canada
163. The Jacques-Cartier and Champlain Bridges Inc.
164. Thunder Bay Port Authority
165. Toronto Port Authority
166. Transport Canada
167. Treasury Board (Secretariat)
168. Trois-Rivières Port Authority
169. Vancouver Fraser Port Authority
170. Veterans Review and Appeal Board
171. VIA Rail Canada Inc.
172. Windsor Port Authority
173. Windsor-Detroit Bridge Authority
174. Yukon Environmental and Socio-economic Assessment Board
175. Yukon Surface Rights Board

Appendix D: Performance Measurement Strategy Outputs

Outputs

Immediate Outcomes

Intermediate Outcomes

Ultimate Outcomes

Economy and Efficiency

Appendix E: Bibliography

• Departmental Performance Report 2013–2014, Library and Archives Canada.
• Departmental Performance Report 2014–2015, Library and Archives Canada.
• Departmental Performance Report 2015–2016, Library and Archives Canada.
• Departmental Performance Report 2016–2017, Library and Archives Canada.
• Directive on Disposition Authorizations, Library and Archives Canada, June 26, 2013, and new version July 28, 2016.
• Directive on Information Management Roles and Responsibilities, Treasury Board of Canada, October 8, 2007.
• Directive on Recordkeeping, Treasury Board of Canada, June 1, 2009.
• Directive on Results, Treasury Board of Canada, July 1, 2016.
• Evaluation and Performance Framework for Program Alignment Architecture ‒ 1.0, Current Government Information is Managed to Support Government Accountability, updated October 6, 2014, Library and Archives Canada.
• Library and Archives Canada Role in Disposition and Information Management for the Government of Canada, presented to LAC's Operations Committee by the Government Records Branch, May 30, 2017.
• Policy on Information Management, Treasury Board of Canada, April 1, 2012.
• Policy on Results, Treasury Board of Canada, July 1, 2016.
• Presentation to Robert McIntosh, Director General, Archives Branch: Program Evaluation, Government Information (Recordkeeping)(PAA 1.0), July 2017.
• Procedures for Approving and Issuing Disposition Authorizations, version 3.3, Library and Archives Canada, February 21, 2018.
• Procedures for the Transfer of Unpublished Information Resources of Enduring Value from Government of Canada Institutions to Library and Archives Canada, June 20, 2014.
• Renewal of the Library and Archives Canada Appraisal and Disposition Program, Government Records Initiatives Division Disposition Committee, February 2017.
• Report of the Auditor General of Canada, chapter 7 "Documentary Heritage of the Government of Canada—Library and Archives Canada," Office of the Auditor General of Canada, fall 2014.
• Report on Plans and Priorities 2013–2014, Library and Archives Canada.
• Report on Plans and Priorities 2014–2015, Library and Archives Canada.
• Report on Plans and Priorities 2015–2016, Library and Archives Canada.
• Report on Plans and Priorities 2016–2017, Library and Archives Canada.
• Terms of Reference, Evaluation of Current Government Information is Managed to Support Government Accountability Strategic Outcome (PAA 1.0), Corporate Planning and Accountability, May 2017.

​