Evaluation of Supporting the Documentary Heritage Communities Program, 2015–2016 to June 2017

Evaluation Function
Corporate Planning and Accountability

October 2018

Table of contents

• Summary
• Evaluation findings
• 1. Introduction
• 2. Overview of Library and Archives Canada and supporting the Documentary Heritage Communities Program
  • 2.1 Brief description of Library and Archives Canada (LAC)
  • 2.2 Supporting the Documentary Heritage Communities Program
  • 2.3 Program governance
• 3. Methodology
  • 3.1 Evaluation period and scope
  • 3.2 Evaluation questions
  • 3.3 Evaluation methods
  • 3.4 Limitations of the evaluation
  • 3.5 Coding of findings
• 4. Findings – Relevance
  • 4.1 Alignment with LAC and Government of Canada priorities
  • 4.2 Meeting the ongoing needs of Canadians
• 5. Findings - Performance
  • 5.1 Performance measurement information
  • 5.2 Effective use of contribution funds
  • 5.3 Program commitments
  • 5.4 Raising awareness about local institutions and their collections
  • 5.5 Increasing the capacity of local institutions to preserve documentary heritage
  • 5.6 Expanded use of local collections
  • 5.7 Other outcomes
• 6. Findings - Efficiency
  • 6.1 Overlaps and similarities with other programs
  • 6.2 Scope of funded institutions
  • 6.3 Efficient use of internal resources
  • 6.4 Program delivery method
• 7. Other observations
  • 7.1 Showcasing diversity
• 8. Conclusion and recommendations
  • 8.1 Conclusion
  • 8.2 Recommendations
• Appendices

Summary

Introduction

The evaluation of Supporting the Documentary Heritage Communities Program^footnote1 (hereafter SDHCP or “the Program”) was performed between June 2017 and May 2018 and meets the requirements of Treasury Board’s 2016 Policy on Results. The main objectives of the evaluation were to examine the relevance of the Program and to report on the progress made in achieving its outcomes since its inception in 2015.

An internal audit of the Program was also conducted at the same time. Both support the improvement of the Program, accountability and decisions that will support its renewal.

Program overview

The SDHCP was launched by Library and Archives Canada (LAC) in summer 2015. The purpose of the Program is to provide funding in the form of contributions to local documentary heritage institutions, to increase their capacity to preserve their collections and to promote and make them available to Canadians. Funding of $1.5 million per year in contributions has been allocated.

Evaluation approach and methodology

This evaluation examined the following issues.

Relevance:

• Do the priorities of the Program align with the priorities of LAC and the Government of Canada?
• Is the Program still relevant and responsive to changing client needs?

Performance:

• Is performance information sufficient and of good quality?
• Is the Program making progress towards immediate and intermediate outcomes?
• Is the Program using resources efficiently?

The following qualitative and quantitative methods and data triangulation were used in the evaluation:

• Review of external studies, reports and surveys;
• Review of internal administrative documentation;
• Review of applicant and recipient files (funding applications, financial reports and recipient performance reports);
• Interviews with 15 LAC employees;
• Interviews with six members of the External Advisory Committee;
• Survey of 225 Program applicants and recipients, with a response rate of 28%; and
• Analysis of recipients’ social media and websites.

Lastly, tests performed by the joint internal audit contributed to the analyses.

Evaluation findings

Relevance

The SDHCP supports LAC’s mandate and strategic outcomes by supporting documentary heritage communities. Through the Program, LAC provides funding to ensure that Canada’s continuing memory is documented and accessible for present and future generations, while taking a collaborative approach with local memory institutions.

The Program also supports government priorities through its integration into the Government of Canada’s Whole-of-Government Framework. Its objectives include supporting the funding of projects related to Canadian culture and improving knowledge of Canada’s history and cultural heritage.

The data collected show that projects funded by the Program are aligned with the needs of Canadians. Digitization is a very important need for 73% of the institutions surveyed. Internal and external respondents emphasized that Canadians need to know that local collections exist and that institutions manage and take care of them. When asked about the needs of their members and clients, institutional respondents said the following needs were very important: development of local collections (58%), research support (48%) and in-person access to collections (40%).

Performance information

Data collection for measuring medium- and long-term outcomes are based solely on interim and final reports from recipients. Since the reports are completed before the end or at the end of projects, they provide a solid source of data to demonstrate outputs and outcomes in the short term; however, they are inadequate for measuring medium- and long-term outcomes. There is no mechanism in place to measure, in a timely manner, the impact of the Program after projects are completed.

Effective use of contribution funds

Program guidelines posted on LAC’s website from 2015–2016 to 2017–2018 stated that the analysis and selection of eligible projects was based on project merit, program priorities, regional representation and availability of funds. However, in interviews, Program officials and External Advisory Committee members confirmed that the Program had not established priorities. Committee members noted that priorities would have made it easier to select projects for funding and would have made decisions more objective and transparent, given the large number of applications received.

Program commitments

The data show that the implementation of the Program partially met initial expectations. On the one hand, the Program met its funding objective by providing $1.5 million in contributions. On the other hand, the commitments made for the number of projects and their distribution between objectives 1 and 2, as well as the commitment for the average amount of funding granted, were not met. However, it is impossible to assess the merit of decisions that were made in relation to commitments that were not fulfilled.

Immediate outcomes

According to the results of the survey, 62% of recipients believe that the Program significantly helps raise awareness about local documentary heritage institutions and their collections. Collection management, cataloguing, conversion and digitization for access are a large part of the activities undertaken. Their implementation by recipients demonstrates that they have increased the visibility and availability of collections to the public.

Of the recipients who responded to the survey, 86% said that the assistance provided by the Program significantly helped increase their ability to preserve local collections. An analysis of recipients’ annual reports shows that the activities performed have indeed helped to increase institutional capacity, whether through training workshops, the hiring of digital preservation specialists, staff and volunteer training, the use of better work methods, or other activities.

Intermediate and ultimate outcomes

Final recipient report data and Program information are not sufficient to assess the medium- and long-term impacts of the Program. It was noted, however, that the Program has been very well received by the documentary heritage community.

Efficiency

There is overlap between the Museums Assistance Program (administered by Canadian Heritage) and the SDHCP: museums and museum sector institutions may receive funding from two separate federal programs that have very similar objectives and funding arrangements. Survey and interview participants questioned whether the museum sector should be included in the SDHCP since the latter cannot meet the demand from other types of institutions.

Participants also questioned the definition of “local institutions.” The data show that the Program funds a large number of institutions whose scope is reported as provincial or greater, and institutions whose spending budgets have increased steadily over the past three years. Several expressed a desire to see a greater number of small local institutions obtain funding. A key concern that emerged from the survey was the perceived under-representation of small institutions.

The administrative expenses of the Program are within acceptable limits when compared with other federal contribution programs. Nonetheless, the analyses performed show a steady increase in costs since the inception of the Program. Financial coding and monitoring mechanisms would help contain administrative costs in future cycles.

Other outcomes

Funded institutions rely heavily on volunteers. Approximately 30% of surveyed recipients said that the Program contributed moderately or greatly to volunteer recruitment. Further analysis would validate this outcome and provide a better understanding of local community volunteer involvement in projects.

The projects funded highlight Canadian diversity, including minority-language, cultural, religious and ethnic communities.

Conclusion

An analysis of the data confirms that the Program helps targeted institutions meet the needs of Canadians by increasing their capacity to preserve their documentary heritage and make it available (in particular through digitization).

Overall, the Program is achieving its short-term outcomes. The data show that activities initiated in the first two cycles have been completed and that a large proportion of the collections processed are available on site or on the Web.

The Program is new, and efforts have been made to meet the needs of applicants and recipients in the first three cycles of its implementation. Halfway through, the Program is at a pivotal stage. The evaluation results provide an opportunity to make some adjustments to its scope, objectives, management and performance measurement.

At this stage in the implementation of the Program, it is important to reflect on the targeted population of the contributions. The definition of “local institution” is particularly worth examining from this standpoint, because it opens the door to thinking about the scope of the Program.

Recommendations

1. The Program should review its scope, logic model and performance measurement strategy, and set annual performance targets for all its indicators.
2. The Program should clarify and communicate its priorities for each funding cycle.
3. The Program should examine overlaps with the Museums Assistance Program to determine whether the SDHCP needs to make adjustments or adopt a complementary approach.
4. The Program should clarify the definition of “local institution” and include it in its guidelines.

Management response and action plan

Management agrees with all of the recommendations and will implement an action plan, as outlined in Appendix A of this report.

1. Introduction

This report presents the findings and recommendations of the evaluation of Library and Archives Canada’s Supporting the Documentary Heritage Communities Program (SDHCP). The evaluation was performed between June 2017 and May 2018 and meets the requirements of Treasury Board’s 2016 Policy on Results.

Library and Archives Canada (LAC) made a commitment in its Treasury Board submission to evaluate the Program. The commitment also fulfills a request by LAC’s Departmental Performance Measurement and Program Evaluation Committee for an evaluation by June 2018, to provide information on the performance of the Program since its inception. An internal audit of the Program was also conducted at the same time. Both support the improvement of the Program, accountability and decision-making for renewal.

Under the Financial Administration Act, the Policy on Transfer Payments and the Policy on Results, LAC is responsible for ensuring that transfer payment programs are, and remain, relevant and effective in achieving their objectives and those of the government. LAC must continually monitor its contribution program and periodically assess its relevance and performance.

The main objectives of this evaluation were to examine the relevance, effectiveness and efficiency of the Program and to report on the progress made in achieving its outcomes since its inception.

2. Overview of Library and Archives Canada and supporting the Documentary Heritage Communities Program

2.1 Brief description of Library and Archives Canada (LAC)

LAC is a federal institution responsible for acquiring, preserving and providing access to Canada’s documentary heritage. The Dominion Archives, founded in 1872 as a division of the Department of Agriculture, became the Public Archives of Canada in 1912, then was renamed as the National Archives of Canada in 1987. LAC was created in 2004 through the merging of the National Archives and the National Library of Canada (founded in 1953). The Library and Archives of Canada Act^footnote2 came into force the same year. It states that the institution’s mandate is as follows:

• To preserve the documentary heritage of Canada for the benefit of present and future generations;
• To be a source of enduring knowledge accessible to all, contributing to the cultural, social and economic advancement of Canada as a free and democratic society;
• To facilitate in Canada co-operation among the communities involved in the acquisition, preservation and diffusion of knowledge; and
• To be the continuing memory of the Government of Canada and its institutions.

2.2 Supporting the Documentary Heritage Communities Program

The SDHCP was launched by LAC following a Cabinet decision in 2015. The purpose of the Program, through a reallocation of a portion of LAC’s funds, is to provide funding in the form of contributions to local documentary heritage institutions to increase their capacity to preserve their collections and promote and make them available to Canadians.

Outcomes

The Program’s activities, outlined in Appendix B (Logic Model), aim to achieve four outcomes:

• Increased capacity of documentary heritage institutions to preserve Canada’s documentary heritage in a more sustainable way (immediate);
• Better knowledge of local documentary heritage (immediate);

• Greater use of the resources and collections of local documentary heritage institutions (intermediate); and
• Expanded access to Canada’s local documentary heritage for Canadians (ultimate).

Applicants^footnote3and recipients^footnote4

The Program is open to local documentary heritage institutions, including archives, privately funded libraries, historical societies, genealogical organizations or societies, professional associations, and museums with archives. As recipients of the Program, the institutions carry out projects related to LAC’s mandate.

Target population

The outcomes of the Program are intended for all Canadians.

Description of activities

The funding provided by the Program has two main objectives. For each, categories have been defined to delineate the types of eligible projects:

• Objective 1. Raise awareness about and increase access to Canada’s local documentary heritage institutions and their collections:
  • Migration and digitization projects for access;
  • Creation (research, design and production) of virtual and physical exhibitions, including travelling exhibitions;
  • Collections, cataloguing and access-based management; and
  • Commemorative projects.
• Objective 2. Increase the capacity to preserve Canada’s documentary heritage in a more sustainable manner:
  • Migration and digitization projects for preservation purposes;
  • Restoration and preservation work;
  • Digital preservation capacity building (excluding digital infrastructure for day-to-day work);
  • Training and workshops to enhance skills and capacity; and
  • Development of standards, performance criteria and other evaluation activities.

Financial and human resources

The 2017–2018 audit found that the Program did not use its project code to track administrative expenditures (for example, wages). The audit team therefore worked with Program officials to estimate administrative costs. Tables 1 and 2 show the planned and actual resources allocated to the Program.

Table 1: Planned and Actual Spending, 2015–2016 to 2019–2020

		2015–2016	2016–2017	2017–2018	2018–2019	2019–2020
Planned spendingtable 1 note 1		$1,700,000	$1,700,000	$1,700,000	$1,700,000	$1,700,000
Actual spendingtable 1 note 2	Contributions	$1,500,000	$1,500,000	$1,500,000	N/A	N/A
	Wages and benefits	$194,880	$245,414	$345,717	N/A	N/A
	Other expenditures	$15,500	$15,500	$15,500	N/A	N/A
	Total	$1,710,380	$1,760,914	$1,861,217	N/A	N/A
Table 1 Notes Table 1 Note 1 Amount or number approved by Treasury Board, including wages and operating expenditures (2015). Return to table 1 note 1 referrer Table 1 Note 2 Estimated by the audit team in 2017–2018. Return to table 1 note 2 referrer

Table 2: Planned and Actual Full-Time Equivalents (FTEs), 2015–2016 to 2019–2020

	2015–2016	2016–2017	2017–2018	2018–2019	2019–2020
Planned FTEstable 2 note 1	1.20	1.20	1.20	1.20	1.20
Actual FTEstable 2 note 2	2.25	3.25	4.00	N/A	N/A
Table 2 Notes Table 2 Note 1 Amount or number approved by Treasury Board, including wages and operating expenditures (2015). Return to table 2 note 1 referrer Table 2 Note 2 Estimated by the audit team in 2017–2018. Return to table 2 note 2 referrer

2.3 Program governance

The management and administration of the Program is the responsibility of LAC’s Office of the Corporate Secretary. The governance of the program is the responsibility of internal expert advisers, the External Advisory Committee and the Office of the Librarian and Archivist of Canada.

Internal expert advisers include employees from several branches within LAC’s Operations Sector. They participate in the implementation of the Program by providing expertise in the evaluation of archival, library, digitization, preservation and client service funding applications.

The External Advisory Committee is composed of members recruited from outside LAC and drawn from diverse backgrounds in Canada’s documentary heritage communities. It evaluates funding applications and makes recommendations directly to the Librarian and Archivist of Canada, who approves funding applications.

The Program planned the use of regional ambassadors to support institutions in preparing their funding applications. Since January 2016, an ambassador is in place to assist institutions with projects related to the North and Indigenous peoples.

3. Methodology

3.1 Evaluation period and scope

Given that the first contribution agreements were signed in Fall 2015, the evaluation examined the first two and a half years of the Program, from April 1, 2015, to June 30, 2017.

In light of the short evaluation period, the analysis focused on the following processes and immediate outcomes:^footnote5

• Increase the capacity of local documentary heritage institutions to better preserve Canada’s documentary heritage; and
• Raise awareness about local documentary heritage institutions and their collections.

An analysis of progress toward the following intermediate outcome was also attempted to identify lessons or courses of action:

• Expand the use of the resources and collections of local documentary heritage institutions.

3.2 Evaluation questions

The evaluation questions examined the Program’s relevance and performance (effectiveness and efficiency). Specifically, the following questions were examined:

• Are the Program’s priorities aligned with those of LAC and the Government of Canada?
• Is the Program still relevant, and does it continue to address the evolving needs of clients?
• Is performance information sufficient and of good quality?
• Is the Program making progress towards immediate and intermediate outcomes?
• Is the Program using resources efficiently?

3.3 Evaluation methods

A literature review was conducted, which included the following:

• A review of administrative and financial documents, performance statistics, and other internal Program documents; and
• A review of the literature on documentary heritage institutions.

Interviews were conducted with managers and employees involved in managing and implementing the Program. Specifically, 15 interviews were conducted with staff members of the Office of the Corporate Secretary and other branches of LAC, and six members of the External Advisory Committee were interviewed.

The analysis was also based on a survey of 225 Program applicants and recipients. The survey was conducted online between December 11, 2017, and January 15, 2018. In all, 62 responses were received, for a participation rate of 28%.

The use of multiple survey methods and data triangulation (Table 3, below) helped to corroborate the findings. The methodology is in keeping with the Program evaluation mandate approved by LAC’s Departmental Program Evaluation Committee in June 2018.

An analysis of recipients’ websites and social media made it possible to gather information on the online availability of local collections.

Table 3: Evaluation Questions and Methods

Evaluation questions	Evaluation methods
	Literature review (internal and external)	Interviews	Applicant and recipient files	Websites and social media	Applicant and recipient survey
Relevance	Covered	N/C	Covered	N/C	Covered
Outcomes	Covered	Covered	Covered	Covered	Covered
Efficiency	Covered	Covered	N/C	N/C	N/C

The evaluation team adhered to policies, principles and procedures approved by the Canadian Evaluation Society and the Treasury Board. The key principles of independence and impartiality were ensured in particular by the following means:

• The design and conduct of the evaluation was led by a team separate from the Program’s management team.
• A random sample was used to select the files analyzed and the stakeholders interviewed.
• The accuracy of the statements recorded was ensured by having two evaluators present at all times during the interviews.
• The integrity of the responses to the applicant and recipient survey was maintained.
• Specific information sources are not named, to maintain confidentiality.
• The information was triangulated using at least two separate information sources to ensure data validity, accuracy and completeness.

3.4 Limitations of the evaluation

1. Performance information did not allowed linking Program funding to results achieved given that the majority of projects also received funding from other sources.
2. Data on medium- and long-term outcomes are insufficient because of the limited number of funding cycles evaluated. Further analysis is needed to determine whether any trends emerge from the results.
3. Detailed financial data by activity (for example, wages and travel costs) were not available. The analysis is based on the administrative costs estimated by the audit team^footnote6.

3.5 Coding of findings

The evaluation findings were categorized by colour to highlight improvements requiring special attention:

•  Green – no improvements required
•  Yellow – potential area for improvements
•  Red – improvements required

4. Findings – Relevance

4.1 Alignment with LAC and Government of Canada priorities

The Library and Archives of Canada Act mandates LAC to support the archival and library communities. The roles delegated to the Librarian and Archivist (section 8) include doing anything that is conducive to the attainment of the objectives of Library and Archives Canada, including providing professional, technical and financial support to those involved in the preservation and promotion of the documentary heritage and in providing access to it.^footnote7

Through the SDHCP, LAC provides funding to ensure that Canada’s continuing memory is documented and accessible for present and future generations by collaborating with local memory institutions. The Program is in line with LAC’s priorities as set out in its reports on plans and priorities (RPPs) over the past three years:

• Adopt a more collaborative approach to achieving the mandate and supporting documentary heritage communities (2015–2016 RPP).

• Be an institution with a strong public profile, showcasing its collections and services (2016–2017 RPP and 2017–2018 Departmental Plan).

The Program also supports the government’s priorities as it falls under the Social Affairs spending area of the Canada’s Whole-of-Government Framework, which aims to foster a vibrant Canadian culture and heritage. In its guidelines, the Program aims, among other things, to support the funding of projects related to Canadian culture and to improve knowledge of Canada’s history and cultural heritage.

4.2 Meeting the ongoing needs of Canadians

Through funding to local institutions, the Program aims to make documentary heritage more accessible and easier for Canadians to use. The applicant and recipient survey made it possible to poll institutions about their perceptions of the most important needs of their members and clients. As shown in table 4, online access to collections is the main need identified by respondents (61%), followed by the development of local collections (56%), support for research (47%) and in-person access to collections (42%). These needs were confirmed by the internal and external document review and interviews.

The data collected show that projects funded by the Program are aligned with the needs of Canadians. In the survey, recipients said that the funding provided by the Program contributed moderately or greatly to meeting the needs of Canadians.

Table 4: Alignment of the Program with the Needs of Canadians^{table 4 note 1}

Needs of Canadians	Applicants who consider this need to be very important	Recipients who consider the Program to be moderately or very responsive to the need
Online access to collections	61%	83%
Development of local collections	56%	74%
Research support	47%	76%
In-person access to collections	42%	68%
Access to permanent, temporary or travelling exhibitions	36%	24%
Showcasing diversity	25%	47%
Other	19%	15%
Table 4 Note Table 4 Note 1 Data collected as part of the survey conducted for this evaluation. Return to table 4 note 1 referrer

Digitization

A significant share of the projects funded involve the digital aspect of documentary heritage. For example, half of the projects involved conversion and digitization activities for access, and 45% involved conversion and digitization activities for preservation. Digitization is a very important need for 73% of the institutions surveyed. In interviews, External Advisory Committee members stated that Canadians would like to be able to access local collections online. They noted the increased demand for online content and said that Canadians need to be more connected to their heritage. One noted that, in the member’s experience, “Canadians have an attachment to their region, their streets, their world [translation].” The need for digitization is reflected in the data from the literature review:

• In 2016, 42% of Canadians said they had visited the website of a museum or heritage institution in the previous 12 months.^footnote8
• In 2015, heritage institutions’ websites received over 203 million visits, up 52% from 2011.^footnote9
• The number of online exhibitions increased 62% between 2011 and 2015, from 2,200 to 2,600.^footnote10
• Overall, 16% of the collections of memory institutions were digitized, while 10% of those were accessible online in 2015.^footnote11

The last figure above shows that there continues to be a need for digitization among documentary heritage institutions, given that only a small proportion of their collections is currently online.

Collection management and in-person access

In interviews, internal expert advisers and External Advisory Board members emphasized that Canadians need to know that local collections exist and that institutions manage and take care of them. This step is essential to eventually make the collections available online. When asked about the needs of their members and clients, institutions identified the following needs as very important: development of local collections (58%), research support (48%) and in-person access to collections (40%).

Once again, the data collected in the literature review corroborate these perceptions:

• 74% of visits to museum or heritage institution websites were in preparation for an in-person visit.^footnote12
• In 2016, 56% of Canadians visited a museum or science centre, compared with 51% in 2012.^footnote13
• The number of research requests increased 10.5% between 2011 and 2015.^footnote14
• 70% of Canadians believe that libraries contribute to their quality of life.^footnote15

Collection management and in-person access are needs for Canadians. In response to these needs, 58% of the projects completed in the first two cycles of the Program focused on management activities related to collections, cataloguing and access. In addition, 19% of the projects involved restoration or preservation work.

Exhibitions and commemorative projects

Access to permanent, temporary and travelling exhibitions was a very important need for 34% of the survey respondents. A smaller proportion of funded projects fell under this category, with 17% for the development of virtual and physical exhibitions, and 10% for commemorative projects. While this component is less significant than digitization and collection management, it is still a need for Canadians.

• The number of permanent exhibitions decreased since 2011. However, the number of travelling exhibitions increased by 51%.^footnote16
• 33% of Canadians participated in an event about local history and heritage in 2016.^footnote17
• In 2016-2017, 42% of Canadians believed that museums, public art galleries and heritage centres or sites contribute to their quality of life.^footnote18

Skills and abilities

To carry out digitization, collection management and access activities, heritage organizations rely on the skills and abilities of their staff and volunteers. They rely especially on volunteers: in 2013, volunteers accounted for 76% of the total workforce of heritage sector institutions. In the survey, 34% of respondents identified staff and volunteer development as a very important need for their institution. The Program funded numerous projects to address this. As such, during the first three funding cycle:

• 34% of funded projects were to provide training and workshops to enhance skills and capacity.
• 29% aimed to increase digital preservation capacity.
• 26% were related to the development of standards, performance criteria and other evaluation activities.

5. Findings - Performance

5.1 Performance measurement information

An evaluation framework was completed in December 2015, and performance measures were modified slightly to prepare for an evaluation initially planned for 2018–2019^footnote19. The Program was to collect baseline data in 2015-2016 and set performance targets starting in 2016–2017. In interviews, Program officials confirmed that they had not set performance targets for the indicators.

Funding applications

Funding application templates were developed to collect performance information. They are closely linked to the performance measurement strategy established when the Program was being designed. In interviews, Program officials stated that the templates are reviewed annually. The results of the applicant and recipient survey show that recipients are satisfied or very satisfied with the funding application templates (69%) and reporting templates (79%). All internal and external stakeholders suggested improvements, in particular the development of an electronic form. It should be noted, however, that the Program has already addressed this request by introducing a new online platform for the fourth funding cycle (2018–2019).

Reporting

An analysis of the Program’s administrative documentation shows that, during the first two financial cycles, all recipients submitted the required performance reports (interim and final) and that the data from the reports were collected and compiled by Program officials in Excel spreadsheets. However, Program officials stated that the information was not analyzed in the past but is planned to be analyzed in 2018–2019.

The evaluation team reviewed a sample of files to validate the robustness of the information provided by applicants and recipients. It was found that there were inconsistencies between the proposal, the signed contribution agreement and the final report. Recipients had difficulty distinguishing between outcomes attributable to the funded project and outcomes for their institution as a whole, such as the number of visits to their website or institution as a result of the project.

Measuring medium- and long-term outcomes is often a challenge for those in charge of contribution programs. In the case of the SDHCP, it is difficult to determine to what extent the outcomes are actually attributable to the Program. In the performance measurement strategy, data for measuring medium- and long-term outcomes are based solely on final reports from recipients. Since the reports are submitted before the end or at the end of projects, they provide a solid source of data to demonstrate outputs and outcomes in the short term; however, they are inadequate for measuring medium- and long-term outcomes. There is no mechanism in place to measure the impact of the Program after projects are completed.

In the contribution agreements, recipients agree to undergo evaluations or audits within the following five years after project completion. Medium-term follow-up with recipients to measure implementation or progress against defined objectives may be difficult and costly for both the Program and recipients. This must be considered in developing the performance strategy for a contribution program.

5.2 Effective use of contribution funds

According to Program guidelines for 2015–2016 to 2017–2018, the analysis and selection of eligible projects was based on project merit, program priorities, regional representation and availability of funds. In interviews, Program officials and External Advisory Committee members confirmed that the Program had not established priorities for allocating the $1.5 million contribution budget. Except for one member who appreciated the flexibility, the External Advisory Committee was unanimous on the need for clearer direction and guidance in this regard. Committee members felt that establishing priorities would make it easier to select projects for funding and would make decisions more objective and transparent. In the past, some funding decisions have been questioned by External Advisory Committee members, internal expert advisers, and some recipients and applicants.

The survey and interviews show that the needs of institutions are great, and they often have the opportunity to submit multiple projects but choose to focus on the one they feel is most likely to receive funding. Program management could influence the type of projects submitted by stating priorities at the outset of the process. As well, a clear statement of priorities would make it easier for the Program to set performance targets and measure results. Members of the External Advisory Committee suggested items that could be used to target annual funding priorities:

• Categories of projects;
• Types of institutions;
• Split between small and large contributions; and
• Number of multi-year contributions.

5.3 Program commitments

The Program made four commitments with respect to the use of resources^footnote20:

• Provide $1.5 million in contributions;
• Sign between 50 and 75 agreements per cycle;
• Provide an average of $20,000 in funding per project; and
• Allocate funding among projects as follows: 40% for Objective 1 projects and 60% for Objective 2 projects.

The data in table 5 show that the implementation of the Program has not met all of the initial expectations. The Program did meet its funding objective by providing $1.5 million in contributions. However, the commitments regarding the number of projects, the average funding provided, and the distribution between objectives 1 and 2 were not met.

Table 5: Achievement of Program Commitments

		Commitment	2015–2016	2016–2017	2017–2018
Signed agreements (per cycle)		50-75	64table 5 note 1	38	38
Average amount of funding		$20,000	$25,207	$50,229	$40,318
Distribution by objectivetable 5 note 2	Objective 1	40%	78%	76%	89%
	Objective 2	60%	68%	55%	74%
Table 5 Notes Table 5 Note 1 A total of 65 projects were accepted, but one institution withdrew and did not sign an agreement. Return to table 5 note 1 referrer Table 5 Note 2 A number of projects were related to both objectives. These objectives are described at page 9-10 of the present report. The Program was to fund more Objective 2 projects than Objective 1 projects. Return to table 5 note 2 referrer

Approved funding

LAC has made a commitment to allocate $7.5 million to the Program over five years, or $1.5 million per year. In the first three cycles, the Program approved total funding of $5,066,967 for projects totalling $8,308,929. In preparation for the fourth and fifth funding cycles, $580,396 has already been allocated for multi-year projects, leaving $2,419,604 for new projects in 2018–2019 and 2019–2020.

SDHCP’s financial contribution represented an average of 61% of the total cost of funded projects. The data show that the Program was not the sole financial contributor to the projects. Although 21% of the projects were more than 95% funded by the Program, recipients also received funding from other public or private sources. Nevertheless, the Program tends to assume a greater financial share of projects over time, with its average share increasing from 51% to 70% between the first and third cycles.

Table 12 in Appendix D contains information on the applications submitted and the funding granted.

Number of agreements signed

In the first three cycles of the Program, 141 projects were approved, or 44% of eligible projects. The number of projects was 65 in the first year and 38 in the following two years. The number of projects submitted remained relatively stable over that period, at roughly 140 per year. The Program did not meet its commitment of funding approximately 50 to 75 projects per year.^footnote21

Average amount of approved funding

The Program anticipated to allocate an average of $20,000 in funding per project. The average amounts allocated ($35,936) exceed the objective, which explains the lower-than-expected number of projects approved.^footnote22 The Program does not have a sufficient budget to approve all eligible applications. The median amount of funding is close to the original target of $19,072. As indicated in the Table 6 below, the average and median amounts of funding vary depending on the type of institution being funded.

Table 6: Average and Median Amounts of Funding by Institution Type^{table 6 note 1}

Institution type	Average amount of funding	Median amount of funding
Historical societies	$39,983	$15,638
Archives	$37,170	$15,000
Museums with archives	$35,050	$20,000
Professional associations	$33,820	$28,808
Librariestable 6 note 2	$20,061	$20,061
Genealogy associations	$10,497	$8,943
Other	$39,945	$35,000
All	$35,936	$19,072
Table 6 Notes Table 6 Note 1 A number of organizations self-identify in more than one category. Return to table 6 note 1 referrer Table 6 Note 2 The average and median funding are the same because two projects were approved. Return to table 6 note 2 referrer

Distribution between objectives 1 and 2

The Program guidelines state that projects must align with one or both of the objectives. The Program made the commitment to allocate more funding to Objective 2 that to Objective 1 as follow:

• 40% of funding to Objective 1 projects (raising awareness about and increasing access to Canada’s local documentary heritage institutions and their collections); and
• 60% of funding to Objective 2 projects (increasing the capacity to preserve Canada’s documentary heritage in a more sustainable way).

During the first three funding cycles, 31% of the projects carried out activities that fall under both program objectives. On average,  the data show that the Program funded more Objective 1 projects (81%) than Objective 2 projects (66%). However, an analysis of internal documentation and interviews with Program officials and External Advisory Committee members revealed that this commitment was not actually used for project selection.

5.4 Raising awareness about local institutions and their collections

Data from the annual reports of recipients in the first two cycles of the Program^footnote23 show that 75 projects were related to Objective 1 (raising awareness about and increasing access to Canada’s local documentary institutions and their collections).

Table 7: Proportion of Completed Projects with Activities Related to Objective 1, 2015–2016 and 2016–2017

Category of activities performed	Proportion completed
Management activities related to collections, cataloguing and access	58%
Conversion and digitization for access	50%
Development of virtual and physical exhibitions, including travelling exhibitions	17%
Commemorative projects	10%

Collection management, cataloguing, conversion and digitization for access are a large part of the projects completed. The implementation of these activities by recipients has demonstrated that these efforts have, to some extent, increased the visibility and availability of their collections to the public (see table 7).

The following are some data on achievements from recipient reports. Some institutions have

• Made their collections available to researchers and the public and extended the accessibility of their collections beyond their physical spaces;
• Increased their visibility and generated new interest in their collections on their websites and social media;
• Generated media interest and raised public awareness about local archives;
• Organized physical and virtual exhibitions of publicly accessible collections and archival holdings; or
• Participated in a commemorative event.

The recipient survey shows that 62% of participants believe that the Program significantly helps raise awareness about local documentary heritage institutions and their local collections.

An analysis of the social media and websites of funded institutions, performed as part of this evaluation, also supported these statements. Some of the examples provided show that, even in a short time, some institutions have seen increased visits to their websites and increased interest in new collections on their social media. In the first two cycles of the Program (which included 75 projects linked to Objective 1), 61% of institutions put their collections online, and 52% referred to them on their website or social media. It should be noted that not all projects were aimed at making their collections available online. Furthermore, 11 of the 75 projects are multi-year projects, and their outcomes were not available during the evaluation period.

5.5 Increasing the capacity of local institutions to preserve documentary heritage

A total of 71 projects involved activities related to Objective 2 (increasing the capacity to preserve Canada’s documentary heritage in a more sustainable way).

Table 8: Proportion of Completed Projects with Activities Related to Objective 2, 2015–2016 and 2016–2017

Activities performed	Proportion completed
Conversion and digitization for preservation	45%
Training and workshops to enhance skills and capacity	34%
Increasing digital preservation capacity	29%
Development of standards, performance criteria and other evaluation activities	26%
Restoration and preservation work	19%

An analysis of information from recipients’ annual reports shows that the activities performed have indeed helped to increase institutional capacity to some extent, whether through training workshops, the hiring of digital preservation specialists, staff and volunteer training, the use of better work methods, or other activities (see table 8).

Here are some data about achievements from recipient reports. Some institutions have

• Reduced backlogs of archives that had long been accumulating under poor conditions and were at risk of being lost;
• Digitized a large amount of material (photos, documents, newspapers, postcards, microfilm, and audio and video tapes);
• Conducted workshops (methods and techniques for archiving analogue and digital collections, copyright, and preserving audiovisual collections);
• Initiated the migration of archival holdings to higher-performance digital preservation tools;
• Fostered the development of new skills and competencies among staff and volunteers;
• Updated internal collection management policies, conservation procedures and best practices; or
• Provided temporary employment for students and specialized staff.

Of the recipients who responded to the survey, 86% felt that the assistance provided by the Program significantly helped increase the capacity of institutions to preserve local collections. Program staff concurred.

An analysis of the projects shows that a number of projects (44/140) performed activities in categories belonging to both Objective 1 and Objective 2.

5.6 Expanded use of local collections

Final project report data and Program information are inadequate to assess the medium- and long-term impacts of the Program. Recipient reports are submitted before the end or at the end of the project. No follow-up is done in the months or the year after the end of the project. During the survey, the evaluators were able to gather anecdotal information on the public’s use of the collections, but this did not lead to any solid findings.

The perception of Program officials is mixed as to the effects of the Program, given its degree of maturity. It was noted, however, that the Program has been very well received by the documentary heritage community.

5.7 Other outcomes

According to a Canadian Heritage survey^footnote24, heritage institutions rely heavily on precarious part-time jobs (18,900, versus 12,500 full-time jobs) and volunteers (115,650). Volunteers account for more than 76% of the sector’s total workforce.

These data are consistent with the findings of an expert committee^footnote25 that raised the importance of public participation. The experts found that the public in Canada and abroad is playing an increasingly important role in the conservation of Canada's cultural heritage, whether through volunteer work, research, the documentation of items stored digitally, or online evaluations.

Institutions that submitted projects to the Program confirmed that volunteer recruitment is a significant need. In the applicant and recipient survey, 75% of respondents felt that recruiting volunteers was important or very important. The data^footnote26 show that the employee-to-volunteer ratio of funded institutions is 1 to 4^footnote27.

Funded institutions rely heavily on volunteers. Approximately 30% of surveyed recipients said that the Program contributed moderately or greatly to volunteer recruitment. Further analysis would validate this outcome and provide a better understanding of volunteer involvement in local community projects.

6. Findings - Efficiency

The Program’s efficiency was assessed to answer four questions:

• Are there overlaps or similarities with other federal government programs?
• Does the Program fund targeted institutions?
• Are the administration costs of the Program acceptable?
• Is the Program delivered efficiently?

6.1 Overlaps and similarities with other programs

An analysis of available financial support offers reveals some overlaps, mainly with two other federal government programs, namely the Museums Assistance Program and the Aboriginal Peoples’ Program.

Museums Assistance Program

Aside from one component in the France-Canada Agreement, the objective and outcomes of the Museums Assistance Program are identical to those of the SDHCP.

The Museums Assistance Program funds projects to

• Preserve and present heritage collections.
• Preserve Indigenous cultural heritage and facilitate access to heritage collections for all Canadians.
• Foster the development of professional knowledge, skills and practices.

The type of projects funded is also the same, namely travelling exhibitions, heritage preservation and presentation, collection management, and the enhancement of professional skills and practices.

However, in the case of the Museums Assistance Program, only not-for-profit Canadian museums, museum associations, Canadian Indigenous service institutions and Indigenous governing bodies are eligible. Furthermore, that program works differently, allocating both grants and contributions, and funding between 50% and 70% of eligible project expenditures.

The SDHCP is open to a broader range of institutions, including not-for-profit museum sector institutions (which are also eligible for the Museums Assistance Program). It also funds projects up to 100%. Of the institutions funded, 26% were museums, which together received 25% of the total funding allocated in the first three cycles of the Program.

There is overlap between the two programs, since museums and museum institutions can obtain funding from two separate federal programs that have very similar objectives, types of projects and modes of operation. For example, some projects funded by the LAC program have carried out activities eligible under the Museums Assistance Program.  Here are three examples:

• One museum received funding to evaluate and plan options for the storage of its archives. The Museums Assistance Program - Collection Management Component includes projects related to collection storage solutions.
• One museum received funds for the migration and updating of its archive database. The Museums Assistance Program - Collection Management Component includes projects related to the implementation or upgrade of a collections management system.
• An Aboriginal governing body received funding for the development of an exhibition. The design, production, promotion and presentation of exhibitions and related interpretive material are eligible activities of the Museums Assistance Program - Aboriginal Heritage Component.

This overlap was raised in the interviews and the survey. In addition, the participants questioned whether the museum sector should be included in the SDHCP when the Program cannot meet the demand from other types of institutions.

Aboriginal Peoples’ Program

The Aboriginal Peoples’ Program has the following objectives:

• Promote, revitalize and preserve Indigenous languages and cultures.
• Strengthen Indigenous cultural identity.
• Increase Indigenous participation in Canadian society.

It offers two funding components to eligible Indigenous organizations: Aboriginal Languages Initiative and Northern Aboriginal Broadcasting. There is overlap between the SDHCP and the first component (Aboriginal Languages Initiative), which supports the preservation and revitalization of Indigenous languages through community projects and activities. Funded projects include the recording, documentation and preservation of endangered Indigenous languages; organizations may receive contributions ranging from $25,000 to $150,000.

However, there is one important difference regarding eligible recipients: the Aboriginal Peoples’ Program targets a wide range of organizations with an interest in Indigenous issues, whether or not they are documentary heritage institutions. The activities funded are more diverse than for the SDHCP. Although a given project may be eligible for funding under both programs, such situations are rare.

6.2 Scope of funded institutions

In its Treasury Board submission, LAC stated that consultations with its partners had shown that local and community institutions and associations were having difficulty obtaining funding and that the contributions provided should support the development of local libraries and archives in Canada. It also stated that the Program would focus on local documentary heritage so that community institutions, incorporated or unincorporated, could initiate major projects for their regions. Finally, it added that national associations could launch horizontal projects that would benefit local heritage institutions. The very essence of the Program was adequately captured in determining the objectives of the Program:

• Raise awareness about and increase access to Canada’s local documentary heritage institutions and their collections.
• Increase the capacity of local documentary heritage institutions to preserve Canada’s documentary heritage in a more sustainable way.

However, External Advisory Committee members, internal expert advisers, applicants and recipients questioned the definition of “local institution.”^footnote28 Some External Advisory Committee members and advisers were surprised that certain institutions were considered eligible. Several expressed a desire to see a greater number of small local institutions obtain funding. The applicant and recipient survey also revealed a major concern, namely the perceived under-representation of small institutions. Although there were no specific questions on that topic, many participants took advantage of the open-ended questions to express their concerns in that regard.

The following are some profile data for institutions funded by the Program:

• In the first three cycles, 58% of funded institutions stated in their funding applications that they were local, municipal or regional in scope. The remaining 42% stated that they were provincial, national or international in scope.
• The median annual spending budget of funded institutions has doubled since the first cycle.^footnote29
• In 2015–2016, 11% of funded institutions had an annual spending budget greater than $1 million. In the last two cycles of the Program, roughly one in five institutions had a budget of this size (22% in 2016–2017 and 18% in 2017–2018).
• Two thirds (66%) of funded institutions had fewer than five employees, and 52% had more than 10 volunteers.

Having a spending budget over $500,000 does not mean that the institution is not local and not eligible for the Program. However, the data show that the Program funds a large number of institutions whose scope is reported as provincial or greater, and institutions whose spending budgets have increased steadily over the past three years.

6.3 Efficient use of internal resources

An analysis of internal documentation and interviews with Program officials showed that salaries, other administrative expenses, and the number of full-time and part-time employees were not accounted for in the financial codes assigned to the Program. This has been an impediment to an accurate and complete analysis of administrative costs and operating expenses. For the purpose of the analysis, an estimation of actual costs was undertaken in collaboration with Program officials.^footnote30

As indicated in the Program’s initial budget and 2018 audit report, annual administrative expenses were set at $200,000, which included Program staff wages, information technology costs, external consultations, travel and miscellaneous costs. To this amount was added 1.2 full-time equivalents (FTEs) to manage the Program for five years.

As indicated in section 2.3 of this report the Program spent a total of $5,332,510 in its initial three funding cycles. During that period, administrative expenses amounted to $832,510, equivalent to a three-year average of 15.61%. Administrative costs have risen steadily: in 2017–2018, it cost three times as much to manage half as many projects as in 2015–2016. The number of FTEs varied from 2.25 in 2015–2016 to    3.5 in 2016–2017 and 4 in 2017–2018.

Table 9: Ratio of Administrative Expenses to Total Program Expenditures

	2015–2016	2016–2017	2017–2018	2018–2019	2019–2020
Administrative expense ratio	12.3%	14.8%	19.4%	N/A	N/A

Table 10: Program Administrative Costs per Application Received and per Agreement Signed

Year	Number of applications received	Number of agreements signed	Total administrative costs	Cost per application received	Cost per agreement signed
2015-2016	136	65	$210,380	$1,547	$3,237
2016-2017	155	38	$260,914	$1,683	$6,866
2017-2018	129	38	$361,217	$2,800	$9,506
Total (3 years)	420	141	$832,510	N/A	N/A
Average (3 years)	N/A	N/A	N/A	$1,982	$5,904

As part of the evaluation, a comparative analysis (see chart 1) was performed against other small cultural contribution programs.^footnote31 It showed that the Program’s administrative expenses are comparable to those of other similar programs, which are generally between 15% and 20%.^footnote32

Source: Appendice E - Comparison of Administrative Expenses, 2015 to 2018.

Chart 1: text version

Table 11: Administrative Expense Ratios per Programs

	SDHCP	Canadian Heritage Portfolio			Small Contribution Programs
		MAP	APP	BCAH	CPP	JPIP	KPRI	CWMP	PDCP
Administrative Expense Ratios	19.4%	21.9%	14.9%	18.7%	19.0%	5.0%	13.0%	11.0%	21.0%

The analysis performed as part of this evaluation shows that administrative expenses were higher than anticipated when the Program was created. Program officials cited new work such as creating templates, establishing relationships with applicant institutions, doing promotional activities and maintaining a website as reasons for the discrepancy. The planned number of FTEs was clearly insufficient from the outset. Other factors also affected human resource requirements, including staff turnover and inexperience in managing transfer payments. Program officials said that further improvements are needed to achieve stability.

Lastly, administrative costs remain within acceptable limits in comparison with other programs. Nevertheless, analyses show that they have steadily increased. Adequate financial coding and monitoring measures would control the costs in future cycles.

6.4 Program delivery method

In general, respondents felt that the Program was effective and efficient and that its templates were straightforward and very useful for institutions that did not have access to other sources of funding. The general comments received were well summarized in the testimony of one recipient: “The Program is very responsive to needs ... like ours and is consistent with new methods ... being used now. The Program is clear and well-explained by the people at LAC, and we feel supported in our endeavours [translation].”

Funding applications

Applicants were satisfied (60%) or very satisfied (21%) with the funding application process. They were particularly satisfied or very satisfied with the following:

• Availability of tools and information in the official language of their choice (97%);
• Clarity of information on documentation requirements (89%);
• Clarity of guidelines (87%);
• Quality of information and tools provided (82%); and
• Clarity of eligibility criteria (80%).

However, they were slightly less satisfied with the ease of use of the funding application templates (69%). Some applicants also stated that they contacted Program officials to clarify certain points and obtain more information to complete their funding applications. Although some of them had difficulty reaching staff members, they noted the valuable assistance they received. Respondents pointed out that Program officials were proactive in contacting applicants before the application deadline to obtain any additional or missing information.

Reporting

Recipients were also pleased with the reporting process. For example, they were satisfied or very satisfied with the following:

• Clarity of reporting guidelines (92%); and
• Ease of use of reporting templates (78%).

Both applicants and recipients were satisfied with the Program’s current method of delivery. Recently, Program officials began to review the funding arrangements for the renewal of the Program. Options are being explored.

7. Other observations

7.1 Showcasing diversity

LAC made the commitment to provide funding to institutions across all the regions of Canada. The members of the External Advisory Committee confirmed that the geographical location of the institutions, while not an evaluation criterion, was nevertheless a factor in the selection of projects.

As shown in table 12, the data confirm that the number of eligible applications and funded projects is distributed in a balanced manner among provinces and territories. However, they highlight the low proportion of applications from Alberta and Ontario, given their demographic weight. As well, Quebec is somewhat overrepresented, in terms of both the number of projects funded and their proportion in relation to the rest of the country.

Table 12: Geographical Representation of Projects Funded Between 2015 and 2017

Geographical Location	Proportion of eligible applications	Proportion of projects funded	Proportion of Canadian population
Territories (Yukon, Nunavut and Northwest Territories)	4.1%	3.9%	0.3%
British Columbia	16.2%	19%	13%
Alberta	6.3%	6.5%	12%
Saskatchewan	3.8%	3.9%	3%
Manitoba	6%	5.2%	4%
Ontario	24.1%	21.6%	39%
Quebec	29.2%	30.7%	23%
New Brunswick	2.2%	2.6%	2%
Nova Scotia	2.9%	2.6%	3%
Prince Edward Island	1.3%	0.7%	0.4%
Newfoundland and Labrador	3.8%	3.3%	1.4%
All	100%	100%	100%

The Program did not set formal criteria for or against projects based on linguistic, cultural or ethnic aspects. Overall, funded projects showcase diversity among Canadians.

In particular, there were projects involving the following:

• Indigenous peoples (8);
• Women (3);
• Religious heritage (18);
• Cultural and ethnic communities (6);
• Minority-language communities (2); and
• The LGBTQ+ community (1).

8. Conclusion and recommendations

8.1 Conclusion

The purpose of LAC’s SDHCP is to provide funding in the form of contributions to local documentary heritage institutions to increase their capacity to preserve their collections and to promote and make them available to Canadians.

An analysis of the data confirms that the Program’s service offerings meet the needs of Canadians, as funded projects help institutions to preserve their documentary heritage and to make it accessible, whether through cataloguing, training, preservation or conservation activities, or by digitizing and making their collections available online.

The Program supports LAC’s mandate and strategic outcomes by supporting documentary heritage communities. It also supports the Government of Canada’s priorities by funding cultural projects and increasing knowledge of Canadian history and heritage.

The Program is new, and SDHCP officials have made efforts to meet the needs of applicants and recipients in the first three cycles of its implementation. In particular, the Program has provided tools to assist applicants and recipients in completing funding applications and preparing reports.

Overall, the Program is achieving its short-term outcomes. Evidence shows that the projects launched in the first two cycles have been completed and that many of the collections processed are available in person at the institutions, or on their websites.

Collection management, cataloguing, conversion and digitization for access are a large part of the activities undertaken. Their implementation by recipients demonstrates that they have increased the visibility and availability of collections to the public.

An analysis of the information shows that the activities performed have indeed helped to increase institutional capacity, whether through training workshops, the hiring of digital preservation specialists, staff and volunteer training, the use of better work methods, or other activities.

In terms of efficiency, the evaluation found that there is overlap between Canadian Heritage’s Museums Assistance Program and the SDHCP: museums and museum sector institutions may receive funding from two separate federal programs that have very similar objectives, types of projects and modes of operation. In addition, the data show that the Program funds a large number of institutions whose scope is reported as provincial or greater, and institutions whose spending budgets have increased steadily over the past three years.

Several institutions expressed a desire to see a greater number of small local institutions obtain funding. All evaluation participants also questioned the definition of “local institution.”

Although the administrative expenses of the Program have increased steadily from the outset, they are comparable to those of other federal government contribution programs of similar size.

Halfway through, the Program is at a pivotal stage. The evaluation results provide SDHCP management with an opportunity to make some adjustments to its scope, management and performance measurement. The definition of “local institution” should also be addressed, as it influences the scope and outcomes of the Program.

8.2 Recommendations

The evaluation of the SDHCP has led to the following four recommendations:

• Recommendation 1: The Program should review its scope, logic model and performance measurement strategy, and set annual performance targets for all indicators.
• Recommendation 2: The Program should clarify and communicate its priorities for each funding cycle.
• Recommendation 3: The Program should examine overlaps with the Museums Assistance Program to determine whether the SDHCP needs to make adjustments or adopt a complementary approach.
• Recommendation 4: The Program should clarify the definition of “local institution” and include it in its guidelines.

​