Summary - Evaluation of the Supporting the Documentary Heritage Communities Program, 2015–2016 to June 2017
Brief overview of the program
The
SDHCP was launched by LAC following a Cabinet decision in 2015 approving the reallocation of a portion of its funds for the creation of the program in May 2015. The purpose of the Program is to provide funding in the form of contributions to local documentary heritage institutions, to increase their capacity to preserve their collections and promote and make them available to Canadians.
The Program’s activities aim to achieve four outcomes:
- Increased capacity of documentary heritage institutions to preserve Canada’s documentary heritage in a more sustainable way (immediate);
- Better knowledge of local documentary heritage (immediate);
- Greater use of the resources and collections of local documentary heritage institutions (intermediate); and
- Expanded access to Canada’s local documentary heritage for Canadians (ultimate).
The Program is open to local documentary heritage institutions, including archives, privately funded libraries, historical societies, genealogical organizations or societies, professional associations, and museums with archives.
Questions and evaluation results
-
Question 1: Do the priorities of the Program align with the priorities of LAC and the Government of Canada?
Result: Yes. The
SDHCP supports LAC’s mandate and strategic outcomes by supporting documentary heritage communities. The Program provides funding to ensure that Canada’s continuing memory is documented and accessible for present and future generations. The Program also supports government priorities through its integration into the Government of Canada’s Whole-of-Government Framework.
-
Question 2: Is the Program still relevant and responsive to changing client needs?
Result: Yes. The data collected show that projects funded by the Program are aligned with the needs of Canadians. For 73% of the institutions surveyed, digitization is a very important need. Internal and external respondents emphasized that Canadians need to know local collections exist and that institutions manage and take care of them.
-
Question 3: Is performance information sufficient and of good quality?
Result: Partially. Data collection for measuring medium and long-term outcomes are solely based on interim and final reports from recipients. These reports provide a solid source of data to demonstrate outputs and outcomes in the short term; however, they are inadequate for measuring medium and long-term outcomes. There is no mechanism in place to measure the impact of the Program after projects are completed.
-
Question 4: Is the Program making progress towards immediate and intermediate outcomes?
Result: Partially. Overall, the Program is achieving its short-term outcomes. Evidence shows that the projects launched in the first two cycles were completed and that many of the collections processed are available on site or on institutions’ websites. Projects implemented by recipients demonstrate that they have increased the visibility and availability of collections to the public. Furthermore, the evaluation showed that the activities performed have helped to increase institutional capacity, whether through training workshops, the hiring of digital preservation specialists, staff and volunteer training, the use of better work methods, or other activities. However, it was too early to assess progress made to achieve the medium-term outcomes. Lastly, the evaluation demonstrated that the Program had not established priorities.
-
Question 5: Did the Program demonstrate efficiency?
Result: Partially. The evaluation found that there is overlap between Canadian Heritage’s Museums Assistance Program and the
SDHCP. Administrative costs remain within acceptable limits in comparison with other programs. Nevertheless, analyses show that they have steadily increased. Adequate financial coding and monitoring measures would control the costs in future cycles. Lastly, the data showed that the Program funds a large number of institutions whose scope are reported as provincial or greater, and institutions whose spending budgets have increased steadily over the past three years. These factors contributed to questioning the definition of “local institution.”
Recommendations and management response
-
1. The Program should review its scope, logic model and performance measurement strategy, and set annual performance targets for all its indicators.
Management response
The
SDHCP will review the program logic model and performance measurement strategy in relation to the Departmental Results Framework. It will establish annual performance targets for all performance measurement strategy indicators and put in place data collection mechanisms to assess medium and long-term results.
-
2. The Program should clarify and communicate its priorities for each funding cycle.
Management response
The
SDHCP will better define the factors that guide funding decisions and update the guidelines on LAC's website.
-
3. The Program should examine overlaps with the Museums Assistance Program to determine whether the
SDHCP needs to make adjustments or adopt a complementary approach.
Management response
The
SDHCP will compare the terms and conditions of the two programs and adapt the terms and conditions of the Program as necessary to ensure that it complements the Museums Assistance Program.
-
4. The Program should clarify the definition of “local institution” and include it in its guidelines.
Management response
The
SDHCP will develop a clear definition of "local institution" in the context of the program and update the program guidelines on the LAC website.
Evaluation methods
The following qualitative and quantitative methods and data triangulation were used in the evaluation:
- Review of external studies, reports and surveys;
- Review of internal administrative documentation;
- Review of applicant and recipient files (funding applications, financial reports and recipient performance reports);
- Interviews with 15 LAC employees;
- Interviews with six members of the External Advisory Committee;
- Survey of 225 Program applicants and recipients, with a response rate of 28%; and
- Analysis of recipients’ social media and websites.
Limits
- Performance information did not allowed linking Program funding to results achieved given that the majority of projects also received funding from other sources.
- It was not possible to assess medium term outcomes.
- Detailed financial data by activity (for example, wages and travel costs) were not available. The analysis is based on the administrative costs estimated by the audit team.