A Science-Metrix Inc. evaluation of the Space Astronomy Missions (SAM) and Planetary Missions (PM) programs for the Canadian Space Agency’s (CSA) Audit and Evaluation Directorate offers mixed results.
The completed report was the first of its kind, and covers the period from April 1, 2011 to the end of March 2016, a five year period. The report was “conducted in accordance with the 2016 Treasury Board of Canada’s Policy on Results and addressed the evaluation issues of relevance, effectiveness and efficiency.”
The reports states that “in carrying out this evaluation, a participatory approach was employed that entailed the use of mixed quantitative and qualitative methods. Specifically, document and archival data reviews were performed, e-surveys were administered to 41 funding recipients, and key informant interviews were conducted with 24 CSA representatives, one other government department, and 3 international partners. Two case studies were completed, one for each of the two programs under evaluation. The focus of these case studies was Canada’s contribution to NASA’s James Webb Space Telescope (JWST) mission and the Mars Science Lab (MSL)/Curiosity rover mission.”
The report notes that in terms of relevance, the SAM and PM programs are highly relevant and meet the CSA’s mandate and strategic outcomes.
The same is mostly true with respect to the effectiveness of SAM and PM programs, stating that they “are very cost-effective and have achieved notable accomplishments which have contributed to Canada’s reputation in the international space exploration community.”
However, the evaluation also found that with respect to effectiveness “that, at times, the engagement of the science teams was delayed due to a lack of grant funding at start-up and fragmented funding for scientific research during and after missions. The program should develop clear guidelines that clarify the definition and scope of science support to missions in order to allow for continuity in science support funding throughout all phases of a mission from pre-definition phase to post-operation activities.”
It also that noted “that the irregular cadence of the SAM & PM programs has resulted in human and financial management challenges for industry partners and universities researchers, and uncertainty among international partners regarding Canada’s financial commitments to ongoing and proposed missions. The enviable reputation that Canada has established over many years with its signature contributions to international joint ventures and high-profile space exploration missions may have been affected over the years due to the lack of investments.”
This is point that the community has been making for some time and was the subject of an opinion posted to SpaceQ by a couple members of the community a few months ago.
With respect to efficiency, “the evaluation found that the program was very efficient with regards to leveraging and cost- efficiencies.”
One example where government inaction resulted in a lost opportunity was Canada’s possible contribution to NASA’s 2020 lander.
The report states “based on the collected group interview data, NASA had expressed an interest in a possible Canadian contribution to the mission in the form of a science instrument or engineering subsystem. The APXS was repeatedly referred to in the 2013 Science Definition Report and was included among the scientific instruments in the ‘strawman’ or hypothetical payload. Based on corroborating interview data from different stakeholders, the CSA recognized the importance of this mission to the Canadian planetary science community and through recommendations made by the Planetary Exploration Consultation Committee. The CSA then published a competitive Request for Proposal that the MDA/UoG APXS team won, and in 2015 it went on to submit their costed proposal of approximately CAN $12 million to NASA for the Mars 2020 competition. However, the proposal was submitted without an endorsement letter assuring financial support from the CSA, as the Government of Canada approvals were not forthcoming. Although Canada was given every opportunity to participate in what might be an historic mission to Mars that may discover evidence of life on another planet, NASA was obliged to select another scientific instrument from another country with slightly different capabilities. The opportunity to secure Canada’s preferred partner status with NASA for the Mars 2020 mission was not pursued.”