Impact Evaluation Consultant

Job description

II. DESCRIPTION OF SERVICES:

IREX is seeking proposals from qualified firms or individuals to conduct an impact study to assess the long-term outcomes of the UASP and evaluate efficacy.

The evaluation will employ statistical matching, or another rigorous quasi-experimental impact evaluation methodology, to estimate the causal impact of the UASP on key individual-level and system-level outcomes (as outlined in the Scope of Work below). Preference will be given to proposals with a rigorous identification strategy and a methodology to identify a credible counterfactual.

The applicant is expected to propose the data collection strategy. Data collection tools might include (but are not limited to):

• Alumni tracer surveys to capture career trajectories, networks, and behavioral changes since participation

• Comparison group surveys with similar retrospective recall questions

• Institutional data audits on changes in RM structures, staffing, and policies

• Qualitative interviews and focus groups with alumni, peers, and leadership at selected institutions

• Case studies of institutions with high alumni concentrations, using timeline mapping, document review, and stakeholder interviews

• Network mapping through surveys/interviews to assess internal networks and peer influence

Scope of Work

The impact evaluation aims to address the following questions:

1. Evaluation Objectives

• Aim 1: Assess the long-term impact of USAP on research management (RM) practitioners’ practice, careers and networks

• What is the effect, if any, of UASP participation on RM practitioners’ professional practice?

• What is the effect, if any of UASP participation on RM practitioners’ membership of interest groups and/or professional credentialling?

• What is the effect, if any, of UASP participation on RM practitioners’ career pathway?

• What is the effect, if any, of UASP on RM practitioners’ networks?

• Aim 2: Explore the long-term impact of UASP on institutional-level changes in RM practices

• What is the effect, if any, of UASP on institutions’ RM policies, procedures and organizational structures?

• What is the effect if any, of UASP on institutions’ grant competitiveness and research outputs?

• Aim 3: Explore the role of critical mass in amplifying institutional transformation.

• To what extent, if at all, does the effect of UASP participation vary depending on the number of UASP alumni?

• In what ways, if any, does alumni density impact individual-level impacts?

• In what ways, if any, does alumni density impact institutional-level changes in RM?

Under the direction of, and with support from, the UASP Program Team, the selected individual or firm will be expected to work closely with UASP alumni and institutions to gain inputs and buy-in to the final impact evaluation design, and to identify and secure access needs and approvals.

It is anticipated that the selected evaluator(s) will travel to a dissemination event in Africa in early September or late October 2025, which will present an opportunity for in-person engagement with approximately 40 alumni.  Additional travel may be considered depending on overall budget (see guidelines under Price Proposal below).

III. Deliverables:

The selected firm will be expected to perform and deliver the below:

1. Inception Report

The evaluation design and plan will be shared through an inception report to the UASP Program Team. The Team will provide feedback on the inception report following the presentation, which should be incorporated into the final workplan. The inception report should include the following:

• Evaluation Design: The selected researcher/firm is expected to develop an evaluation design using statistical matching, or another rigorous quasi-experimental impact evaluation methodology.

• Data Collection Plan: The data collection plan should include: (a) plans for creating, piloting, and finalizing data collection tools, (b) address any ethical concerns around data collection, © plans to manage data collection and ensure data quality. The selected researcher/firm is expected to manage all data collection and analysis related to this project in coordination with the UASP Program Team.

1. Data Collection Tools

The tools can include (but are not limited to): surveys, interview and focus group questionnaires/guides, institutional data audits, and/or other assessments.

1. Data Validation Workshops

The selected individual/firm is expected to prepare and facilitate at least 1 workshop with representatives from the IREX UASP Team and select alumni. This will be aligned with a key stage in the evaluation e.g., data analysis, findings and recommendations.

1. Final Report

The final report should include an executive summary, background and introduction, methodology, findings and analysis, conclusions and recommendations.

1. Final Report Presentation

The selected individual/firm is expected to present a summary of the report and its main findings to the team in a PowerPoint format.

The selected individual/firm is expected to present the full draft report to the IREX-UASP Program Team. It is expected that the report will need another round of review incorporating comments.

1. Raw Dataset

The selected individual/firm is expected to create and share a cleaned, de-identified raw dataset with the Program Team

All deliverables must be submitted in English. The selected individual/firm should expect a cycle of review, feedback, and response for each deliverable.

IV. Proposal Submission requirements:

The IREX UASP Program Team reserves the right to reject all offers, to add, delete, choose item/items or services, and modify any element of the solicitation at any time without prior notification and without any liability or obligation of any kind. This request for Proposal (RFP) does not obligate IREX to enter a contract nor does its obligate IREX to pay any costs incurred in the preparation of submission of any Response.

Proposal should be submitted by email which contains two separate attachments:

1. The Technical Proposal;

2. The Financial Proposal.

Technical Proposal:

The Technical Proposal should include the following:

1. A detailed technical approach outlining the methodology for how the evaluation will be conducted. The technical approach should include details of the steps the researcher/firm will take to establish a credible counterfactual group (as applicable). A work plan with timelines for each task and deliverable, with work anticipated to begin the first week of July 2025 and conclude by March 1, 2026. The workplan should factor in an in-person session with alumni in Africa during September or October of 2025.  The final report draft and PowerPoint should be submitted at least one month before the end of the project to allow time to receive and incorporate feedback.

2. Information on the qualifications and experience of the proposed team members.

3. Past performance to include information on similar evaluations conducted by the individual or firm.

The Technical Proposal should be limited to 10 pages, excluding attachments.

Financial Proposal:

1. Total amount of funds available is $50,000. Budget ceiling should not exceed this amount and must include one trip to East Africa for 5 nights in late October. If vital to the project design, additional travel within Africa may be budgeted for under this ceiling.

2. Price proposal must be valid for a minimum of forty (40) days.

Proposal should be submitted online with the following documents uploaded:

Submission Deadline Information:

For best consideration, vendors must submit their application documents to IREX by email no later than Friday, July 11, 5:00 PM EST. The deadline for submission may be changed at any time in the sole discretion of IREX.

• Release of RFP: June 16, 2025.

• Bidders’ Questions submitted by email to [email protected] by June 23, 2025, 9:00 AM EST.

• Answers to questions shared with all bidders by email: June 25, 2025, by 5:00 PM EST.

• Submission deadline to [email protected] on July 11, 2025, 5:00 PM Eastern Standard Time.

Point of contact for questions: Louisa Manzeske, [email protected]

V. Period of performance:

The implementation period will be effective from date of signing contract (anticipated beginning of August, 2025) through March 1, 2026 (all deliverables must be submitted before February 1, 2026, with a one-month period to respond to questions from IREX).

VI. ELIGIBILITY CRITERIA:

To be eligible to participate in this competitive solicitation, individuals or firms must provide the requested below as part of the technical proposal:

1. Past performance demonstrating at least 7 years of previous experience working on impact assessments.

2. Reference List of at least 3 clients to include (Project Name, Size, Contact Name, Contact Number).

Vll. Evaluation CRITERIA:

The UASP Team anticipates one award for a firm fixed price contract to implement this impact study. IREX will evaluate Proposals based on the following criteria:

Evaluation Criteria

Percentage (100%)

Technical approach and methodology

30%

Experience with similar evaluations

25%

Qualifications of the team

15%

Price

25%

Up to three shortlisted firms scoring 70% or above may be called for a virtual presentation to determine the winning bidder.