We’ve noticed you’re visiting from NZ. Click here to visit our NZS site.
Independent lapsing funding evaluations that show if your program is still needed
Whether it's your board, your funder or the Minister deciding on your program's next round of funding, they'll want to know the same three things.
- Did the program achieve what it set out to do?
- Was the investment value for money?
- Should the funding continue?
We run the lapsing program evaluation that answers those questions with evidence from your program's data and from what participants, staff and stakeholders tell us.
-
- For a Victorian health program near the end of its initial funding, our evaluation showed worthwhile outcomes achieved, efficient use of resources, and an ongoing need for the program.
-
- For a small program, a rapid evaluation built on documents and interviews may answer every required question. A statewide program may also need analysis of service data, focus groups with staff and the sector, and an economic analysis.
-
- Many of our team members have worked inside government, and have commissioned and managed lapsing evaluations. We know what the people who commission an evaluation expect from it, and what your team will need to supply.
How do you show what your program achieved before its funding lapses?
Your program's funding ends in a year or two, and the decision to continue, reshape or end the program will depend on an evaluation. Victoria's Department of Treasury and Finance sets standard questions that every lapsing evaluation must answer. If another state, a territory or the Commonwealth funds your program, a similar review may apply, called something like a scheduled funding review or a periodic investment review. Either way, the evaluation depends on the evidence your program has built up over its life, and on what we can still collect.
- Show that the need continues and that the program is achieving its outcomes. A report on what the program delivered does not show whether the need it was set up for still exists, or whether the program achieved its intended outcomes. Your theory of change sets out what the program was meant to achieve, and we test each step in it. We ask whether there was a demonstrated need, whether the program met that need, and whether it achieved the outcomes claimed.
- Establish what your data can support before we choose the evaluation methods. Many teams arrive at evaluation time with incomplete information, missing baselines or data they cannot share with an evaluator. We check with you what data you have before we settle on the methods, and conduct a data feasibility on whether it shows the outcomes and whether you may share it.
- Put a defensible number on value for money. A value-for-money claim is only as strong as the cost and outcome data behind it. The data feasibility assessment tells you early whether your data will support a cost-benefit analysis. If it will, we deliver one; if it will not, we tell you at that point and build the value-for-money case another way. In our experience, those at Treasury who read lapsing evaluations can tell an evaluation designed to answer a question from one designed to defend a decision already made.
- Prepare for your first lapsing evaluation with a team that knows the process. If this is the organisation's first lapsing evaluation, your team may not yet know what the standard questions ask for. We walk you through the process and the types of evidence each question needs, as we did for Melbourne Screen Hub, a small entity that had never been evaluated.
- Get an independent evaluator who still understands how your program runs. The Department of Treasury and Finance's requirements call for an evaluator who is independent of program delivery, and the findings still have to fit how your program runs. Your staff tell us how the program works day to day, and we draw the conclusions from the evidence ourselves. Our independent conclusions keep the findings credible, and your staff's knowledge keeps the recommendations workable for your team.
How we run a lapsing program evaluation
We start with your program logic or theory of change, because the evaluation tests whether that logic held true. If you do not have one, or nobody has revisited it since the program was designed, we build one first. Then we map your program's evidence, design mixed methods to fill the gaps, analyse the evidence against the standard questions and write recommendations your organisation can act on.
-
Assess data feasibility with you at the start. Our data feasibility assessment maps what data exists, its form, what permissions apply to using it and whether it is fit for the analysis the evaluation needs. The assessment shifts the question from what story you want to tell, to what story the data can support.
-
Combine your program's records with what participants and stakeholders say. Most lapsing evaluations need new evidence collected for the purpose: interviews or focus groups with participants, providers, or community members; surveys of staff or stakeholders, or case file reviews. We set that new evidence alongside the records your program already holds.
-
Arrange interviews through the people who already work with participants. If your program serves people experiencing homelessness, people in the justice system or people from culturally and linguistically diverse backgrounds, reaching them takes time and requires good relationships. Together with you, we arrange interviews through the staff or services who already know those participants; learn more about our stakeholder engagement here.
-
Engage early when the program serves First Nations communities. If your program is for First Nations communities, early contact between your team and the relevant community governance bodies shapes the evaluation design as well as the approvals process. We look to engage First Nations evaluators with cultural knowledge and community relationships.
-
Write recommendations that name who will act on them. Good recommendations are specific, grounded in the evidence and aimed at improving the program. Whether funding continues is for the government, board or funder that pays for the program to decide, not the evaluator. We aim for a small number of priority actions, each assigned to the person or team responsible for carrying it out.
Our process
A lapsing evaluation works back from the date of the funding decision, so we plan each stage to get the findings to the people who decide before that date.
What we deliver:
- A data feasibility assessment and an evaluation plan. What data exists and what evidence it can support, the program logic or theory of change, and the evaluation questions mapped to the standard questions.
- A lapsing evaluation report against the standard questions. Findings against each question, an economic analysis if the data supports one, and recommendations.
Understand
We confirm with you when your board, funder or the Minister will decide whether to keep funding your program, and what they need to know by then. We read your program logic, the evaluation brief and a list of the data you hold. We also check whether the consent people gave when their data was collected allows its use in an evaluation.
Plan
We use the data feasibility assessment to choose the appropriate evaluation methods. We then list the people we need to hear from and the consents and ethics approvals required, and set the fieldwork calendar. Last, we agree with you which evaluation questions answer each standard question
Deliver
We run the interviews, focus groups and surveys the plan sets out, review your program documents and analyse your records. If the brief requires an economic analysis, we build it on the costs and outcomes the feasibility assessment confirmed.
Review
We share emerging findings with you and your program staff in sense-making sessions. Your staff add what they know about how the program operates, and that knowledge helps us write recommendations your organisation can carry out. We ultimately make the decision on what the findings say, ensuring the independence required of the review.
Close
You receive a short report that answers each standard question and names who is responsible for each recommendation. If this was your first lapsing evaluation, your team also finishes it knowing how the process works.
Related Projects
Related Projects
Projects
Less than a minute to read
Independent Evaluation of the Star Ratings for Residential Aged Care program
Read More
FAQs
Frequently Asked Questions
What is a lapsing funding evaluation?
It is an independent evaluation, run when a program's funding is ending and whoever pays for it must decide whether to continue, reshape or end it. In Victoria, the Department of Treasury and Finance's standard questions cover the evidence of continued need and the role for government, effectiveness, delivery within scope and budget, and efficiency. The evaluation also covers the risk of ceasing funding and the funding required if the program continues.
When should we start preparing for a lapsing program evaluation?
Start as early as you can, because several of the steps can take significant time. Procuring and onboarding an external evaluator can realistically take months, and data sharing agreements, privacy impact assessments and ethics approvals take time to negotiate. Count back from the date your board, your funder or the Minister needs the findings.
If the evaluation is a year or more away, you are in the best position. Map your data, check that your consent forms cover evaluation use and revisit your program logic now.
What if our data will not support a cost-benefit analysis?
We build the value-for-money case from the service records and case notes you already hold, combined with qualitative evidence such as client stories and practitioner observations. For social programs whose impact is hard to put a number on, that mix is often more persuasive than numbers alone. You know whether your data will support a cost-benefit analysis before we settle the methods, so the report makes no economic claim your records cannot support.
How do you evaluate a program whose outcomes take years to show?
We look for proxy indicators, which are early measures that credibly predict the long-term outcome. If an outcome will not be visible for years, we identify those indicators and check whether your data holds them. A proxy is legitimate only if there is a credible argument for why it predicts the outcome, and the report sets that argument out.
How big does a lapsing evaluation of a small program need to be?
The evaluation should be as thorough as the funding decision requires, and a decision about a small program usually needs fewer sources of evidence. For Melbourne Screen Hub, set up with a $1.36 million three-year grant, we reviewed more than 30 documents, ran four interviews and wrote a short report with recommendations. A well-run evaluation with a narrower scope is worth more than an ideal one that never happens.
How do we keep decision-makers engaged before the funding ends?
Share evidence with the people who decide before the last minute. Short, regular interim reports that show emerging outcomes keep your board, your funder or the Minister engaged and build a record of progress. Budget cycles have fixed decision points, so the evidence has to reach the people who decide before those points pass. Present the evidence to help them make a better decision, whether that is continuation, redesign or transition, rather than to defend the program.
15 Topics
+90 Resources
Is your program's funding lapsing?
Book time with an evaluation expert.
+40 Webinars
Learn More
Related resources
Past webinars
45 minute watch
From evidence to action: Using lapsing funding evaluations to drive better decisions
Read More
Past webinars
45 minute watch