We’ve noticed you’re visiting from NZ. Click here to visit our NZS site.
Independent reviews that say what’s working and what to change
A scheme, a framework, or operating model that has run for years reaches a point where it’s crucial to check whether it still does its job. We carry out independent reviews that combine a desktop review of your documents, a literature review of the published research, and consultation through interviews, surveys and workshops. Then we translate all the information into actionable findings and recommendations.
-
- We assess how well your scheme, framework or operating model does the job it exists for, and what options would make it more efficient and effective.
-
- We examine the published evidence and the scheme's own documents, integrating that with what the people who run and use the scheme tell us.
-
- We ask you whether each recommendation is workable as it’s developed, so you don’t end up with suggestions that can’t be implemented.
How do you know whether a scheme, framework or operating model is still fit for purpose?
A review answers that question, though often you do not choose when it happens. An Act may require one every five years, or a sunset clause may end a regulation unless there’s a case for continuing. The decision the review informs is yours, your CEO's, your board's or the Minister's – but whoever has the final say needs to know what works and what doesn’t.
- Answer the question behind the mandated review. An Act or your board's own regulations may require a review every few years, and the decision to commission one is never just administrative. We scope the review to the questions you need answered, so it meets the mandate and tells you what you need to know, along with what other decision-makers require.
- Name what keeps a part of the scheme from working. A scheme can run for years with part of it not working and no clear account of why. We interview and survey the people involved and the organisations that support or accredit them, and anonymise what they tell us. The findings then name the barriers and the factors that would help that part work.
- Base the next strategy or regulation on the evidence. If a strategy or a regulation is due for an overhaul, the evidence it needs may be spread across dozens of reports, articles, datasets and legislative documents. We read them, analyse them for themes, and produce a report organised around the decision you have to make.
- Get the results for a transition you made under pressure. If you brought an operation in-house, merged two functions or restructured at speed, your board, your CEO or the Minister will ask how it went. We review the documents, survey the people affected and interview those who ran it, so you can answer with an evidence base.
- Own the recommendations from the first briefing. Implementation begins when you receive the report, and nobody acts on a recommendation nobody owns. At the first briefing we ask who in your organisation will act on the recommendations.
How we do our reviews
The methods follow from your terms of reference, and the brief that sets what the review must answer. A desktop review of the scheme's policy, process and governance documents establishes what the scheme was meant to do. Interviews, surveys, focus groups and workshops show how the scheme works in practice. If the review needs consultation across a whole sector, learn more about our stakeholder engagement here.
-
Set benchmarks from published research. For a legislated framework, a literature review of the international evidence sets the standard the framework should meet. We then test that standard against what practitioners tell us is happening.
-
Count how many people share each view. We survey the people a scheme affects, such as the people who use it and the staff who run it. We then compare the survey results with the interviews to see whether many people share a view raised in an interview or only a few do.
-
Understand where the sources disagree. We compare documents, surveys, interviews and workshop notes, identify the themes that are consistent across them and say where views diverge.
-
Quantify what the decision depends on. If a decision weighs costs against benefits, we present the result in a form a decision-maker who is not an economist can use. For a package of 12 tobacco control measures, we showed that 73 people a year quitting smoking permanently would cover the package's regulatory burden.
Our process
Two things shape a review before fieldwork starts: the question it asks and the terms of reference that set its scope. We work through both with you in the first stage, then plan the review, carry it out and report on it within those terms.
What we deliver:
- A review plan against your terms of reference. Questions the review will answer, documents and research we will read, who we will consult and how, and how we will report to each audience.
- A review report with findings against each question. The evidence behind each finding, and recommendations written so you can decide what each one requires and in what order to act on them.
Understand
We start with the question you need answered, beyond just the ones an Act or governance rule requires. We work out what your own reporting cannot tell you, why an outside view matters, and how you will need to communicate the findings. What we work out shapes the terms of reference.
Plan
We match the methods to the questions: work out which documents and published research to include; and who to interview, survey or bring into a workshop. We also set out what the analysis needs to show.
Deliver
We read the documents and the research, run the interviews, surveys and workshops, and analyse the results, checking each source against the others. Under a short deadline, we run the document review and the interviews at the same time.
Review
We bring the emerging findings to sense-making sessions – meetings where we share what we have found so far and get your perspective If the review has stakeholders, they take part too. We use the sessions to clarify whether each recommendation is feasible while the work is running.
Close
You receive the report, with a summary or slides for any audiences who will not read the full text. Each recommendation names the person or team who will act on it, as you told us at the first briefing.
Related Projects
Related Projects
Projects
Less than a minute to read
Post Implementation Review of Insourcing the Domestic Operations of the Pacific Australia Labor Mobility Scheme
Read More
Projects
Less than a minute to read
Supporting Unpaid Carers: Building the Foundation for Australia's National Carer Strategy
Read More
Projects
Less than a minute to read
Gap analysis of barriers to general practice accreditation
Read MoreFAQs
Frequently Asked Questions
Is there a right moment to commission a review of a scheme or a framework?
You rarely get to choose the date, but you control how ready you are for the findings. A review is useful if the commissioning team has already decided what it will do with findings that challenge the current approach. On three recent reviews, each client had prepared to receive a challenging finding, and thus was prepared when the results came in.
Will an outside reviewer understand our constraints?
You set the brief, then let the reviewer do the job: the terms of reference lay out the parameters, and within them the findings follow the evidence. Practical recommendations come from testing the published evidence against what is happening in practice, through consultation with the people who run and use the scheme. We report what the evidence supports, whether or not it is the answer you expected.
How quickly can you do a review if the question is narrow?
A rapid review typically takes six weeks or fewer, and we have delivered some in a week or two, usually after an event that requires an immediate response. The trade-off is scope: this kind of review suits specific questions whose recommendations you can put in place without building something new.
During one rapid review after a critical event, we had very little time and more than 700 documents to review. We categorised them with the program team to find the core set, then ran the document review and the interviews at the same time.
How do we put the recommendations into practice?
Decide before the review starts who will own the recommendations. Then go through each one deliberately and work out what it would require before committing to it. This typically separates the organisations that act on reviews from those that do not. Treat the implementation as a project in its own right – with an owner, an order and resources.
Do we need a review or an evaluation?
A review assesses something you already run or must comply with: a scheme, a framework, a process, an operating model or a policy. It asks whether that thing is still fit for purpose and what to change. An evaluation assesses an initiative, such as a program, a service, a pilot or a strategy, and asks what changed for the people involved and why. The methods overlap, and we scope the work to the decision you face; learn more about our mixed-method evaluations here.
15 Topics
+90 Resources
Is a scheme, a framework or an operating model due for review?
Book time with an evaluation expert.
+40 Webinars
Learn More
Related resources
Past webinars
45 minute watch
From Review to Real-World Impact: How Independent Reviews Strengthen Public Policy
Read More
Past webinars
45 minute watch
Evaluation Myths Debunked: Reflections from 200+ evaluations
Read More
