We’ve noticed you’re visiting from NZ. Click here to visit our NZS site.
Submissions analysis and consultation design, delivered to your timeline
When a Bill or a proposal draws thousands of submissions and the deadline is tight, we can read and categorise every one. Every submission counts in the numbers for and against, and we code in full every submission that gives reasons, so the themes reflect what people argued. We cross-check those themes with text analytics, and you get a report built for the Select Committee or the Minister that will read it. If the public consultation is still ahead of you, we write the documents and the questions with you, so the responses give you the answers your decision needs.
-
- On the Treaty Principles Bill we read and categorised 292,527 submissions, coded 3,000 in full and delivered the final report on time with a five-week turnaround.
-
- Our analysts code the submissions, then we run the whole dataset through text analytics as an independent check that no significant theme is missing. If someone challenges a figure, you can show where it came from.
-
- The analysts who will read the responses help you write the questions, so the answers give you what the decision needs.
When the consultation is contentious and the deadline is fixed
People who take the time to write a submission for a consultation expect someone to read what they wrote. The Select Committee, the Minister or your Board expects the report to reflect what submitters said. If the volume of submissions is large and the timeframe short, the only thing you can do is bring in expert assistance. We take the reading, coding and analysing off your desk and give you a report you can stand behind.
- Thousands of submissions, a short deadline, or both. We work with you to agree scope, reporting requirements and risk pathways, scale the team to match the volume, and deliver before the deadline.
- Figures that may show up in the media. We read and categorise every submission, so the final numbers accurately represent the whole dataset. Then we cross-check the themes using text analytics, so you can say where the numbers came from.
- Duplicate files and form submissions. We remove duplicates by hand. Form submissions and identical template responses, count in the numbers for and against, but stay out of the thematic analysis. That way themes reflect what individuals believe rather than just campaign volume.
- A small number of dense, technical submissions. We go in-depth to research and validate the evidence each submitter presents. Then we include as much of this depth in the report as you need to offer convincing answers to specialist questions.
- Starting a consultation from scratch. We write the consultation documents and questions with you and plan the timing and channels around the people you need to hear from. If you are a government agency, those people include Māori as statutory partners.
How we analyse submissions and design consultations
When submissions come in, every one is read and categorised. We then code in full the submissions that give reasons, against your original proposal or questions, and cross-check the themes with text analytics. We structure the report for the people who will use it: clause by clause with recommendation tables for a Bill, or question by question for a Board paper. The same method runs from a few dozen dense submissions to hundreds of thousands of public responses.
If you have not yet designed your consultation, we can assist you through the four steps of the consultation process: planning, writing, collecting and analysing. We design the public consultation stage to get diverse input, including from Māori, subject matter experts, and interested or impacted groups. If you also need interviews, focus groups or workshops alongside the written process, learn more about our stakeholder engagement here.
-
Themes checked two ways. Our analysts code the submissions first. We then run every submission through Leximancer, a text analytics tool, as an independent check that the coding captured all the significant themes. On the Fast-track Approvals Bill, human coding of 2,539 submissions in NVivo and the Leximancer check produced consistent results. That met the evidential bar a politically sensitive Bill requires.
-
Te reo Māori and Te Tiriti content handled with cultural expertise. We engage Māori speakers to translate submissions written in te reo Māori, and we publish the original text and the interpretation together. Submissions on rongoā and Te Tiriti o Waitangi go to analysts with the cultural competency to handle them with accuracy and respect. That way the full weight of what submitters said reaches the report.
-
Human-led, AI-assisted analysis for the largest volumes. For very large volumes, we bring in AI to do the coding, with people setting the rules and checking the results. We build the codebook first: the list of themes and the rules for applying them. We then give the AI one analytical question at a time, and feed it the submissions in small batches. A person checks the outputs and writes the synthesis at the end.
-
A report your team can use straight away. We write it so it fits straight into your departmental report or Board paper. That means recommendation tables for each section, clear answers to each consultation question, and quantitative data that shows where submitters stand.
Our process
Whether you need a consultation designed from the start or thousands of submissions analysed by a fixed date, the work runs through the same five stages. If you are running a large-scale survey with open-text answers instead, we analyse the open text the same way.
What we deliver:
- A consultation plan and the documents that go with it. The plan sets the timeline, which your stakeholders know in advance, and the system for receiving and logging every submission. The documents carry the questions we’ve written with you.
- A submissions analysis report. Every submission counted for or against, those that give reasons coded in full, the themes checked two ways, and findings set out clause by clause or question by question.
-
Governance reviews – independent assessment of management structures and processes to enhance your business strategy and create measurable value.
-
Strategic support – delivering expert consultancy services that combine strategic assessment with targeted risk management to build robust governance framework.
Understand
We start with the decision the consultation or the analysis has to inform, who will read the report, the deadline, and what makes the topic sensitive. We agree scope, reporting requirements and risk pathways with you at the outset, and we ask whose voices have not been heard.
Plan
If we are designing the consultation, we write the consultation plan, the documents and questions, and set a timeline that gives stakeholders enough time to respond. If the submissions are already in, the Bill or proposal and the questions you asked set the framework we assess them against. We test the framework on a sample of submissions and agree with you how to prioritise the sample for full coding.
Deliver
We collect the submissions and clean the dataset, removing duplicates by hand. We read and sort every submission, separating the ones that give reasons from bulk submissions or those that only state support or opposition. We code in full the submissions that give reasons against the framework in NVivo, with quality checks throughout. If new themes emerge, we refine the framework.
Review
We run every submission through text analytics as an independent check that the coding captured all the significant themes. Senior team members then review every theme before it goes in the report. On the largest Bills, our relationship manager checks in with regularly through the busiest weeks of coding. We track milestones and team capacity in real time, so you know where the work is.
Close
We deliver the report on the agreed date, structured for the Committee, Minister or Board that will read it, and ready for public release.
Related Projects
Related Projects
Projects
Less than a minute to read
What 25,000 New Zealanders said about the Fast-Track Approvals Bill
Read More
Projects
Less than a minute to read
Navigating 16,000 submissions on complex health reform - Therapeutics Products Bill
Read More
Projects
Less than a minute to read
The Nation's Voice Heard: Analysing 292,527 submissions on the Treaty Principles Bill
Read MoreFAQs
Frequently Asked Questions
How quickly can you deliver submissions analysis?
The timeframe depends on volume, density and what the report has to do. For the Environmental Protection Authority we analysed 29 in-depth submissions against 31 consultation questions in one week and delivered the full report within two weeks. The projects above show the other end of the scale: hundreds of thousands of submissions on a Bill, analysed inside a parliamentary timetable. We agree the deadline and the sampling approach with you before we start.
If a figure is challenged, will we be able to show how it was reached?
Yes. Every figure in support or opposition is a count of the whole dataset, because we read every submission and categorise each one. Every theme traces back to submissions coded in full against the agreed framework, with quality checks throughout and senior review of every theme. Text analytics run over the whole dataset is the independent check that no significant theme is missing. So, if someone questions a figure, you can point to the count behind it, the category each submission went into, and the themes that emerged.
How do we know the right consultation questions to ask?
That depends on the decision you are trying to reach, but it’s usually best to keep them short, in plain English, and only asking about one idea each. We can write the questions with you before the documents go out, to get the answers your decision needs. We mix questions that give you numbers with open ones, and add a catch-all question on the proposal overall. We also offer a short set of five to ten questions alongside the full set. Before anything goes out, someone outside the project reads the questions to check they make sense without the context you have.
Can your team stay on to help us write the departmental report?
Yes, if you need it. Our report serves as a working document for your departmental report, so your team can use it directly. On the Therapeutic Products Bill, the Ministry retained three senior Allen + Clarke team members in-house to support its own report, a clear measure of confidence in our work.
How do you use AI in thematic analysis, and how do you know it is accurate?
We use AI as a tool inside a human-led method: people build the codebook, the AI applies it, and people check the result. The AI records its reasoning and a supporting quote for every code, so a reviewer can see why the AI applied each code and check it quickly. We use blind sampling to spot-check as we go. We have used AI on large volumes of submissions since 2023 and checked that it analyses accurately.
How do you stop the analysis losing nuance or minority views?
We read every submission, so a view held by a few people still counts. Form submissions count in the numbers for and against but stay out of the thematic analysis, so the themes reflect individual reasoning rather than campaign volume. There is no upper limit on the number of themes a codebook can hold. We can also flag submissions that need review by an expert, such as those written in te reo Māori.
When should we start planning a public consultation?
As early as you can, working back from the date your final decision or document is due. We plan for the consultation to be open for at least six weeks, with stakeholders told when it is coming. That notice reduces late submissions and requests for extensions. We also look at what else is going on for the people you need to hear from, so the questions do not land in their busiest period.