Kia ora koutou, ko Steph James Tokuingoa. I'm a Senior Consultant here at Allen & Clark and this webinar is on running effective public consultation. The focus of this webinar will be on running online written consultation processes, rather than in-person consultation.
Consultation is a key part of the policy and legislative cycle. Designing effective consultation documents, survey tools and other submission collection is important to ensure that stakeholders engage and provide targeted and constructive feedback. Here at Allen & Clark we're experts at analysis of submissions, which means we've collected a lot of best practise tips for creating the best consultation documents and questions over the years.
We've completed submissions analysis on anything from 20 small submissions to 20,000 large submissions. The main challenge that we observe is that key stakeholders and communities just aren't brought in early enough in the process and I'm going to talk a little bit about co-designing your documents later on. Other key challenges are inaccessible and complex consultation documents and survey questions.
A lot of consultations just have way too many questions and questions that are too long or not written in plain English, leading questions rather than open questions. We often find that clients realise at the end of the process that the questions they've asked didn't really get them the right information, for example the level of support for their overall policies. Another key challenge is when clients expect or want to see certain themes coming out of a consultation and they're not open-minded to what is actually there and what the public actually thinks.
So during this webinar I'll try to cover off how to avoid these challenges and how we can help you to make sure that you're not making these mistakes. So I'm going to talk you through this process. This is the whole process for developing your document and you'll see that consulting on a draft strategy is in there.
There are four main steps in the consultation process which I'm going to run you through. Planning, writing, collecting and analysing. So start with the planning process.
There could be a couple of reasons why you need to run a public consultation. You might need to run a consultation on your policy or strategy or your plan because of legislative or statutory requirements or as a select committee process on a bill. Or you might just be creating a document that you really want to get your community's views on.
Before we get into consulting on the draft document and before we even start drafting it, what you really want to think about is who your key stakeholders are and who might need to be involved in the drafting process. This could be the local iwi, local councils or communities that the policy will really impact. It might be that you want to think about bringing these communities into the drafting stage to help co-design the policy document.
One of the challenges that we often come across with engagement is that stakeholders see a draft of a plan or a document and they feel that they haven't really had input into the first stages. So it's really important to bring in key stakeholders early if you can to help you really co-design that document. That way when they see the draft they don't feel that it's already a done deal.
For the rest of this webinar I'm going to talk about the example of a strategy because I think that that's useful for local governments, central government organisations and NGOs. So you might be wondering how do I know who my key stakeholders are and how do I involve them in early processes of drafting. So what you really need to think about is who is this strategy going to impact.
Local iwi, local councils, communities that you serve. Hopefully you already have some relationships with those communities that you can draw on during this process and if you don't then you need to start thinking about building these relationships early. Broader than consultation it's really important to set up genuine relationships in your community before you need to consult, just so that your relationships don't come across as being really transactionary.
Engagement with Māori as statutory partners is particularly important and needs to be thought about early. Tap into internal teams with relationships and contacts to draw on to engage with Māori communities. My colleague Jackie Ngawaka ran a webinar a couple of weeks ago which is available on our website which went into way more depth about engaging with Māori communities.
Even if you think your strategy isn't relevant to Māori it's actually not your decision and so you still need to consult regardless of what you're consulting on and it might be that your community tells you that they don't need input into it. A lot of you mentioned that rangatahi or young people are a problem for you and you really struggle to consult with them. You really need to be proactive and targeted if you don't have those existing relationships.
Doing some research to identify youth representation groups for example Generation Zero, universities, community groups or using social media can be really useful. Young people in particular are a bit disillusioned so you might if you want them to listen to you you might need to find some sort of representative or make relationships with people who can be more influential in that space. Be creative with who you might need to get to be the face of your consultation and build those bridges with young people.
So once you've thought about who your key stakeholders are and you're in the process of drafting your strategy you want to think really early on about timing your consultation process rather than as an afterthought in the process. Work backwards from when your final strategy is due whether that's a statutory deadline or a self-imposed one and build in a really good consultation process. You need to ensure that you're giving stakeholders enough time to consider your content and really comment on it meaningfully and provide the feedback that they want to provide.
A really big challenge that we often see with consultation is that it's too short to allow people to really think about the content and respond effectively. This is particularly true for hard to reach communities because you need to factor in time for it to filter through channels to them. Key communities such as local iwi are often really over consulted so you also need to consider the fact that they might be having requests coming in from multiple people and it could be on the same issue that you're consulting on or different issues.
They might just see you as another government department or organisation that's asking for something of them so that's where building those relationships really early on comes in useful as well. If you're consulting with companies or organisations or local councils then you also need to think about their sign-out processes because sometimes their feedback will need to be approved and signed off by different teams, managers and maybe even a board. This can really take time and we often see that this results in requests for extensions and things like that.
At a minimum we would recommend having consultation open for six weeks and giving stakeholders an idea or a heads up about when this will be in advance and trying to stick to that timeline as much as possible. This reduces the risk that people will want to give you late subs or ask for extensions which creates a lot of risk further down the line of things being delayed. So then you need to go back to thinking about who your key stakeholders are whose opinions you really want and think about what else is going on in their world at the time when you're planning on consulting.
For example a lot of you mentioned rangatahi or young people and finding it difficult to engage with them as I said before. So think about the timing, is it school holidays? Is it during their exam periods? Is it during the summer? If you're engaging with companies or local government think about elections, think about the end of the financial year or even the end of the calendar year when a lot of people just really check out. School holidays are relevant for pretty much all stakeholders as a lot of people take time off over them.
If you can avoid it try not to engage with Māori communities just before or during Matariki and also consider this for other cultural groups or cultural holidays. This is why it's really important during the planning process to think about whose opinions you really need to hear from and planning to ensure that your consultation caters to those people. It's not always possible to consider all these things if you have a deadline and we're aware of that but it's just important to consider them as much as you can.
So then when you're planning you also want to think about the outcome of what your consultation will be or the end product. How are you going to effectively communicate all the feedback and ideas that you've gotten from your stakeholders? This can take time and adds a big step into your strategy development process. Think about whether your team has capacity to pull together all of this feedback or whether it's something that you're going to outsource and if think that you will outsource it try to bring those people in as soon as possible because they can really help you to craft your consultation questions and ask you some of these tricky questions that will help to improve your process.
What do you want to present to the public? Does it need to be a really detailed analysis of the submissions or something a bit more high level or is it fine to just say that you've incorporated feedback into the end strategy? So the next step is writing your consultation document. Let's envision that our draft strategy is really complex and long and covers a lot of important issues. Another key challenge that we've observed is that people try to consult on the whole thing so they're essentially expecting the public to read their entire document from start to finish.
This could be hundreds of pages long and then they're expecting the public to answer a lot of detailed questions about the strategy. For example we've seen consultation documents that are hundreds of pages long and include 100 plus questions that they're expecting answers for. This is super unrealistic and it leads to annoyed frustrated and really disengaged stakeholders.
Think about the accessibility of your survey in terms of colours, questions, font size and having a shorter document. Make sure that it's written in plain English. Is it easy to read? Does it include graphics and if it does include graphics and graphs do they have accompanying captions? Are you providing your submitters with various ways of responding? Do you need to translate your document into different languages? People in media and comms teams will have really good ideas about how to make your consultation document as easy to read as possible.
So if you have those teams within your organisation try to engage with them as soon as possible. We would really recommend creating a much shorter document that signposts the specific stakeholders to different areas of the strategy that you want them to focus on. You can also give people an idea of how long it will take them to read and fill out the questions so that they can plan their time.
If you can it's also good practise to have two sets of questions. One that answers everything for someone who has a lot of time and a lot of interest in the area and another shorter set of questions that quickly covers the key points. Think five to ten questions max.
Another important factor is to list your questions in order of importance. That is with the most important questions at the start so that if people do lose interest midway through at least you've got the important stuff covered. If you need to report on specific information such as demographic or geographical information of your submitters you need to ask those questions at the beginning.
We often find that clients realise midway through the analysis that they need that information and by then it's really hard to draw it out if you haven't asked those specific questions that you need. Make sure that your questions are as simple as possible in plain English and short and include only one idea. Break them up if you need to.
Include a catch-all question that gauges how people feel about your draft strategy overall. This can actually provide really useful information. It also makes analysis much easier.
Use some questions that will provide you with quantitative data for example yes no or options and then back those up with qualitative questions. But be aware that your quantitative data might be unreliable because people often contradict themselves in the more open-ended questions so you need to factor that into your analysis as well. Think about whether you can be a little bit more creative with the way that you present your data.
For example we recently run a series of clients a series of surveys for a client rather than one big survey. What this meant was that their stakeholders only had to answer 10 to 20 questions in each survey rather than 60. There are obviously pros and cons to this approach and it wouldn't work for a set of stakeholders that are already really over consulted.
However you want to make sure that you're not leading to that kind of consultation fatigue. The risk of not using some of these tips is that the data that you get is not as accurate and not as rich and leads to a lot of limitations. Always be open to the possibility that stakeholders might not use your survey as well and they might just provide it in their own format.
No matter how perfect your consultation document is and key questions are, stakeholders might just choose to write their own comments. This limits our ability to have data and adds analysis time and it's important to be aware that this often comes from key stakeholders like local councils and big organisations. These free text answers are often very long as well.
People sometimes come to a consultation just to say what they're going to say and give their views and not necessarily answer the questions. So make sure that your questions are open rather than leading. It's also a really good idea to get someone else who doesn't know anything about the content to review your questions for you so that they can tell you whether they're too complex, written in plain English or if they're going to be easy to understand for the average person who doesn't have as much context as you do.
So here's a summary of the key tips that we have for creating the best consultation document. Make sure that your consultation document is as short as possible and if not have a shorter two-page summary document. Have two sets of questions, one longer set and one shorter set which focusses on the key points.
Make sure that your questions are to the point in plain English and some that will give you quant data and some that will give you qualitative data. List your questions in order of importance and think outside the box. Could you run a series of surveys rather than one big one? Try to get your questions reviewed by someone external to your team who doesn't have knowledge of the subject area.
Okay so we've planned our consultation and now it's time to actually write our consultation document. How are we going to get the best information out of our key stakeholders? The draft strategy is out in the world and you need to get as many high quality submissions as you can. Make sure that everything is on your website and it's in one web page with clear signposts.
Make sure that submitters don't need to click through too many links to get to the survey. Think about other ways that you can engage your audience. For example, going back to the issue of rangatahi, you might want to think about contacting universities to help mail out your survey or using various social media platforms.
If you have a media and comms team, you should work with them as early as possible because we often find that they have competing priorities for social media and website space. Also think about how you might be able to collect your information. You might decide that alongside your survey you want to run some public webinars or workshops and focus groups to help explain your draft strategy.
As I mentioned earlier we'll be running some future workshops on a best practise in-person consultation. If you are going to run in-person consultation alongside your written consultation, you'll need to think early on about how to integrate this into your analysis with the survey answers. Also be aware that you'll probably get the most traction towards the end of the consultation process right near the deadline for submissions.
People will leave it to the last minute. You need to build this into your planning and be aware that you won't be able to have analysis and a submissions report ready straight after the survey closes. Also have a clear system for how you're going to receive submissions, where they'll go, who will collect them and who will be responsible for logging them and making sure that you've captured everything.
This is much easier to plan at the start. Whatever form of submission that you're inviting people to make, ensure that you're collecting them appropriately and recording them. As I said earlier you should think about what your final product is during the planning phase because this will help you to craft your consultation document and think about the questions that you want to ask.
You may be able to analyse your information within your own team but you should also think about whether this is something that you may need to outsource. If you are going to outsource it, make sure you bring in those people who are going to analyse the information as early as possible because they can really help you to craft your consultation document and to write the best questions. You also need to think about who the audience of your final report is going to be.
What information do they need to see? How much detail do they need? Do you need to publish and report on your feedback back to the public? This is a really important thing to think about during the planning phase as well because it will help you to really craft the best questions. Infographics can also be a great way to fill out your report and make it more accessible but think about that early because again that will really drive the way that you ask questions. Okay that's all the content.
We've got some time for some questions. We have a question from Andrew Kerr asking can you point to some consultations you've seen or been involved with that are good? Yeah of course. So last year the Ministry for the Environment consulted on their recycling plan for the future.
So they had a really long strategy document but then they also had shorter summary documents which was really useful for stakeholders and they had two sets of questions and really pointed clearly to how you should answer those questions. So they gave people the option to have a longer set of questions or a shorter set of questions and that really helped too. They got a lot of submissions on that and it really helped because people knew that they could answer a shorter set of questions rather than the full set of questions.
We ran a consultation process for a client recently where we decided to do it in three surveys rather than one big survey and we kind of broke that up into quite simple high level information for the first survey and then got more into the detail in the second two surveys and we did find that we got really good engagement because people were aware that they'd be getting more complex information for the last two surveys and we still managed to get quite good traction throughout. What do you recommend for getting to hard to reach areas? For example not everyone has good broadband and lots of consultation documents are online. Yeah that's a really good question and it comes back to that point that I made earlier about key stakeholders.
So that's where you might want to consider actually going out into those communities that you want to reach and running some workshops or some focus groups so actually going to the communities and you might want to take some hard copies of the surveys rather than relying on online surveys because as you say a lot of places will not have broadband access or they might not have computers in their homes and so what you really want to do is make the time to send people out into those communities and meet them where they are rather than asking them to come to you. What is the minimum viable product if you don't have time to do all you've described? Yeah that's a really good question. Obviously you're not going to have time to do all of these best practise tips.
Hopefully you do but in a lot of cases you don't. So I think the minimum is just making sure that your questions are really clear and in plain English. I think that's a really big problem that we see in a lot of consultations is that you might think it's really easy to understand but you've got years of context and years of understanding behind that.
So I think the best thing is to get someone who doesn't know any of the information to read the survey or read the consultation document and make sure that it's written really clearly and really plain English that will be understood by stakeholders. I think that is probably the key tip to take away from this if you can only take one. What do you think is the maximum length consultation or discussion document should be? Yeah thank you.
Maximum is really hard because obviously if you have a document you need to consult on it right. So I think you can make a consultation document that can be 50 to 100 pages. I think your summary document that you expect everyone to get the key information from should be no more than 5 to 10 pages with 5 to 10 questions.
I think that's really the maximum especially if you're trying to consult with really over-consulted communities or young people or people that might have not have a lot of time on their hands. You need to make it as easy as possible so that's why we would say 5 to 10 questions max in that short document. You can have a longer document with a longer set of questions that covers everything but you just need to be aware that you're not going to get as much engagement with that really long document.
Have you seen a change in buy-in from senior leadership for good practise consultation? Many times what is recommended by those doing the consultation is overwritten by those higher up. Yeah that's definitely a problem and it is something that we see a lot is that your senior leadership teams will have an idea in their head of how they want something to go or what questions they want to ask and it is a real push to get what you know to be the best practise consultation out of them and you do need you need sign off from the senior leadership team at the end of the day. So I think planning early and giving them a heads up really early about how you want to run something can help because then they feel that they've had time to consider it.
Bringing them along on the process and explaining the risks I think is really useful and what kinds of problems you might experience down the track if you're not running the consultation process in the best possible way. So that would be my advice for getting senior leadership on board. If you're short on time and urgently need to get a public policy out can it be an interim policy and how would you recommend going forward from there? Yeah I think that's a really good idea.
It obviously depends on what kind of statutory or legislative process you have behind that policy so you might be limited by that but assuming that you can have an interim policy I think that's a really good idea and it shows your community that you're open to that feedback still. So yes we're putting this policy in place but there's a timeline on it, there's a deadline, this is just the interim and we're looking for more feedback and more consultation on this policy before we put in place the final. So yeah I think if you're there's no restrictions an interim policy is probably a really good idea to just keep your stakeholders engaged and keep them feeling like you're still listening.
A question from Ashlyn asking if you can expand on how you see youth is disillusioned with engagement and is the issue really that the typical means of consultation and participation are outdated and typically cater to older audiences with more time on their hands and organisations need to change to better match the current climate. Yeah yeah I think that's probably all true. I do think youth are disillusioned in that they don't see themselves in the government process as much anymore and that's partly because I think a lot of young people expect stuff to happen really fast and so they can see government as being a very slow moving old-fashioned thing and then I agree with you I think that a lot of the processes that we try and capture people with are just not the right processes for young people and that's why I say that having someone who can really champion your cause or be the face of your consultation who does bridge that gap and kind of is relatable and gets along well with the young communities that you're trying to get in touch with can be really helpful and there's a couple of examples that I can think of that I'm happy to kind of talk to you in more detail about later on.
A question from Andrew he's asking if you have any experience working with rural Māori in isolated areas? Yes yeah definitely a lot of those rural and isolated areas and particularly Māori communities you just need to build those relationships as early as you can and as I said Jackie's webinar a couple of weeks ago would have talked about some of that as well but rural communities you just need to go out there and you need to talk to them rather than expecting them to come to you and especially when things like internet access or computer might be a problem but yes we do have a lot of experience in that area and we can provide you with some really specific advice if that's what you're looking for. What are some other options for additional summary of consultation for example someone who that has low literacy but is a key stakeholder? Yeah that's a really good question and I think rather than having just a big written document you might want to think about videos, you might want to think about infographics, pictures and webinars that explain through either your strategy or policy or as you say your summary of submissions so you could run something like this where you explain all of your different feedback that you received. If you've got designers you can there's some really clever stuff that you can do with infographics and pictures and graphs and things like that to explain to someone who might not have as high literacy as you do and those are all some really good ways of getting getting across and being a little bit more accessible.
What measures would you put in place to ensure the consultation does not give expectations for all ideas and feedback being taken on and showing what has occurred as a result of the consultation? Yeah that's a really good question and obviously it is really important because some stakeholders will expect to see what they've said actually implemented and that's just not realistic right and it's not something that you can always do because you're restrained by maybe legislation or what's actually feasible so I think you can put that in your consultation document you can say look we really want to collect your feedback we want your comments but you have to be aware that it might not be implemented in exactly the way that you're expecting and I think it's fine to be transparent about that. How do you make use of data from open-ended questions? There's a couple of ways that I could answer this question so open-ended questions are really important because you do want your stakeholders to feel that they're actually being asked a question and not led in a certain direction so what we do is we use like a couple of bits of software and we have a team of people that kind of analyses and reads each of those submissions and things them up in a way that makes sense and then pulls that into the final report so we actually do just read them all and analyse them and that's how we kind of make use of that data and then as I said you can use it to back up your quantitative data so your yes no answers or your options. So Jo is asking what's the average response rate to surveys and how do you best account for bias? The average response rate is very different depending on what the content is, when you've sent your survey out into the world, who you're sending it to and all the other things that are covered like school holidays or timing and things like that.
Obviously if it's something that's really highly political and interesting to people you're going to get a lot of responses so as I said before we've had select committee processes for example where we've had anything up to 20,000 plus responses. That's a really extreme example, sometimes we've had responses more in the 20s or 30s so it really depends on what your content is and how well you've gone about engaging with the stakeholders that you want to be involved. Accounting for bias is really important obviously, you can have a section in your report that talks about limitations.
The fact of the matter is that most of your responses are going to be biassed because everyone comes from their own personal circumstances so most most submitters will have some kind of bias whether it's good or bad and so you just need to caveat that in your report so you need to say and that's where collecting demographic information is really important because then you can link it back to the answers so you can say from this group of submitters we had these answers and then it's really clear that it's biassed by the kind of submitters that they are. For example they could be some specific type of organisation, they're going to have their own views about it but if you've collected that demographic information then you can be really clear on why it is that you're getting those answers. A question from Nicola asking if you can publish a verbatim comment if you have not told the submitters that that may happen.
I think it's always important to have a check box at the start of your survey which just asks that are you happy for your submission to be published and then you can get the answer yes or no and then you know whether you can provide a verbatim comment or not. You can also ask submitters whether they would prefer to be anonymous or if they're happy for their name or organisation to be published so there's definitely easy ways that you can just ask that at the start just to make sure that you are only publishing comments that people are happy for you to publish. And a question from Adam asking have you used AI to manage large volumes of submissions and how does that work? Yeah thanks Adam we have used AI to analyse large volumes of submissions it's at the moment it's still it's still very much in its infancy and we do check to make sure that it's analysing things accurately but if you've got the right questions asked in the right order then you can definitely use AI if you've got a really really large volume of submissions and that's just something that is going to improve over the next couple of years obviously.
All right so that's the end of the Q&A thank you so much for attending this webinar and thank you for all of your thoughtful questions. My information is on the end of this webinar including my email address and my phone number so please feel free to reach out if you have any specific questions about running the best consultation process. Ka kite anō.
