Stop Wasting Money on Consultants: How to maximise value from external experts

Tēnā koutou katoa, welcome to this Alan and Clark webinar, Stop Wasting Money on Consultants, How to Maximise Value from External Experts. My name is Jason and it's my pleasure to welcome you to our kōrero today. So I am the Director of Business Development for New Zealand and I've been at Alan and Clark for 11 years, work across New Zealand and Australia.

So I have seen most of the circumstances that we discuss today in some form or another. And today we actually have a large contingent of you who are joining us for the first time for our webinar, so you may not be familiar with Alan and Clark. So we're an Australasian-based consultancy dedicated to making a positive impact on communities throughout Aotearoa, Australia and the Pacific.

So our areas of speciality include strategy, change management, programme delivery, policy research and evaluation, just to name a few. And today we're discussing some practical tips and tricks on how to get the best bang for your buck when you're hiring consultants. So a lot of tips for you today, and we will be moving through quite quickly, but don't worry, you will get a copy of the video and the slides, so you can go back to anything that you may have missed.

And with me today, as you can see, two of my esteemed colleagues who are now asked to introduce themselves. Hi everyone, my name is Matthew Allen. I'm the managing partner for Alan and Clark's New Zealand office.

I've been a consultant for 24 years since Alan and Clark was established. I still work in the tools, with a focus on project and programme management and have a particular interest and focus on regulatory design and review projects and in development work in the Pacific. And I previously worked across central and local government.

And hi, I'm Charlene Harvey, and compared to the others on the panel today, I'm a relatively newbie consultant. I've only been a consultant for about three years and prior to that, I was 20 years in the public sector. So in terms of today's session, I have hired consultants, worked alongside big four consultants on projects, and now I am a consultant.

So I suppose I've seen the good, the bad, the ugly, and both sides of the client consultant relationship. So I can make sure that I add this into today's session. Well, thanks for it.

And so a quick bit of housekeeping. So there is a live Q&A at the end. We will also take questions as we go.

So pop them into the chat as they come to you. And a reminder that after the session, we'll send a copy of the recording, the slides and a summary of the key points. And so today my role is to keep these two on track, but also to act as if someone who wants to buy services from consultants.

So I'll ask some questions for them to answer, prod them and try and make sure that we get some good insights out of them. And so please make sure you give me some good admonition via the chat. So my first question is, why did you propose this topic for a webinar beyond it just being a good marketing hook to get people in the room? Well, I like the idea of pointing the finger at ourselves and other consultants in an environment where there's been a bit of anti-consultant feeling.

So sort of taking the James Blunt approach. I really do believe there's high value that can be obtained from consultants and contractors, but I've seen a lot of stuff go down that I don't really agree with. I'm a taxpayer too.

So I thought a candid discussion would be a lot of fun and actually also helpful, I'm hoping. And so let's get into it. So firstly, it's good to define what a consultant is.

And so fundamentally, they are problem solvers. So they come, take your problem, come up with solutions and then support you to design, test, and in many cases, support you to roll it out. So then they leave and you don't have the ongoing cost.

So it's a very simplistic summary, but at its heart, that's what it is. But consultants can cause problems too if they're not managed well. So they need boundaries, direction and a firm hand, otherwise they'll just cost you money and stress.

So very much like children. So continuing on that, so as a commissioner of work by consultants, so one of the fundamental things is deciding when do you really need a consultant and then follow a good process to let someone and then manage them carefully and firmly. So we'll cover each of those in the webinar, but first question is when should you consider bringing in consultants and when should you not? So as you can see up on the screen, to get the best bang for your buck for when to use consultants, you want to use them in areas where the business doesn't have the specialist skills and expertise on hand.

When you want to have an objective eye on something, so you want that kind of level of independence. When you're thinking about or you really want an innovative approach to the problem or even how you might go about optimising the business or the operating model. You might even want a different way of working, a different way of thinking or some different approaches and you're just not able to see the possibilities.

So consultants bring in best practise from other jurisdictions and multiple sectors so they can bring this into their work. You might also use them when you want a real efficient resourcing solution. It could be surge support.

So you've got a blip in work and you really just need some hands to the pump, but only use this in the short term, like really, really short term. And then you've got the extreme scenarios such as an emergency situation, such as cyclones, earthquakes, COVID. What you don't want to do is bring in consultants for work that should be resourced internally, unless they're in and out really quickly.

And the problem is that agencies aren't always very good at that and moving quickly and making sure it is a short surge. So we've got examples of people being in-house for a very long time, continuously extended. Our longest is actually five years and that was some time ago.

It doesn't happen a lot these days, but it's not really good for anybody. It's not good for our consultants either who want the different experiences and experience different sectors. So what are some tips to make sure your short term stay is a short term stay? Yeah.

So a couple of key things. If you are using a consultant to backfill for a permanent role, you want to get onto the recruitment approach really quickly. So start thinking about how you're going to backfill.

Don't delay. And the last thing you want to do is keep extending the contract. Think about how you can try and get extra value from them, such as looking at what is some of the capability build you can get for the team that you can get from working with the consultant.

And we'll cover this a little bit further on in today's chat. Also think about whether you go for a team approach. So you're getting a blend of both senior and junior people.

So not only are you getting a blend of capability, but you're getting a blend of capability at the different prices. The last thing you want to be doing is using an expensive senior resource for something that you can get from one of their junior colleagues at a much lower price. There's also a temptation, because we are lovely people, to make consultants part of the team, but honestly, we don't need to go to every single team meeting or staff briefing.

It's a total waste of money. Just get the consultant onto the mission-critical work. That's great.

And so one of the key things to ensure that you get what you need when bringing a consultant in is making sure that you have a good brief in advance. Is that right? Yes and no. So I agree that it's really important for any project that you have a clear scope, but what we often find is that we've got clients who know that they've got a problem, but the exact scope of the problem, its root cause and the potential solutions are just not clear.

And the worst thing that you can do as a client is leap to a solution and commission the work because you get to the end of the project and find that it hasn't addressed the original need or problem. So if you are not clear on what you want or need, some of the things that we'd recommend are commissioning the project in part. So bring someone in to do an initial piece of scoping work with the budget for the full project to be set at the end of stage one.

And you wanna make sure that you're including an outcause. So if you decide you don't wanna proceed, you can activate it. You wanna make sure that as a client, you're working really closely with the team that are carrying out the scoping and designing of the project so that you're comfortable that you're getting what you want and how you'll want it to add value to your business.

And remember that it's totally okay to pull the plug on the project completely or change direction if it's needed. Earlier this year, I had a client where they were wanting three operating models, so quite a big piece of work. And we recommended going with a staged approach to work through the problem and the scope of the work.

And at the end of the scoping stage, we jointly came to the conclusion that what they were wanting was something completely different and they could do the next steps internally. So this was a much better and cheaper solution for them. Yeah, and another way of doing it in terms of getting scope clear is actually to get some free input.

So subject to procurement rules for your jurisdiction and for your agency, consider engaging directly with potential providers. So one option is bringing together possible providers into a group to brainstorm project approaches. And that's something we've only been invited into about twice, I think, in 24 years, if I can recall, but it's a great approach.

All the providers, potential providers, want to impress. They'll feel pressure to speak up in a group setting and they will deliver to you a collective understanding of what scope of project could look like. And a further approach we've had is we've had agencies actually come to us and ask for advice to help them design and roughly estimate the cost of a project to inform the procurement process, so before that process starts.

And we did that a couple of months ago for an agency, provided a rough possible methodology and a range of costs. Now we knew there was no guarantee of getting any work. There was no obligation to us, no expectation from us, but we're always open to forging links and being one of the agencies that might be considered for that piece of work.

But just check with your procurement people. Not every agency is happy doing this, but my view is why not try and get some free help on a no commitments basis? And before we move on, while you want to be as clear as possible on the brief of the work, another thing that can go wrong and which we've seen go wrong is not having all the people lined up internally within your own agency. So not keeping the internal stakeholders abreast, not understanding the organisational dynamics and politics.

So we've had really happy clients before, but it turned out that they hadn't been keeping their bosses in tune with what was happening and so their bosses weren't happy and we found out that the work was binned. So a complete waste of money and time and effort. We've also had instances where clients have called for innovation and innovative approaches and then halfway throughout the project, we found out that it's actually been too innovative.

So you want to test the approach as the work goes. You want to check with your internal stakeholders. You want to ensure that you're reading the room and that the internal appetite for change and innovation is actually in line with what you're getting from the project and the consultant.

You also want to give your consultant some leeway to come to you and propose changes of direction that is within your original budget. I think we've had a question come through that's linked to that around the phasing approach. So Kim's asked, are consultants offering truly objective advice or could their recommendations be influenced by their interests in future engagements? Yeah, I mean, that's a risk, it is.

But I mean, most consultants would have internal processes where there is that consideration. You know, we often have conversations about that here and it does come down just to the ethical standards and values of the organisation, frankly, and we'll talk shortly about values and how you can choose an ethical provider. You'll also see it, you'll also see it in terms of the way they approach that scoping, whether they're blatantly approaching it in a way that makes the assumption that they're going to get it, yeah.

Or further work. And so assuming then that we've all agreed that we need consultants and that we can't do it in-house, we know what we want, so the next area to discuss is how to choose a consultant or a consulting firm. So the key question here is what do you need to assure yourself of when you're selecting a consultant? So obviously key things when looking at a provider is, you know, do they have the technical skills and experience that you're looking for? Are they providing value for money? Are you going to get a good deal? So these are some basic requirements, but then there are probably some other factors too.

So what should people be looking for when they're choosing a consultancy? Yeah, so you mentioned two, which is having the right expertise and being value for money. So they're usually pretty easy to determine through a paper-based exercise. However, putting them aside for the moment, I'd argue there are two key things you need to assure yourself through a selection process.

Firstly, you want the confidence that who you choose will own the problem and solve it for you, or for your boss or your board or your minister or whomever you report to. So you're actually seeking and buying peace of mind. Linked to, oh, here we go, right to that last question.

Linked to that, whomever it is you want to work with, you need to ensure they have the integrity and alignment of your values. So in other words, someone that you can trust that they have their best interests in mind. So when commissioning, don't just focus on expertise and experience.

Look for a provider who meets a wider range of attributes as per this slide. So how do you do that? Well, not through a paper exercise. I'd argue that you should always, almost always, probably always, meet the top provider or two before making a decision.

90% of the time we're commissioned through a paper-based, 90% of the time we are commissioned through a paper-based exercise. And only about 10% of the time would a full proposal be followed by a panel interview. So think about reversing it.

A simple EOI process, followed by time spent with potential providers, going through the approach, testing insights, their working style and values. And if you can't do that, at least have a quick catch-up online before signing on the bottom line to get a feel for their style. Also, I'd suggest look for providers who put their chances of winning on the line by challenging your proposed approach or suggesting improvements.

That's the sort of provider you should be paying particular attention to because they're looking at the value and how they can give that value back. Some consultancies will say they can meet timelines or budget with the intention to just renegotiate when they win the contract. Look out for that and please do hold them to account.

And there's also a question of how do you choose an ethical provider, someone that you can trust? Yeah, exactly. So there has been some dodgy behaviour by consultancy companies called out by governments and the media in recent years. And this has sparked a few enquiries.

So a recent example is in Australia. There was a Senate enquiry into the management and assurance of integrity by consulting services. And that was sparked by revelations that senior partners of PWC in Australia had misused confidential Australian government information to help big multinationals avoid paying more tax.

And there have been other enquiries into similar ethical issues. So practically speaking, how do you know if a provider you're looking at is ethically sound? So again, there's a paper aspect to it. You can ask them what their company purpose and values are, how they apply them, their mode of operation, how they support their people, whether they have any industries they won't work for, et cetera.

So these are all easy things that can be asked. But then there's the testing. Ask the referees, yes, but also ask around.

Have others in your organisational network use them? Meet with the proposed team members and ask them how they would approach an ethical issue that might arise. And if you're asking a consultancy to do a review of your internal policies and processes, particularly when it's in response to a problem, so an example here might be sexual harassment in an organisation, you need to actively challenge the provider that they are squeaky clean themselves on such matters. And finally, apply a trust but verify approach to delivery.

Set the expectation in your contract and discussions with them. Require an NDA when needed. Ask them to report back that any sensitive data they have collected during the project is deleted.

I've just a question related to the data. Natalie's asked, our issue is that we ask for the data and we get a cut back version, so we can't then bring it in house. So the IP must be included in the RFP and actually followed up on.

Absolutely, it needs to be in the contract. It needs to be quite clear what you're going to deliver. And we often have that through our contracts.

An example would be summaries of submissions. And we do a piece of work, we're given quite a lot of data and we provide that as a database. So I think it's just making it clear.

We do sometimes have situations where the stakeholders don't want the full data to be given. But again, it's all about making sure that it's clear in the scope, what you are and aren't getting and making sure that that matches what you're trying to do. Yeah, and we need to be very clear for stakeholders if we are collecting data, but personal data will not be shared sometimes with agencies.

So this is just about making sure there's clear expectations between us and the agency. That's great. And so one of the things we mentioned in our promo for this webinar was avoiding cookie cutter solutions.

So they're normally defined, relatively derogatory terms, as something having a similar or identical appearance created by some standard or common means, often with the implication that result is common, boring or not fit for purpose. And so a lot of clients will say that they don't want cookie cutter solutions and then every consultancy will then say that they don't deliver cookie cutter solutions. But so what extent should purchasers be worried about cookie cutter solutions and how do they look out for them? That's a lot of cookie cutter solutions.

This is one of the main grumbles I get. And I hear about consultants in that I've also experienced it myself, you know, when I've used consultants, especially some of the bigger and the older ones. And the key is making sure that you get a solution which is really bespoke to you.

Now it's quite valid for consultants to use tried and tested methodologies and in some ways their key skillset and capability is to have robust processes and approaches to derive outputs. And as a client, you want to have confidence that the approach that's gonna be used, especially for things like evaluations and analysis of submissions for policies, you are bringing in consultants for the quality of their breadth of experience and innovative thinking that can be applied to their approach and also to your problem. So you want to have confidence that the consultant brings their insights and relevance to the project, flexibility of their approaches, and that their approach to engagement is robust and authentic, especially if the project involves engaging with people who have got lived experience, Maori or different ethnicities.

And one of the things we look out for when we're proofreading proposals that we don't say tired and tested methodologies. So, you know, make sure that they are up to date. So a couple of things to look out for.

So in the proposal, you want to make sure that there is a really clear business needs section, which demonstrates that the consultant understands your industry and your business and the problem which you're asking them about. You want to ensure that they are building in flexibility in their proposed approach. You don't want them to have an over-reliance on that set tried and true methodology.

And you want to make sure that they've got the capability and experience around dealing with the stakeholders that are going to be involved with the project. You know, at the end of the day, you need to have confidence that they understand your problem, they'll engage appropriately, and the output is going to add value to you. Yeah, a bit unrelated, but also associated for sorts of things, sorts of companies that do churn out that same sort of cookie cutter stuff time after time, no names, is the tactic of selling you the senior personal specialist and leaving you with a B or junior team.

You want to get the people you're promised. So to combat that, look out in the proposal and your discussions with the provider for reassurance that you're not getting good, sorry, that you are getting good senior people. Make sure they're not just going to show up for the meeting greets and to give you the bill.

Ask for a budget, a good tip, I think, is ask for a budget to be provided with specific hours set against each person on the project team, and consider requiring reporting against that. Matt, do you have a nickname that you use internally for this tactic? I didn't actually think you were going to go there. I call it Shit House Fade.

Bait and switch is probably a better term. Excuse the profanity. So thanks.

We've now chosen a contract. How do we, we want to make sure that we get what we want and need and for a good price. So there's some obvious things that are needed.

And so there's a slide up there that shows a bunch of tips that I won't go through all of them. But one thing I will note is health and agree appears there. So getting both parties on the same page about what's happening and what the output will be is the key.

So all that stuff that we can take as read. But I will ask each of you to give a couple of key insights into things agencies can do when bringing in consultancies to make sure that they deliver a really good product on time and to budget. For me, the most important thing would be building in the checkpoints and the contracts and actually checking in.

The risk is that people get too far along in a project and what you ask for is not what you really need. Or even worse, the consultant goes completely rogue and heads off down a track that you didn't agree to. The other problem we face is either having too much or too little involvement with the client.

And it's important to get that balance right. You know, too much and you're wasting time and money on meetings and potentially a little bit of micromanaging is going on. And too little, you're not forging good, trusting relationships with the consultants that they can feel comfortable making suggestions.

Yeah. So my biggest issue would probably be budget tracking. And this slide shows a few common areas where budgets can blow out, just a few.

But even if a project is a fixed price contract, I feel like it's in your interest as a purchaser to require the provider to report on their budget so you know how the provider is tracking. It gives you an early warning system if a provider is not managing their budget well. Yes, the onus is on them, particularly if it's a fixed price contract, but it is still an opportunity for you to assert early that they need to sort it out as there's no more money coming.

And if I'm putting my purchaser hat on, how do I get free extras? So let's say I've contracted you for a particular deliverable and we've got a problem to be solved, but how can I then leverage that to get some more out of you? Yeah, okay, fair enough. I think just ask. Depending on how budget's going on a project, consultants are likely to be open to adjusting the scope or method to accommodate other needs or perhaps even doing a little bit extra for the same budget, a few extra stakeholder interviews or whatever.

And of course, they will be very happy to add on to the work with extra budget. And joking aside, I mean, sometimes it's just a small extra investment at the right time if the funds are available can actually enable you to leverage off a project to get quite a significant value add. So it's worth just thinking about that.

Something that I would push for, and is a real value add, is for consultants to offer some capability building for the team. And I personally, as a consultant, it's something I really enjoy. And so I see that there's a few examples on the slide where consultants can help indirectly build the capability while delivering a project.

Now, this can be done at minimal or no additional cost. You just need to be really clear up front with the consultant that you want them to run the project in a way that others can learn, observe, and be part of the project team. It means that as the consultant goes through the process, they can explain the rationale for their approach and some of the tips and tricks in the application of their process or methodology.

So build this into the project so it's an expected deliverable. And so moving on to the last formal part, we do want to get to your questions. Looking at the money side of things.

So how do you maximise your return on your consulting investments? So some of this is covered in what's been said already. So sitting or agreeing to clear brief, choosing the right consultants, careful scope management. And so my first question on this area is, what level of emphasis needs to go on the price during procurement? Putting down a purchase ahead on, how do you screw that price down? Well, price should be a key focus.

It's important that a consultancy commits to delivery within a set budget. But if you pay peanuts, you get monkeys. So don't necessarily always go for the lowest price vendor as it's more important to focus on a good return on your investment.

So there'll be times when you'll be really looking for confidence that you have the best provider you can get and pricing will be an important secondary consideration. So there's a few tips on this slide. I'll just touch on a couple around getting a good price.

So for basic projects where people are just rolling out a standard methodology, this is the time to really go to the prospective provider and ask them to sharpen their pencil. That is an opportunity, I think, to really screw the price down. For others where it's more innovation, more thinking is required, and you want that top of the field consultant, you'll want a dialogue.

I guess the key there is to provide as much information as you can about what you want and what your price is, and then provided you have a competitive process, you can keep a lid on and negotiate a good price. And related to that, the second point from the slide I would like to emphasise really is that I argue very strongly, and not purely from a point of self-interest, that it's seldom to anyone's benefit not to share the budget for a piece of work. Basically, you want to be focussing the dialogue and competitive instincts of providers on how much value you can get for a price versus leaving providers in the dark about the funding that's available and receiving a different range of different proposals that are not directly comparable.

So what about the options for setting fees? Is it only fixed price and hourly rate? Which is better? Are there other options that people should be considering? So from a client perspective, you usually want to go with a fixed price. There are considerable benefits from this, but as a purchaser, you know what your total financial commitment is. And that way, the risk is all on the provider, the consultant, to meet the budget, and you can focus the conversations on what can be done within their overall budget.

You can also use hourly rates, but just keep in mind they need to be tightly managed. But they can be really useful in a couple of areas. So earlier we talked about slicing up the project into stages.

So this is a good way of using hourly rates. So you'd use it for the first part of the project where you're paying to scope out the work, and then put the consultants on a fixed price for the rest of the project. And it's really good for work on call or in-house work, assuming it's for those short periods of time that we talked about.

But be super clear on where the consultant should focus the efforts. Consider an hourly rate with a cap so that you can keep tabs on the budget and manage it effectively. There are some other fee arrangements that are on this slide that you can consider.

We don't see a lot of these offered in the New Zealand market, especially in our space, but there is scope for purchasers to look at some more innovative ways at commissioning work. We had a question come through from Natalie around proposals that include experts. Would the team recommend bonds on people, bonds being monetary guarantees that the experts in a submission remain with the consultancy making the bid? So a way to force the team to front up.

To use the experts. I've never heard of that before. Yeah, I guess my immediate brain went straight to what happens if they get sick or they're just not available for a reason.

So I guess I would be slightly nervous about that. I think you can, going back to what I said before, if you're looking at, if you're provided an outline of people by task or by hour, by price, you can hold providers to account in that way. Bonds, I guess we'd consider it.

I hadn't really thought of it. Yeah, I'd think about the risk because there must be a reason why they wouldn't be using the expert in the project. So maybe it is the bait and switch.

I'm not sure. Well, we'll come back for more questions later, but what tips around getting the best price or saving money? So there's some obvious ones, tight negotiations on price, scrutiny of hourly rates, keeping the scope of work tight and including doing some of the work in-house if that's gonna be appropriate. And so we regularly see clients focus on all of these, but is there anything else around how to screw the price down or keep the best price? I think most of it does come down to those things you mentioned.

If you can negotiate a set fee in advance and manage scope and performance carefully, you will get what you need for a good price. But I do have a couple of other specific tips. I'll mention just a couple from this slide.

Firstly, consider a secondary panel for particular work when you know there's a long-term specific need. So here's an example. We were just chosen recently as one of two providers on a New Zealand Ministry of Justice three-year panel for analysis of submissions.

So we were asked to sharpen our pencil a couple of times through that process. And because we knew it was gonna be a three-year panel, we knew that it was only gonna be one or two providers, we were highly incentivised to give a good rate, and we did. Secondly, and we mentioned this earlier, apply the right people to the right tasks, specialists versus generalists and senior versus junior.

At the right price point, the amount of time you want on them, if you're focussing on that, you're getting very efficient delivery and save money. For voluntary sector, Alan Clark runs a pro bono round most years, supporting a number of projects completely free, so look out for that. But a number of consultancies, including us, will consider pro bono support for NGOs as part of a deal.

In other words, you chip in funds or resources to work with us, and we do the same. Basically, you can maximise your spend. So feel free to ask us and other consultancies for that.

Yeah, I've had another question for, you know, bonds. Are you talking about subcontract agreements or as part of the lead contractor? You know, it's just, I don't know, just a follow-up question on the previous one. Yeah, I mean, we would always, when we're entering into engagement with clients, we would always be either the lead provider and sometimes a subbie, but there's only one party who is responsible.

So if we have subcontractor relationships for a project, we're responsible for it, we're ultimately responsible for everything, is that? I think that's what they're asking you, if you're getting specialist subcontractors, make sure that they're actually gonna be available for the project. Yeah, don't go for a situation where you've got a couple of contracts with different providers to work together. I think you hold one person accountable.

Yeah, makes it a lot easier. And so we've really moved to that quite quickly. And so, as I said, we will send out the recording and the slides afterwards so you can see all the tips in more detail.

And before we get into the questions, I just wanna let you know about the next webinar. So that's gonna be on effective public consultation, our tips, tricks and processes. And so in our session, Steph will cover choosing the most suitable process, some common pitfalls in public consultation, strategies for ensuring that your questions get answered, to make sure the ones that you actually need answered are sort of prioritised, those sorts of things, and tips for handling unexpected challenges.

But now moving on to questions. So remember, if you have any, please add them to the chat and we will do our best to answer them. So moving through to there.

So first up, a question from Adam is, how can consultants justify charging $800 an hour? Well, we don't. I guess I can imagine a scenario where you've got, you know, the top, the expert top of their field worked across internationally, but you have a renowned consultant, so that's the specialist. $800 is quite a lot, I think, for a consultant.

There is a range of price points that you get for different consultants. And I suppose we'd recommend that for what you're needing, what is the appropriate expertise that you need and how much of it do you need for the project? How much, exactly, yeah. So you'd have a little bit of that 800 and probably a lot more of a lower level.

That's good. So how can you tell if the consultant is simply recycling solutions from past projects? Well, I have seen a couple of instances where people have not replaced, they haven't searched and replaced the name of the previous client. So I've seen that.

If you see another client's name on the work, it's a good indication. That's a giveaway. You can look at, we're talking about a similar thing recently, actually.

You can actually look at other people's websites and see similar work and see if it's a similar sort of thing. I think sometimes you just notice, you just see it. It just feels formulaic, template-y.

It doesn't, you can't see the value in your initial problem and value in the work. It feels too generic. It doesn't feel bespoke.

You'll see it. And yeah, as Matt said, throw in a Google search and see what comes up. You'd be surprised what comes up, actually.

So we had a question here about how can you ensure that consultants don't become a permanent fixture leading to dependency? Yeah, and that does happen. It does. In fact, I could, yeah.

Anyway, I think it just, I mean, it's a discipline in the organisation, really. I think at the organisation level, you need to have the policies that you don't have, that you don't allow consultants, contractors to come in for a period of time without testing the market. And there should be a business case or some sort of value proposition analysis to look at whether there are different ways of doing it.

Now, there may be individuals who just have that particular expertise. So again, it would go back to the building your own capability and building that into a contract over a transition period. Yeah, I mean, nobody's indispensable or replaceable.

So you shouldn't need one person to stick around for years and years. You just shouldn't. That's actually too much of a risk for an organisation.

Yeah. Yeah, what happens if they win Lotto and aren't here on Monday? Correct. The more positive framing that I've started using.

So with AI rampant, how do you differentiate between genuine work carried out versus AI generated solutions being charged for? Oh. And it's gonna, something that's gonna become increasingly something to be aware of. Yeah, I think there's a bit of a gap, I think.

I was actually talking to an academic recently and she was saying that she was getting a lot of AI stuff through her class. And she'd look at it and she'd just know it was AI generated, but she couldn't prove it. And that's not a very empirical way of doing it.

To be honest, AI is not my bag, I'm afraid. Yeah. You'll notice that at the moment, so at the moment with, when you look at any AI generated output, especially in the more creative space, so strategies, frameworks, you'll notice that the level of thinking, connection, innovation isn't there.

So yeah, I think you'll be able to sense it. It's not that good yet. It's good for repeating, consolidating, but actually formulating good recommendations, hypothesis, synthesising, creating good outtakes, it's actually quite limited.

So at this stage, until the robots take over, I actually think you'll be able to sense it. Yeah, and hopefully the consultant is ethical and values-based and they won't be charging you for time that they're doing. And there may be a time when AI could legitimately speed up a process and save you money.

And I think it's a lot of the ethical, like, are you being clear that you're using it? And especially if you're creating it for actual outputs, that's quite different from- That's quite different. Consolidating research or something. You could also ask the agency for the AI part, AI use policy.

So we've developed one here. We use it for internal marketing and such like, but not for a lot more yet because of the trust issue in terms of the accuracy. Correct.

It's certainly not for client products without permission. So someone's asked, the consultant market in New Zealand is small and if you have an open tender process, the non-price attributes become watered down. How do we address this? Become watered down.

I don't quite understand what that means. So it's more focused on price because it's open tender. I guess so, yeah.

Yeah, I don't know quite what to say to that. I mean, in one of the slides, we mentioned two envelope procurement. So you focus the process very much on the content and what the offering is first before you even look at price.

I don't know if that partly answers it. A lot probably depends what you're asking for as well. There's going to be times when anyone or any capable consultancy would be able to provide a solution.

And so it's really about the feel of working with them and some of those things. That's when some of those things around testing, like do you actually want to work with these people doing the interview stage rather than just relying on the... I think that I would generally not go open tender anyway. We don't have to in New Zealand now with all the government panels.

Or you want to ask around and find out who's good and tap a few people on the shoulder and then have the conversation. Choose a procurement process that will get the best outcome for you. And sometimes that means having some firm conversations with your procurement people in order to get a consulting or a consultant or even whoever that you're going to be working with that you actually agree with, right.

And yeah, that works for you. So yeah. Just feel comfortable with them.

Yeah, yeah. On the AI question, someone said that AI does have watermarks so you can use detection tools to assess the likelihood of your content being AI generated. As AI, you have AI battles against each other.

So Hargan has asked, a huge risk is that the agreement to engage is made at a high managerial or executive level with a real lack of detail in the deliverables outlined. So for example, deliverables will be agreed during the term of the contract. So how can this be addressed? For staging, I think.

I mean, yeah, sure you can agree deliverables during the course of a project but I would argue probably pretty early in the course of a project. Otherwise you just run out of line and you're not necessarily getting what you want. Yeah.

If it really is that unclear, yeah, stage gate it. Stage gate it. And as soon as you can get certainty on the next part and just be mindful of those out clauses, yeah.

So one of the questions we've got is, how do you create terms of reference for the work? So can you provide an example of what good looks like? Not orally. I mean, I guess what you want, again, it goes back, you want a clear scope, you want a clear timeframe, you want clear quality performance standards, you want clarity on who's going to be doing the work and obviously the breadth of engagement and such like. I think they are largely the main bits of most briefs we would be looking at.

I think you can sometimes ask the providers to provide that through the procurement process. We're quite often asked through evaluations in particular, to even go to the extent of identifying key evaluation questions, a stakeholder mapping, draft communication plan, et cetera. So some of those things you can ask through procurement.

I would ask not to go too crazy because obviously in a procurement, it's a lot of work, but you can ask for the outline, the skeleton or the key approaches you'd apply, yeah. And one of the things we often ask in those meetings is like, what is the product going to be used for? And so if you're clear about what's the next step for that product, you can normally then make the terms of reference clearer. Does it need to be published? Does it need to be for internal further work? Is it whatever it is? And so just being clear about what's the next step after this deliverable can help you make the terms clearer for this one.

Which basically makes your inception meeting critical. So that's where you get a lot of it until a lot of the business needs become clearer. So having a really good inception meeting and then you can refine terms of reference if you had them before.

And I guess give the consultants opportunity to feed into that, because sometimes the procurement process is quite rigid. This is exactly the question you want to ask. These are the enquiry questions.

This is who you want to speak to. But through the inception, you can actually come up with sometimes better or alternative ways of doing it. Someone has asked around how can you, is there any red flags around methods or practises that should be red flags for inefficiency? I would be mindful around, there's a lot of jargon in this industry.

In some ways it's an industry in itself who can come up with the spunkiest terms for a set methodology. It's this, it's that, and actually you unpick it and you figure out it's just a different kind of meeting. So if people are throwing jargon at you, ask them what it actually means.

What is it? And the thing is, you want to be really clear on what it is and will it add value? So if you're getting bamboozled, just ask lots of questions about what it is. It's not really a red flag, but just it puts you in a position where you've got a little bit more of a sense around what it is that you might be going through in terms of a process. Yeah, I'd call them on it.

Nobody needs to use fancy words and try and bamboozle, as you say. Yeah, the other thing I think is a bit of a red flag is where it's through engagement. There's very often a place where you get to with data saturation, whether it's focus groups or one-on-one interviews or evidence reviews or whatever.

And it's very easy for, it would be very easy to get into a place where you're not actually adding value, you're just doing more and more interviews, more and more time with people and you've already hit that data saturation and you're not adding any further value to the client. So there's a few areas like that where you just want to make sure the consultants are doing what they need and not any more. A couple of questions here on NGOs.

So one's around using inappropriate consultants doing the work at special rates and getting bad results, e.g. a recruiter giving advice on HR. Any comments around that for the NGO sector in particular? Very price sensitive. Yeah.

Yeah, it's difficult, isn't it? Because as I said before, there are some providers who would do some pro bono or chip in some extra time. So I think you could look out for those. I mean, yeah, it's difficult.

But again, asking around who's used other people. It's the experience. And I suppose if I think about the work that we've done with NGOs, again, it's being really mindful and deliberate around what's the need.

Because again, people will jump to, oh, I need that. And when you go in, it's not that. And so we price is so sensitive and every dollar will make it.

This is where I'd be really deliberate on working with a potential consulting firm to help you understand you've only got X amount of dollars. What's the best bang for the NGO that'll take you forward? And working together. Yeah.

So we have had a number of engagement of NGOs where we've worked very much as a team. In fact, sometimes it's a stage of here's how we might design this piece of work. You go ahead and do it.

We'll come back in, check, send a check with you and maybe do the quality assurance at the end. So you're adding the technical speciality in, but you're not charging for doing some of the stuff that they can do, which is difficult for NGOs too, because you've got less people, less resources, right? Yeah. Pro bono.

And a current thing that we're hearing at the moment is how can NGOs navigate the requirements for independent reports without spending all of their valuable money feeding consultants' profits? Well, it's the same thing, right? It's just where can we add value and not soak up your time and money. And ask for prices. Because there are different ways to get different levels of reports.

So what is the appropriate level of report that will get you across that next governance hoop? And I think with SROI in particular at the moment, it is a big flavour of the month for the government. And there is a really good use for SROI in making decisions around things. But it's like, there are some really strong gates around when it isn't as inappropriate.

And so I think just having a discussion about whether that is the thing. And so one thing we find is often going to the theory of change and the M&E framework, but not actually doing the complex calculation part because they don't often need that. And just having the theory of change and the M&E framework to then gather data to then make the calculation cheaper in the future is one of the ways that we're doing at the moment.

And so question here around how do you keep consultants accountable for delivering long-term results after the project's finished? I guess you came and did the strategy, dropped it on the desk and then walked away and then never matters what happens afterwards. So I'd say, you know, a couple of things, you know, part of the dynamics of a consultant is that you're relatively time-bound with the project. If you've done the job right, and it's really important that the project is delivered in a way that you can successfully implement it, they would have taken, the consultant, would have taken the right people along the journey so it can be successfully implemented or implemented easier.

So I'd be looking at that. I'd be looking out for that. Part of the dynamics is who's really accountable for delivery or implementation of the output.

And if it's really mission critical, build it into the contract that the consultant needs to be part of the implementation to assure that long-term results are being achieved. Or you could withhold part of the money. Phew, you're tough.

Yeah, or you could withhold part, yeah, make it part of the payout plan. Obviously, probably many consultants wouldn't agree to that for a long, long time. For years.

But there are some immediate intervention metrics or success metrics, rather. Yeah. You could hang it on.

Or get them in to do checkpoints or quality reviews or something. Actually, one of the other contracting mechanisms was that we had in the slide was- Contracting for outcomes? Yeah, yeah, there was one around, you get basically, I can't remember what the phrasing is, but you basically get a lump sum based on outcome success, yeah. So you get that little bit left over.

Yeah. That's great. And so, unfortunately, we have run out of time to answer all the questions, but we're happy to catch up with you and answer any further questions that you have.

But that's all for today. So, Namahi, thank you very much for joining, and we will see you at the next one.