The Robots Are Coming – Conference Post-Mortem

Intro and background
Lesson 1: Bad news: Feedback is painful. (Good news: this means competitors ignore it)
Lesson 2: Nobody will say what they think (unless you convince them you’ll listen)
Lesson 3: We need to be better at positioning our events
Lesson 4: We have (mostly) solved organisation
Lesson 5: Our facilitation is better (but not perfect)
Lesson 6: The venue was a success

“2 stars 😤” AI conference post-mortem

In this post I’ll share what we learned from hosting our most recent event – The Robots Are Coming – the agency AI conference we held at the Royal Institution in January 2024.
To make it interesting I’ll share the bad comments as well as the good ones.
If you attended the event it’ll be interesting to see what other people thought.
And even if you didn’t attend the event, you might still be interested to see how (and why) we capture feedback – maybe it’s something you could do!

Lesson 1: Bad news: Feedback is painful. (Good news: this means competitors ignore it)

We always ask for feedback after we run a conference. But the problem with getting feedback is that it feels horrible.

You’ve just given birth, and now you’re inviting people to say how ugly your baby is.

Asking for feedback feels so unnatural to me that – even now – it takes considerable mental effort to email out that feedback request form.

But we do. And what comes back is never universally positive. In this post I’ll talk about why this is useful to do.

The first thing you notice when you ask for feedback is that, however you think it went, you will usually get some very low scores:

This always feels surprising. When you run an event, people do not stop telling you how pleased and satisfied they are.

If you’re not careful, you start to think: this went great! Why bother with the feedback? But of course, only people who are pleased and satisfied do this. People who aren’t happy keep quiet.

So what you discover is – nope, it wasn’t the home run you thought. You have disappointed some people.

At first this is tough.

While you obviously get a lot of positive comments too (unless you really screwed up) the temptation is to disregard the good ones and obsess over the bad ones.

There are lots of emotions wrapped up in gathering feedback.

But here’s what I try to remember:

If we don’t collect feedback, we are throwing away something that’s indescribably valuable to our business.
The people who attend our events are – generally – intelligent and successful. When they share their honest thoughts, they are basically acting as our consultants for free. (We probably collect £100k of free consultancy each time we run an event.)
Most other people in our space do not collect feedback for most of the reasons above (i.e. it’s painful and it can upset the team.) So if we can do this consistently, it is a significant advantage 💪.

In theory, if we just keep asking people what they want and GIVE IT TO THEM, there is almost no way we will fail.

Lesson 2: Nobody will say what they think (unless you convince them you’ll listen)

The first problem with collecting feedback is that people won’t give you any!

I get it. Me, I never give feedback. Email asking me to review a cyan ink cartridge? Delete.

The main reason is that I don’t believe anyone will listen. It’s either some automated sequence, or part of some campaign to juice a TrustPilot ranking.

Here’s how we manage to get a decent number of responses:

We try to be super clear that we will read everything that people tell us
Also, we promise to publish a post like this where we explain what feedback we got – and what we’re doing with it

Lesson 3: We need to be better at positioning our events

So onto the feedback results from our event The Robots Are Coming. We had about 200 people attend and we got back 39 responses.

Firstly, we asked people to rate the event out of 10. (“How satisfied were you?” The average was 7.5 out of 10.

⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ 3 responses
⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ 8 responses
⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ 11 responses
⭐️⭐️⭐️⭐️⭐️⭐️⭐️ 12 responses
⭐️⭐️⭐️⭐️⭐️⭐️ 12 responses
⭐️⭐️⭐️⭐️⭐️ 1 response
⭐️⭐️⭐️⭐️ 1 response
⭐️⭐️⭐️ 2 responses
⭐️⭐️ 1 response
⭐️ 0 response

Generally, that’s not bad – but it wasn’t a home run. So let’s see what we need to improve.

Reading the comments, the main problem is that some people were expecting one thing but experienced a different thing.

We potentially have a positioning problem where some people are not getting what they expect.

This is bad, because when people think they’ve been sold one thing and then find it’s another, that is very difficult to recover from.

First, here’s the 2-star review. You can see it’s quite thoughtful and coherent. They basically felt the event was too focused on ChatGPT and not enough time was spent on other tools:

⭐️⭐️ “Firstly, I’d like to say that I’m a fan of Agency Hackers. I wasn’t aware an event of this theme took place last year. In that case, 12 months later, I was disappointed by how much of the content was about ChatGPT and text-based content generation. I can understand why ChatGPT could have been the big theme last year, but now – there’s hundreds, if not thousands of AI tools out there, with unique use-cases beyond content. There was no proper discussion about image generation with tools like Midjourney and its ilk, or (beyond fleeting comments) intellectual property, or how agencies are using AI tools to improve operations, or how people may be using AI for strategy and audience research or data crunching, or how AI could be used to speed up the design process.”
Anonymous

This was interesting. At this event we try to be to be “tool agnostic” and put the focus on use cases, culture and adoption, rather than the nuts and bolts of how you actually hack a Python script.

But this person didn’t feel we hit the mark.

Another problem we have is where people don’t feel the content is pitched at the right level – or they were expecting more demos vs sharing experiences:

“What would have been useful is a knowledge rating for workshops to make sure you’re sat in the right one. For example, I am Head of Digital for an agency of 8 in North Devon, our clients are not that sexy & really I am looking for more understanding on what AI can do for us (especially operationally or admin’y).”

Other people felt the talks they attended weren’t what they expect:

”I found that the titles of the talk didn’t always represent the content and it was hard to choose between the three breakouts.”

”If it were me, I’d run a workshop type session where agencies have more opportunity to think through AI applications in their own agencies and discuss/ideate with other agencies about their individual experiences/challenges.”

⬆️ Based on the feedback, the stuff above is probably the most significant challenge we have to address at the moment.

We will think what to do. We had the same problem with Good Agencies Summit, so it’s clearly valid.

We already communicate quite heavily what’s going to happen at our events. Our event landing pages are fairly unique, in that we publish about 4,000 words about the speakers. You can see pretty much exactly what each speaker is going to say.

I think we need to find a way to communicate be more explicit about what you will take away from this – but what you won’t necessarily get.

Lesson 4: We have (mostly) solved organisation

People said the event was well organised.

This feels significant because we used to sometimes get comments about things feeling a bit disorganised or home-brew. But now, nobody ever says that.

In fact people comment on how well organised and professional the event is. This is a direct result of collecting feedback. It’s unlikely we’d have ever improved this, if we hadn’t asked people what they thought.

”I thought it was extremely well organised. The communication and channels used were very slick, the venue was lovely and the event flowed well during the afternoon.”
“Big thanks to you all for putting on such good shows. The underlying organisation of it all, which I know is really difficult to pull off, is rock solid.”
“I thought Ian and his team did a great job. It’s not easy to keep such a diverse group of people engaged and organised, but they managed it with what seemed like ease.”

Some things we do that helps the event feel well-organised:

We communicate with people a lot before the event
We have name badges available (first name printed large, laid out alphabetically)
We try hard to run to time and we are quite strict on not running over

Lesson 5: Our facilitation is better (but not perfect)

This time, we got a lot of comments saying the hosts in the talks were friendly and helpful.

“I thought the hosts in the talks (who took people’s questions) were warm and friendly. They had insightful questions of their own and helped create engagement within the various rooms. The venue too was beautiful.”

At our last event in November 2023, we got feedback that some of the hosts didn’t seem prepared – or they didn’t ask very good questions. This came up quite a lot. But this time we largely fixed that.

Behind the scenes we had done two things:

1 – We had people just prepare a lot more 🤷‍♂️ Kind of simple really.

2 – We made sure there were two hosts in each room. The first person’s job was to interview the person on stage and pull the material out of them – and the second person’s role was to engage the audience.

Sometimes what happens is that when you have a guest speaker or a panel, the person running it doesn’t involve the audience enough and the room goes to sleep.

That’s bad because some of the best material comes from the experiences in the room. So you want almost like a ringmaster figure – somebody who’s competing for attention with the host.

Ideally there will be this back-and-forth, push and pull between the speaker and the audience where they’re both competing for attention.

We did get some comments that one are two members of our team could have been better facilitators.

We’ve actually never provided any formal facilitation training, so one thing we’ve done to improve is booked some with Kirsty Lewis who runs School of Facilitation.

Lesson 6: The venue was a success

The venue we chose was The Royal Institution and this got a lot of praise. It’s rare for a venue to be such a home run.

We’re trying to pull off larger, 200+ person conferences and we need a venue with breakout rooms because we prefer to let people move around the building and pick and mix different sessions. Based on this we’ll probably use this venue more. Some people did say the seats weren’t comfy after a while, but people do always say that – unfortunately we’ve never had a venue where that wasn’t the case 🤷‍♂️

“Loved the venue and the breakouts, big crowd but the team kept it really friendly and engaging.”

“The host led sessions that kept it conversational and audience participation was brilliant.”

“The venue was great, it was fascinating to be in somewhere with such historical significance.”

Conclusion

In wrapping up, it’s clear that feedback is crucial for our growth. The responses highlight areas for refinement and underscore the value of open communication.

Our next event is the Agency Growth Summit. Since the main takeaway from this feedback was the need to improve how we position the event, I doubt we’ll have time before March 21st 2024 to properly figure that part out. But for events later in the year it’s something that will form the groundwork for improvement.

Thanks to everyone who took the time to share their thoughts. Hope you find this interesting.