Eventbrite, the San Francisco-based ticketing platform, recently unveiled AI-driven tools to automatically draft event descriptions and copy for marketing campaigns, as well as to create images. Early tests found that event organizers were able to launch ad campaigns 30 percent faster using AI to generate copy. Chief Product Officer Ted Dworkin spoke to InnoLead about Eventbrite’s experience with generative AI, and the company’s philosophy on rolling out new features and empowering engineers to use AI.
We saw the news around the new AI features. Could you walk me through how that came about, and how it’s working under the hood?
The idea came about because we have a really creative, active, empowered set of developers. They’re really customer-focused. And so we couldn’t help but be really curious and really excited by what was happening in the world of AI. We started playing around with it from the premise of what we can do for customers — what this technology could potentially enable that would make our event creators happier and would make them more successful.
We pretty quickly arrived at a couple of opportunities… Within a matter of weeks, we were able to actually not just model but experiment with, put in front of customers, get data, and realize that there was real benefit there. We put them in production, and now they’re shipping at scale.
The first two that we started with are in the area of marketing tools. We started there because we know that one of the best things we can do for creators is find an audience for them. The ability to help effectively find their audience and help sell tickets on their behalf is such a powerful thing that we can do at a scale that a creator alone — as entrepreneurial and as capable as they may be — they struggle with.
The first [tool] allows for the automatic generation of ad copy, so that when they use our tools for paid social ads, the marketing copy is effectively optimized. They don’t have to work very hard to get there. The second one is a similar scenario, but in email. We can generate automatic copy for email campaigns when they use our email marketing tools. All of that reduces the work that the creator has to do, and increases the likelihood that the copy that they produce has somebody interested in their event and eventually buying a ticket.
How did you come up with those ideas, and how did you determine when they were ready for production?
The first thing that we do is we make sure our teams are organized around customers. They don’t actually begin with the idea that they’re a technology team. They begin with the idea they’re a customer team, and they engineer for that customer.
The second thing that we did in this particular case is started talking about the opportunity that AI might present. We actually sponsored a company-wide engineering hackathon for AI. We took a few days off from the normal course of business of delivering specifically against a roadmap and said, “Use your creativity, get excited about this, but do it with a lens of problems that can be solved.”
We actually sponsored a company-wide engineering hackathon for AI. We took a few days off from the normal course of business…
With these particular projects, what made you realize they were a good fit? Did you do beta testing before rolling them out?
We started running them in controlled experiments in production. We release them to small cohorts, and make sure that they aren’t having a negative effect, first and foremost. And ultimately, we’re able to measure the positive effect. We looked at adoption. Were creators using them? Did they understand them? Did they see a value in using them? The answer to that was absolutely, yes.
We saw really pretty significant adoption rates. And then we look at the impact of the campaigns themselves. And that takes a little bit longer to determine because the events are in the future. …But those are quite positive as well.
Which AI systems are you using?
We’re using ChatGPT 3.5-Turbo as the AI engine underneath. And we’re doing image generation and using Midjourney for that.
Do you work with AI specialists and engineers, and is that a position you hire for?
…Really, this is just going to be part of the way we work and the way we engineer.
The other thing we did is create the capacity for AI capability to be used by all of our developers… It is just going to be part of how we engineer for customers, where we’ve already seen that we can pretty quickly achieve a real solution using that technology.
Do you have policies in place, as far as confidentiality and what type of data people can share with the AI APIs?
“Don’t share confidential data.”
But we don’t have any constraints beyond people just following the norms that we expect. There’s no lockdown. There’s no limits. Anybody who’s an engineer here has capacity to do it. And we trust that they’re going to be able to use their judgment.
When you put the focus on solving for the creator, you’re using the content that creator provided. So from Eventbrite’s perspective, it’s a pretty low-risk scenario, because we’re really focused on the customer, not about an independent business aim.
What types of metrics are you looking at now to see how well the products are doing?
We continue to look at adoption. Of course, we also continue to look at feedback. We look at impressions, the size of audience that these campaigns are able to attach to.
For event creation, we’re looking at production time for the event creator. We’re also looking at the success rate for that creator.
With GPT and Midjourney, was there a learning curve in figuring out how to use the APIs and how to prompt the AI?
There was a learning curve. But it was pretty quick.
I will say that we actually started looking at the image generation tooling some months ago, and it was not strong enough. The models themselves are evolving so rapidly that every month, we need to make sure that we are using the latest of what’s available… That’s a credit to the kind of rapid innovation broadly in the industry around these large models.
Are you continuing to evaluate other models, or are you committed to GPT and Midjourney?
We built it in a way that it’s actually quite pluggable, if we find ourselves in a situation where another model comes along, and we want to be able to harness the fact that people are innovating like crazy on these models. Getting locked into a single system right now would really preclude our being able to take advantage of the ecosystem of innovation.
If for whatever reason we decide that next year’s model is even more effective and powerful in certain ways, then we’ll move.
Are you doing any work on building models internally?
We’re not ready to talk about the path forward, but we’re going to continue to invest in AI significantly. And to your point earlier, we are recruiting actively for engineers that are excited about these technologies…
The product itself is, in many ways, the best recruiting: We show what we built. We don’t just talk about what we built.
What avenues do you use to recruit those engineers?
We have a very active talent acquisition team that’s out there telling the story of Eventbrite… We’re on all those recruiting platforms that you would imagine. We do active recruiting events where we have our engineers actually out front telling the story of the kind of work that they do.
We use the folks that are doing the work, because they’re actually most excited to be able to pitch. And we put them in forums in which they can connect — job fairs, technology fairs, online platforms.
The product itself is, in many ways, the best recruiting: We show what we built. We don’t just talk about what we built. And that really lands the point that we’re serious about this.
Have you seen signs that people are more likely to use Eventbrite, or any shifts in how people use the platform as a result of these new features?
Well, the platform is definitely growing. And we think that’s definitely related to a whole suite of things that we’re doing. So we wouldn’t ascribe a specific growth number to this. What we ascribe our growth number to is the fact that we have, in many ways, the best self-service tools for event creators on the planet, and the fact that now AI is enabled at production scale…is just another example to our creators of how quickly we’re going to make sure that technology works on their behalf.
| Key insights… |
| • Don’t get locked in to one set of AI platforms or models. The arena is evolving too quickly, and new ones will inevitably emerge. • Stay focused on features that can help customers get more results with less effort. • Identify metrics for success, and look for both positive and negative impacts. • Resist the urge to overdo policies — if you trust your people. • Let engineers play a role in recruiting additional engineers; don’t turn it entirely over to recruiters. |
