Make creativity a priority in your CRO process

*This article makes mention of Google Optimize. Google has officially announced the sunset of Google Optimize. Both Google Optimize and Optimize 360 will sunset on September 30, 2023…. Read More

 

 

Curious about how to make creativity a priority in your CRO process? Our Founder and CEO, AJ Davis, explores just that in this CRO.CAFE podcast episode hosted by Guido Jansen. AJ uses her expertise in user experience strategy to describe how to level up your CRO user research by valuing the use of creative means in your CRO process.

---

Outline

• Introduction (0:10)
• Working on the user experience of Google Optimize* (2:07)
• Something (10:12)
• Other stuff (12:43)
• Closing stuff (22:33)

Or read the abbreviated transcript

Guido Jansen 0:10

Today we’re going to talk about how to make creativity a priority zero process, and how to level up your zero research. And we’ll be doing that together with AJ Davis. AJ is an industry expert in user experience strategy with a proven track record for delivering measurable value to clients. He’s the founder of optimization agency experiment zone. And before that she led optimization strategy for Fortune 500 companies during her tenure at Kia has. And she was also the lead UX researcher for a product that we all at least played around with, which is Google Optimize*. My name is Peter, we also welcome to zero cafe, the podcast where I show you the behind the scenes optimization themes, and talk with their specialists about data and meaning driven optimization and implementing a culture transition and delegation. In case you missed it, in a previous episode, I spoke with somebody about why empathy is the most important skill for any professional, you can listen to that episode on the zero have a website or in the podcast app, be honest. Now, this episode of zero Cafe is made possible by our partners sites back online influences contentsquare, comfortable calm and online dialogue. Welcome to season two, Episode 33 to AJ a warm welcome to coffee. And let’s start off with how you got started with UX.

AJ Davis 1:36

Great, thanks for having me on. So what I do is I do conversion rate optimization cross with user experience research. So I started my career as a user experience researcher. And then I moved into conversion rate optimization. I was working at Google for a while and I got the chance to work on the Google Optimize* product and to see how people do CRM and what AV testing is all about, and decided to go and do that. So I worked for a company called clear head in Austin, Texas for a while. And then I started my own owning I started my own agency called experiment then three years ago,

Guido Jansen 2:07

what would stand out the most if you if you think back on looking at how people use Optimizely or sorry, Google Optimize*? What stands out the most? What’s the what are the weird things that you remember people doing?

AJ Davis 2:20

Well, I was on my team from the very beginning. So I was we were in a hotel room and California. And not like a actual hotel room we were in actually we were we were we were in that we were at a hotel like as a conference center. And we had a chance to group in teams of seven to 10 people for Google sprint week. So it’s falling the standard textbook girlfriend, so five days of starting from an idea to having a product. And so I was coming to that group at with the lens of a user experience researcher and asked a lot of questions about who the target users were. And at the time, we were just starting from scratch and starting from zero. And it was a chance to really learn about all the varieties of ways people use testing. And so I think what surprised me most was how mature some testing programs were in places that you might expect, like newsagencies, and how immature it was, and other places where you would expect it to be really mature. And they might have one person spent a little bit of time thinking about it, but not really having a full system. And what I walked away with was that why I wanted to go and do this was because it was so cool that we got a chance to do user experience, and then have an opportunity to validate that it in fact, truly was the right decision or not. And that’s why I wanted to go and take the lens of crossing and intersecting user experience research where you’re thinking about what the problem is, how we might solve it, and then validating whether or not that solution works. So your background is

Guido Jansen 3:43

in human factors. How does this connect to zero? How did you become working someone that works in zero? Because usually, up until recently, zero was not something you could study it at any college? Yeah, there’s a lot of people here, zero with different backgrounds. How did your journey look like? How did that come to be?

AJ Davis 4:03

Yeah, so I say economics and undergrad. And my last semester I encountered Dan Ariely and got to interview him for this behavioral about his behavioral economics book. So it was in in this economics journalism class, got a chance to spend an hour with him and really first got exposed to what behavioral economics was. So we had operating the system of people being predictable, and us being able to chart out how people would behave based on market forces. And it just planted the seat for me. And I wasn’t really sure how that could be applied in the real world. And I went and worked in banking for a few years. didn’t really like that. So I decided to switch career paths and get a master’s in user in human factors and information design. So I was applying the concepts of behavioral economics in the case of qualitative data. And then when I finally encountered ABX testing at Google, I had I felt like a ha moment like everything came together. It was like I went from being an analyst. And really getting into the data questioning if those assumptions were true. And then being able to take real world experimentation to understand what’s really happening. So it all kind of came together. And CRM

Guido Jansen 5:11

is basically a match made in heaven, right? combining those two fields of work. Use that to optimize whatever you want to optimize. As long as there are a bunch of customers, you can do that. And today, we wanted to talk about bringing more creativity into into zero, instead of being overly analytical, because I think a lot of people have that background, more and less background, or I think also in a lot of people from coming from SEO, having a more analytical mindset, more data driven mindset. So first of all, how would we define creativity and deceit? How would you? How would you look at that? I think

AJ Davis 5:57

what often I what I’ve often seen when I talked to other people who do testing is that we were really solution focused. And we’re trying to justify that solution using data. And I think if we all took a step back from that, and said, Let’s use data to define where we’re heading, and what the problems we know about are. And then we can plug in creativity to explore different types of solutions. So solutions don’t have to be constrained to I see this particular problem, and there’s only one solution, but instead giving ourselves the possibility that there’s one problem, and there might be 100 solutions. And so giving time and space to explore that. So you can ultimately put the best thing out there for testing and sensor. So

Guido Jansen 6:38

how would we how would one develop this? And how would you know, if you already have some creativity? I’m often surprised when people say, hey, that’s very creative, because I don’t do not see me see myself as a creative person at all. But sometimes I have a lightbulb moment, apparently. And sometimes I’ll just recognize. So how would you find out

AJ Davis 6:58

I think it’s giving permission to the to knock down ideas right away. So what I often see is the sort of like, if you’re in a conference room, and you’re kind of like, you’re have to present your case for something, and then you only you’re presenting one case for one solution. And there’s not really this opportunity to say, but I might be wrong. or there might be better ways to do this. And like, let’s open up that conversation. So when I think about something like a sprint week, you take a week, and the first few days, you’re not saying no to anything, yet, you spend time literally saying write down every idea. There’s no bad idea, get them all out there. And then don’t worry about judgment yet. So I think it comes down to postponing judgment and separating the space between this is creativity and brainstorming. And then this is when we’re evaluating which things we should move forward with just postponing

Guido Jansen 7:47

judgment on yourself. But also that sounds like something that needs to be

AJ Davis 7:54

a cultural thing that you need to have in your team, right? There’s not necessarily that’s that’s for just one person that needs to be fostered and read by the company itself. Absolutely. It has to be something where you can if you can define the ground rules before the conversation, and then keep reinforcing them it will make it will create a safe space or create an environment where people can put those vulnerable ideas out there. They don’t have to worry about whether that idea makes them seem smart or informed. But instead you explicitly state like there are no judgments, we will delay any conversation that negates these ideas. And instead take something I would borrow from improv which I do a little bit of that on the side. And for improv, what we do is we say yes, and so there’s this idea of somebody has an idea, and you build on it, as opposed to saying, No, that’s not true, this world doesn’t have that thing to be true. And so it’s even just laying ground rules like yes, standing in the conversation, as opposed to let’s shoot down every reason why this is a bad idea. And when you do that, it really just creates negativity and doesn’t allow people to make those more risky ideas.

Unknown Speaker 8:59 Yesterday’s brainstorm was so good. I really liked staffs idea of running that test on the call to action buttons, making them orange will really make them stand out, don’t you think?

Guido Jansen 9:10

Yeah, right. Do you want to design real AV test winners and achieve enormous conversion uplift, then stop brainstorming and take a scientific approach. If you can read Dutch, follow the steps then online input, the best seller management book.nl or enroll in the author’s course and become an expert in applying proven behavioral signs yourself. Go to online forums.com for more information and free downloads, I want you to do that. If you start working with a new client. And you you notice that they don’t really have this culture. They just want more we just want solutions basically. Just want you to find the improvements and implement them maybe even how would you go about trying to get them convinced that adding that

AJ Davis 10:00

Creativity is a good thing. I think there’s two tactics, I think one is you can start with it and say, this is how we’re going to do it. And here’s an activity and you can kind of come in and play fun police, you’re like, we’re going to come in and be the fun people, we’re going to spend a half hour having fun, being creative and finding new solutions. And people actually really like that, right? It’s a great way to get to know people, you can just like kind of open up and talk about your ideas. And some cultures don’t allow that. And they’re gonna say, Get to work, give me those tests, I don’t want to have any workshops, I don’t need, my team doesn’t have time. And so if that’s the case, then you you can delay that and build towards it. So in the cases where it’s very transactional in the conversation, they’re just wanting to improve conversion, improve conversion, they’re not wanting to think about, are we doing the best approach to it. In those cases, I like to interject as we’re talking about research fine, or as we’re talking about AV testing, finding. So if we see a test that last or it had an unexpected outcome, then we can talk about are there other ways that we could have approached this, and so you can start inserting those, let’s talk creatively about this, I often find that if you take the same problem, and you tackle it twice, and you don’t find a solution, that’s when people are open to starting to really have these brainstorming sessions, or to take in some qualitative input, as opposed to saying, like, you guys know it, you’re you’ve done this 1000 times, like, I just trust you to find a solution, because you’re their audience is different than every other audience. And so you do need that creativity and focus discussion around that particular audience.

Guido Jansen 11:27

When I get to new clients, I like to do a little exercise called the marshmallow spaghetti challenge. Maybe you’re familiar with that one? Do you have any other specific exercises to do to get them in the mindset of being creative being open, maybe before actually looking at the direct problem that you’re looking at?

AJ Davis 11:50

Yeah, I have some just really basic exercises, I think it’s a really important thing to do something like an icebreaker, so that everyone in the room has made themselves vulnerable in a semi semi personal way. So it might be something like, name your worst haircut you’ve ever had, or, you know, tell me about your like the first concert you ever went to. And there’s an angle of just like opening up a little bit personally, and allowing everyone in the room to open up like that, that sets a standard of like, Hey, we’re human. And we’re all trying to work together to do something as opposed to, like, I’m coming in with my suitcase and my suit, and I’m putting on a show for you. So I think that is a really important aspect to just get started on the right foot. And to present the ground rules like we talked about earlier. I also just I love giving introverts a space to be creative as well, a lot of the times there’s just this culture of like sitting and throwing out ideas at each other. And so I think sticky note exercises are really great, where you can sit and doodle or have sketch pads and like everyone can kind of sit quietly and then report back. And that’s a nice opportunity to go. If you take the time for everyone to sit and reflect individually, and then you go around and everyone presents it before anything any feedback is given. It avoids this like herd mentality that sometimes happens in creative discussions to where people kind of jump on a bandwagon because the first person said it and then other ideas get left behind. So that’s those are some of the principles that I like to operate with is give everyone a chance to think and get it up or want a chance to speak before even weighing in on any of the options. Yeah, it’s

Guido Jansen 13:23

actually the default idea I would have from a brainstorm session about UX is sticky notes.

AJ Davis 13:30

Yes, always Sticky,

Guido Jansen 13:32

sticky notes. So when you go and do these kind of exercises, trying to be creative, is there certain group composition that you would be looking for that’s ideal, maybe certain people you specifically don’t want to have in such a session or do want to have such a session?

AJ Davis 13:51

Personally, I think having people who represent different points of view within the company can be really helpful. So coming from a product background, that would mean having someone from the engineering team, someone from marketing somebody from sales, like having representation across the board, because everyone has a different experience and lens into your customer experience. And so having those different inputs just lead to very different ideas, and then you can feed off of each other. So what I wouldn’t recommend is having a room just of analysts, or a roomful of just engineers, like you want to make sure that the group is diverse. I think on the negative side of people you might want to leave out, you may want to leave out, especially in cultures where it’s very top down, you may want to leave out leaders, you might want people to be pretty much at the same level, so that you have a chance to align as a group and then present the idea as opposed to feeling like people are trying to impress the boss. I think that’s the division so a diverse room from different perspectives, and then trying to keep people where they can all be equally on the same equal footing.

Guido Jansen 14:52

Would you also include actual customers or actual users of the of the product or service into such a session or would it be a different session? I

AJ Davis 15:00

would personally do a group internally, so people get on board with the creative process before bringing in customers, I think often with customers, you want to have the customers talking to each other. And so in an ideal, like, there’s no budget time is open, like, let’s do the best possible thing, you would do an internal session, you would do a group with just customers. And then you would do a group where you’re mixing customers with internal. So that way, you get those different mixings of perspectives. there’s a there’s a tendency, when you bring customers in that they kind of become the centerfold of the conversation. And you it’s kind of like they’re the boss, right? Like you’re there people that you want to make sure feel like you’re a team smart. So you want to make sure that you you can give people really comfortable situations to really open up and talk about different solutions.

Guido Jansen 15:46

Usually, those sessions where I’m mixing clients, and customers that usually it’s not about the session itself, it’s more about your awareness of getting those clients aware, or the people working in, for example, digital marketing teams aware, hey, there’s actual, there are customers and they look, there’s real

AJ Davis 16:06

people on the other side. Yeah, yeah, there’s, I mean, the value of bringing customer voices in is just, like, unlimited, there’s just so much power in bringing them in at the right time and for the right moments. So I think when we’re thinking about brainstorming sessions, about solutions, I tend not to think about bringing customers in, but you absolutely need them. Because if their voices in the room, you’re really missing out a huge opportunity to learn and to get inspired. So maybe in my ideal situation, instead of those three buckets, you would first start with user research. So you have the data of what how users are responding to some of their challenges. And then the creative solution finding would be those three stages in the

Guido Jansen 16:45

overall optimization process.

How often would you do these kind of sessions? And then when process of optimization, would you do these?

AJ Davis 16:54

I think ideally, you do them every quarter. So you’re always thinking about what have we learned? And then where do we go next. And I think quarterly feels like the right pace for a lot of businesses, because you can get through several cycles of solving problems, and then reflect on it. So for some businesses that might realistically be once a year, and then and then you’re building out the roadmap for the rest of the year. But in an ideal scenario, if you’re really moving fast, you want to make sure you take that time to reflect and be creative moving forward. Of course, a lot

Guido Jansen 17:23

of people that do user research, a lot of people unfortunately, don’t do user research or these companies that don’t include it yet. In their optimization programs mainly focus on analytics, and basically the the hard data that they do have from tools like Google Analytics, or whatever and tools are using and do experiments based on that. So what what would you say is the value that does user research brings to that, in general, I

AJ Davis 17:55

think very simply, we get the white from the quantity, quantitative data and the y from the qualitative data. So we can always see what customers are doing looking at Google Analytics, so we can only infer why they’re doing it. And if we talk to customers and have them think aloud and user research, we can really get a chance to really understand what’s happening and why they’re doing it. So I love to bring research in when we just don’t know why tests didn’t turn out the way it did. Like I have a client who’s been working at the same concept on their product page over and over for the last six months. And we’re at a stage now where we have to do user research, because otherwise we’re going to have this endless cycle of trying to put something out there that isn’t informed by wire customers not responding to this very thoughtful approach. So for businesses that aren’t yet really thinking about user research in the process, I think that’s the easiest and most natural place to first plug in user research is asking, why didn’t this test performing the way we expected? And can we get inspired to do something better and improve on it?

Guido Jansen 18:54

For over 10 years now, online dialogue advises about evidence based conversion optimization with a focus on data in psychology, we see that analyzing data and recognizing customer behavior results in a better online dialogue with your clients and a higher ROI. The team of strategists, analysts, psychologists and UX specialist gathers valuable insights in online behavior of your visitors. And together with you, they optimize the different elements of your zero program, through redesign, expert reviews, AB test and behavioral analysis. For more information about their services, go to on dialogue. Now before does that you just see, like a weird drop off point somewhere in your funnel. It can be very eye opening to do those. It’s very, fairly easy to start with, right? I mean, you can you can of course complicate user research as much as you want but in like basic user research an interview with someone or just having someone perform. buying something on the e commerce sites can be very simple things that you can just observe People people doing. And like I said, you can you can brainstorm, for hours on why something is the way it is. But you can also just look at those users. Now. What are some examples that you did you find? There’s your phone before, where something was unclear looking at the data was fairly obvious. When you did user research,

AJ Davis 20:23

I think navigations often something that you can’t get a really clear sense of why people are doing the things they do. And if you do a T test or card for it, you’re suddenly having these aha moments like, oh, our reflect our navigations, reflecting our business, not what our users need, or expect. So I think navigations a case where over and over every time, people are really surprised for the kinds of things you see. So we did a test with a customer where we took some messaging from their homepage and moved it to their product page. And it was value proposition messaging, and it just like a very foundational marketing thing, as you tell customers, why they should buy from you. And the test didn’t win, it didn’t do anything, it was inconclusive, when you move that content over, so we were kind of trying to figure out why that would be the case, because we saw in the heat maps that people were scrolling and seeing it. And it wasn’t impacting their decision. But when we did a user study on it, we got a chance to get qualitative feedback on how people were responding to it. And time and time again, across all 10 participants, people were like, so what this is the same as every other company would say. And so we learned the message messaging itself that they had used elsewhere, and we’re using it in their sales and their marketing and all over, wasn’t resonating. And so they didn’t ever really took the time to reflect on the words itself. And we just thought that we all made some assumptions that that was effective. And by the time we got to the point of user research, like it kind of just created this tidal wave across their company things they need to go revisit. So user research can be really insightful and can really pull back some assumptions that the business has been making time and time again, for a really long time.

Guido Jansen 22:01

The most obvious one from my side and example that I’ve used before is that that we saw a big drop off in sort of step in, in the checkout process. I don’t think we add fields texture, we didn’t necessarily see which field was the was the problem just Patria was on. And, of course, there’s all of those little things happening on those pages. So you can you can think of anything and brainstorm, for hours. But we did a user research, it was very obvious, there was a, there was a gifting company, and they there was a dis fields that required you to enter the phone number of the person you were sending the gift to. And there was no explanation why why the company needed that. And it was and when we sat down with those customers, like five out of six of the day, they they complained started complaining about I don’t have it. I don’t want you to spend the person or it’s, it’s a surprise. So why would you? Why are you surprised? Oh, yeah. So it stands out. So clearly, if you do user research, sometimes not always, of course, but some of it stands out. So clearly what the problem is on such a such a page, and

AJ Davis 23:14

yet how you can improve that I find that every time I deliver a user research report, I’m like, this is going to feel obvious. And it’s because it’s now obvious, it wasn’t obvious before. And we got to this point and had to ask and learn. And it becomes obvious because you’re hearing it over and over from customers. But it also makes sense. Like in that example, you just gave that there’s a lot of questions. It’s like, why are you asking this. And whenever you introduce uncertainty, you almost inevitably hear that in user research that they also are feeling uncertain. So set expectations, just a good principle set expectations for how stuff will be used when it will show up. And then that will remove a lot of that friction. Another study that came to mind was actually a survey that we did on an exit where we were wondering why we were seeing such a big drop off and check out. And overwhelmingly we saw that people were leaving because of coupon codes. And it was something that we thought could be the case. But the customer wasn’t really clear that that made sense. Because they said, oh, there’s customer there’s coupon codes all over. But 80% of the people said, I’m leaving to find a coupon. And it really just changed our strategy about talking about coupons. How do we present it? Where do we present it? How do we make it easy? If we want to be a discount focused brand? How do we make it easy to grab the code or apply the code? So just asking the questions when you’re seeing that drop off and data that took five minutes to set up? You know, these things are not complicated. The tools are out there now that can make it really low cost to get feedback from customers. Whether it’s usability study or a survey, just Jesus

Guido Jansen 24:41

spoke about, you mentioned about delivering those reports to clients, what’s your favorite way of basically telling them What’s wrong? What are the what the results were from those those studies is like like video presentation, or how would you go about doing that?

AJ Davis 24:57

It really depends on the scale. So for the the example About the survey, it was just something we shared quotes. And we did it in context and our weekly meeting and everyone was like, yep, makes sense. Let’s move forward. So sometimes I like to present it really simply. So it doesn’t feel like an over thought out resolve. This is an insight just like an analytics insight, which pulled it from GA, here’s the data. We pulled it from customers, here’s the data. But in cases like the one I mentioned, where it really had a major cultural impact to the business, it was really important that we showed recording. So I like to deliver it as a very simple executive summary. So it’s easy to share. A lot of the time, like C level, folks don’t have a lot of time and are used to kind of flipping through things in email. So we like to have quick reports, quick insights pulled out in the email, a very short report with some summary with some simple takeaways on the first slide. And then for people who want to really understand and hear the customer and have the time to do that, we’ll build video clips, where we’ll weave together audio clips, or video clips of customers. So it all depends on what you’re trying to communicate

Guido Jansen 25:59

Exactly. And might also depend a bit on the maturity of the client, or how much extra context you need to give and also

AJ Davis 26:05

how much they know you, right. So if you’re very early in a relationship with a client, sometimes you need to present a little bit more information to build that credibility. And for customers who have been working with for several years, they don’t really need us to spend that time to build a formal report, because we got the insight we needed to be able to move forward. So

Guido Jansen 26:23

what would be your favorite methods of doing research? Or do you do you have a favorite? Or does it depend on the only issue that you’re facing?

AJ Davis 26:31

So from the lens of creativity, and getting to the best test ideas, I think usability studies are the best. So the reason is, it gives you a chance to really see somebody use your product from start to finish. And an e commerce. It’s a pretty standard flow for most sites. So it’s easy to catch main things at every part of that website funnel. So for me, that’s my favorite, because it lets me level up all my test ideas on every part of the site, I’m inspired by some new insights that I didn’t have before. And it’s a great tool to get a baseline. So you can do a usability study every quarter, once a year. And you can keep rerunning that same study and understanding changes over time and what customers expect and how they’re responding to the new elements on the site. So I think that’s by far my favorite

Guido Jansen 27:15

is usability study, you mean, inviting participants, giving them a case to work on? Do something to walk through the websites, videotaping that,

AJ Davis 27:25

yeah, I should explain what it is. That’s a great call.

Usability study is there’s several different approaches to it, that’s where it started was often in a lab setting where people would physically come to your lab, be at a computer, you would sit next to them, and you would give them a script and moderate as they’re working through an application or a website. Technology has enabled us to do a lot more remote research as well as remote unmoderated. So tools like usertesting.com, try my UI, user interviews.com all these tools can enable you to just quickly get feedback from customers that are good representative sample of who your target customers will be. So for me, I like the unmoderated remote studies, because you can get that feedback really fast. And it’s low cost. So that trade off of time and cost to inspiration and insights is one of the best I think for CRL. That’s when it’s online, it’s relatively easy to want to segment the type of users you get. Most of those services allow you to pick people from a certain country or certain experience level or they also let you add screener questions. So you can ask we’ve segmented our user usability studies based on is this a current customer of this business is this somebody who’s never purchased from them, but if purchased from their competitors, you can tear them based on their spending habits, there’s all kinds of things you could introduce into the survey so that they get closer to your target market. And you can get insights that say, for new customers, these are things that we might need to do for existing customers. These are some of the things I might expect. And so you can go really deep and start really segmenting down but at a very basic level. Just getting people who are, reflect the country, the basic income characteristics, maybe certain brands that they shopped before is a great

Guido Jansen 29:14

starting place. It’s going to be fun, marketing budgets have suffered. And the share for a B testing has been impacted too. If you want to keep testing to enterprise standards, but save 80% on your annual contracts, you can consider comfortable with their summer release, you can take advantage of full stack and hybrid features, strong privacy compliance, no blink, and enterprise grade security. feel good about your smart business decision, infest what you save Becky, your CRM program, check out www.att.com slash 2020 comments I got from a couple of months ago they said yeah, we can do a user study. But we don’t want to use any of those those kind of services before Cuz those are people that are used to doing user studies. So they are very adept at going through those, those websites. And those are not our those are not normal users of jewelry is your responsibility that

AJ Davis 30:13

yeah, I think there’s two ways to mitigate that risk. Because it’s true, there’s going to be some bias introduced because of the way you’re doing recruitment. But that would be true. If you recruited people from Twitter, right? If you decided to post and say, We need people for our study, let’s sign up on Twitter, you suddenly are just looking at the Twitter segment of your customers. So from for a tool that’s unmoderated research, you can you can mitigate some of those things by saying how many studies have you done in the last six months in the last 12 months? And then screening out people that way? Some of those tools will have some transparency. And so that’s another thing to look into when you’re deciding which tools to use is, how much are they enforcing that are providing that insight back to you. And then I think the other the most robust thing you can do is to introduce opportunities for people to opt into your user research across any interaction with your business. So for very large companies, many of them have their own databases for people. When I was at Google, we had a mix, we had our own internal database. And we worked with external recruiters and we worked with the unmoderated tools as well. So sometimes it’s also just about diversifying where you’re getting the insights from

Guido Jansen 31:21

exactly I think, if I got the chance, I prefer doing a mix of different types of research because then you can mitigate those those biases that each method has an NCO okay. But if you still get a consistent picture over these different methods, and properly refined, we can we can move to the next step.

AJ Davis 31:39

And the same thing is why we combine qualitative and quantitative, if you just look at the qualitative data, it’s not going to give you as much information as putting the two together. So same, same thing with recruitment, finding participants talking to people, you can do usability studies, plus interviews, plus analytics, and you’ll just be way better off than if you just did interviews, so won’t be some things that you’re changing or improving in the coming months in regards of the way you work with clients. I think one of the things we’re really working towards is creating a product around this combination of user research and AV testing. We’ve often just plugged in user research as the relationship develops, because people have a better understanding of what it means to work with a company for CRM services, and might be more hesitant to make that investment for research. And we’re we’re looking to really formalize and productize that so that every company has a consistent, like, let’s start with user research and analytics exploration, so that our roadmap is just automatically elevated versus waiting several months before we start doing some of that work. So for us, it’s, you know, getting the word out and talking about it, productizing it and having that be the way that we we do our services, we had

Guido Jansen 32:48

some discussion in also in our Facebook group about how you manage billing clients. So how do you go about that? Is it like a fixed fee for certain periods? Or do you have like a monthly retainer? Or what’s your preferred method of doing that?

AJ Davis 33:02

Good question. We are doing time and materials at the moment. So we will set a six or 12 month engagement, and then we’ll work towards that budget and track to it each month. So we just we have a culture of transparency. And so we deal with our customers every week or every other week. And we’ll report out on how we’re doing against that overall budget. But the short answer is we do time and material. So do you

Guido Jansen 33:26

have some goals that you set as an

we will improve your revenue by X percent?

AJ Davis 33:35

Yeah, we tend to be talking about and we tend to frame it in the in regards to how many different tests we will be doing at which level of complexity. So we might be doing five simple tests, and two medium level tests, and one user research study. So well, we’ll tend to estimate how much time all that stuff will take and talk about those deliverables. And when you get into the math of like calculating specific revenue, we do estimates based on individual tests, and we’ll talk about that, but we generally don’t make that as part of the commitment or requirement for the program itself. I think

Guido Jansen 34:08

a lot of people are struggling with that. I mean, ideally, you would think do it based maybe on on revenue. But of course, if if you hit something like COVID-19 and a global pandemic, then you’re screwed. And if you’re doing a really good job, the client probably doesn’t want to pay you anyway, if you if you do revenue based that’s that’s also an issue a lot of those zeros encounter

AJ Davis 34:33

on the other end deals from the study wants to do with hourly based the salt is also tricky, because it’s not scalable. I think the hardest thing about what we do is that if we’re doing a really good job, we’re creating so much more value than just what an individual testers saying. So we’re recruiting insights that potentially your whole business can act on. And so to quantify that value, and to calculate that is it is hard, and I think all the models have shortcomings for how we can

Guido Jansen 35:01

All models are wrong, some are useful. So as a final question, any books that you would like to tip to, to our audience, we have some avid readers, or audio book listeners, and obviously podcast listeners in our audience, so what would you like to tip them,

AJ Davis 35:18

I actually like to challenge them to insert a little creativity and fun. So rather than a book recommendation, I’m going to recommend to book two games that are really fun, but still database. So Dan Ariely, who I mentioned early on has a game called the irrational game, which is a great game for kind of guessing how people are going to behave and how irrational they are. And then there’s a really fun game called chardee Party, which is very simple, very similar to Cards Against Humanity. And it’s about charts. So you’re describing what you’re seeing in the chart with different pairing of pairings of words. You know, in the age of COVID, we can read lots of books, but it’s good to create some interactions with people. So I would encourage them to check out those two games. So can we do those these online? No, I was. I bet you could do you know, if you each had a version of charity party, you could probably play it, like over zoom. Okay,

Guido Jansen 36:07

so we’ll have a look at that. I’ll include links to those, those games in the show notes from the podcast, so you can all have a look at that. AJ, thanks so much.

AJ Davis 36:15

Yeah. Thanks so much for having me. Thanks for

Guido Jansen 36:17

sharing your experience with with user testing and being being a bit more creative, and bringing that to this year. Okay. Thanks

AJ Davis 36:24

so much. You’re welcome. Thanks for having me. All right.

Guido Jansen 36:30

To Episode 33 of the zero cafe podcast with AJ Davis. Next week’s episode is in Dutch where we’ll talk about soft persuasion with Misha Koster. And the week after that is again in English together with a Joel fettke. And we’ll be talking about zero copywriting a few then and always optimizing