User Testing vs. Usability Testing (& Why it Matters)

Do I owe you an apology? My inboxes (Facebook, YouTube, email, you name it) have been absolutely flooded lately. I’m still trying to catch up.

So if you’ve been trying to reach me with a question about Ugly Unicorns, my coaching services, or to yell at me about why Sketch is the best thing since sliced bread, I’m sorry. I am doing my best to respond to everyone I can. But you still will never make me love Sketch, so ha. (I give you permission to picture me thumbing my nose at you right now!)

Competing at My First Jiu-Jitsu Tournament

I’m sitting here today with my first jiu-jitsu medal. I got third place, and the tournament was such an interesting experience. You can check out a bit of that here if you want to.

I wanted to go into the competition pretty relaxed, but when it’s your first match you still don’t really know what to expect. Everyone kept telling me to “be first,” so that was my mentality. But I got foot-locked, which felt like a stupid mistake, and I lost that first match. My opponent did a great job planting his foot so he had a good base, and I couldn’t figure out how to break his grip at first. I used some different techniques, and also ended up learning a lot about which ones would have been more beneficial at certain times!

Nothing can really prepare you for the difference between a training session and the real thing, so that’s why I’m glad I had multiple matches. The first one was mostly about realizing this was real, the second was a little better because I knew more about what to expect.

Watching back video of my matches, I can see a lot of opportunities where I could have given more force or been more strategic, but live and learn. The tournament jitters definitely got to me, but this was my first competition and it was definitely a cool experience overall.

Will I Compete Again?

People are already asking me if I am going to do another jiu-jitsu competition, but I honestly don’t know! It’s a bit tricky because I am at the edge of my weight class. The guys I was competing against were definitely stronger and more technically advanced than me, and probably were more hungry for those wins, too.

For example, side control has been like freakin’ kryptonite for me and that was evident in the competition as well. I’ll probably be drilling side control for the rest of my life! Whether I compete again or not, it’s cool to have had an experience that tells me where my weaknesses are, what I need to work on. I highly recommend you give jiu-jitsu a try if you want to test yourself as a person!

Okay, on to UX.

Knowing Your Terms

Let’s talk about words and what they mean, because I’ve been doing some research. Specifically, I’ve been looking into the difference between user testing and usability testing. Because that difference matters!

I found an article on Medium that made me really think about what we say we’re doing versus what we actually are doing when we are building and validating UX.

For example, here’s what my job has been working on - and calling user testing - lately:

• Integrating two kinds of features into a system

• Trying to understand how those features could work with an application

• Presenting certain data based on questions the user answers

All of those things we’re working on, we’ve been calling user testing. But is that the correct term? It turns out that no, this should actually be called usability testing.

The distinction is important, so let’s take a look.

User Testing vs. Usability Testing

It turns out that user testing is all about testing your concept. Do users need your app? Is there a real demand for it?

But usability testing is different. It’s about testing if users can use the app. Does it work? Does it do the cool stuff we set it up to do? Can it be used for the purpose it’s intended for?

What that means is that a LOT of UX designers and people in the industry have been using these terms wrong. In other words, if you’ve already got an app and you already have people testing a prototype, that is not the user testing phase. Nope. You’re already in usability testing.

A User Testing Example

One rainy day in Paris, a guy named Travis Kalanick couldn’t find a cab. Which eventually led to the creation of Uber, but first there was user testing. What did that look like?

Travis started asking people at bus stops and building exits: would they like an app which could bring a taxi to them with basically one click? In other words, he asked them if they needed an alternative to what they were already doing. This was user testing.

Now, sometimes users don’t actually know what they want until you give it to them. But it just makes sense to test the waters a bit, rather than going all in on an app or product without any validation of your concept.

What if he had waited until after the app was built before asking people if they would use iit? Would it still have been successful? Maybe. But he sure as heck would have been taking a big risk, making it all about his own guesses rather than real user needs.

Next Steps

Once Travis had heard enough people say they were interested in his app to grab a cab, what came next? He had to design. He spent time booking and riding around in taxis, looking for the parts that were not user friendly. He thought about how to make it all easier on his app.

Then, of course, he made the app. Easier said than done, but once the prototype was finished? It was time for usability testing.

Usability Testing

At this stage, Travis started showing users how to use the app, but not just so they’d adopt it. He wanted to study how they interacted with it, which areas they got stock, and what could work better. That usability testing research helped him tweak and adapt the app until it was as easy and effective to use as possible.

I think this is so important, because far too many companies run full steam ahead with an idea they’ve dreamed up, and then they don’t understand why their idea isn’t really appreciated. Because if you don’t let real users use your app or product, you won’t actually know what changes are needed. You’ll be going in blind, and that’s never a good thing.

Micro vs. Macro Testing

There are many ways you can test and validate your ideas at different levels. For example, you might start with a sketch, maybe not even with mockups or wireframes yet. But you shouldn’t move too far past that sketch until you ask people some questions, and figure out if this is even something they will really want.

In other words, start small: is this needed? Because if you start at the macro level, trying to take an existing product and retro-fit it to consumer feedback, you’re going to end up with some kind of Frankenstein: pretty close but somehow never quite right.

Wrapping Up

Simply put, here’s the difference between user testing and usability testing:

User testing is:

• Gathering information about your target demographic’s wants and needs

• Identifying pain points that your app or product can address

• Formulating solutions to those pain points

• Validating your proposed solutions with your target demographic

Usability testing is: 

• Giving users hands-on experience with your app or product

• Identifying what works and what needs improvement

• Thinking realistically about whether your app or product is achieving its goals based on user feedback

Understanding the process and making the best possible products go hand in hand. And at the end of the day, that’s what being an Ugly Unicorn is all about, right?