Menu Search
Jump to the content X X
Smashing Conf Barcelona

You know, we use ad-blockers as well. We gotta keep those servers running though. Did you know that we publish useful books and run friendly conferences — crafted for pros like yourself? E.g. our upcoming SmashingConf Barcelona, dedicated to smart front-end techniques and design patterns.

Chatbot UX – Does Conversation Hurt Or Help?

Chatbot fever has infected Silicon Valley. The leaders of virtually every tech giant — including Facebook, Google, Amazon and Apple — proclaim chatbots as the new websites, and messaging platforms as the new browsers. “You should message a business just the way you would message a friend,” declared Mark Zuckerberg when he launched the Facebook Messenger Platform for bots. He and the rest of the tech world are convinced that conversation is the future of business.

But is chatting actually good for bots? Early user reviews of chatbots suggest not. Gizmodo writer Darren Orf describes1 Facebook’s chatbot user experiences as “frustrating and useless” and compares using them to “trying to talk politics with a toddler.” His criticisms are not unfair.

Further Reading on SmashingMag: Link

Here’s an example of a “conversation” I had with the 1–800-Flowers Messenger bot6 after I became stuck in a nested menu and was unable to return to the main menu. Not exactly a pleasant or productive user experience.

Chatbot UX7
The 1-800-Flowers bot user experience can be frustrating. (Image: Mariya Yao, TOPBOTS6155514743403529252218148) (View large version9)

To Chat Or Not To Chat? Link

Designers who are new to conversational interfaces often have the misconception that chatbots must “chat.” At the same time, they underestimate the extraordinary writing skill, technical investment and continual iteration required to implement an excellent conversational user experience (UX).

This article explores when conversation benefits and when conversation hurts the chatbot ux (user experience). We’ll walk through case studies for both sides of the argument and compare divergent opinions from Ted Livingston, CEO of Kik, who advises10 bot makers to deprioritize open-ended chat, and Steve Worswick, the creator of “the most human chatbot,” who encourages developers to invest in truly conversational experiences.

As you’ll see from the examples below, both strategies can lead to successful chatbot experiences. The key is to choose the right level of conversational ability for your bot given your business goals, team capabilities and user needs.

The Case For Chat Link

Steve Worswick is the developer behind Mitsuku11, one of the world’s most popular chatbots. Mitsuku has twice won the Loebner Prize12, an artificial intelligence award given to the “most human-like chatbot.” The popular chatbot has conversed with more than 5 million users and processed over 150 million total interactions. 80% of Mitsuku’s users come back for more chats.

Mitsuku, developed by Steve Worswick13
Mitsuku converses better than some humans. (Image: Mariya Yao, TOPBOTS6155514743403529252218148) (View large version15)

The longest a user has chatted with Mitsuku is nine hours in a single day — a testament to the bot’s extraordinary conversational abilities. Mitsuku does not help you find makeup products, buy flowers or perform any functional utility. The chatbot’s sole purpose is to provide entertainment and companionship. You won’t be surprised to find out that Worswick thinks “chatbots should be about the chat.”

Building a conversational chatbot that isn’t awful is extremely hard. Worswick nearly gave up many times when Mitsuku repeatedly gave unsatisfactory answers and users called her “stupid.” One major breakthrough occurred when Worswick programmed in a massive database with thousands of common objects such as “chair,” “tree” and “cinema,” along with their relationships and attributes.

Suddenly, Mitsuku could give sensible answers to strange user questions, such as, “Is a snail slower than a train?” or “Can you eat a tree?” According to Worswick, “Let’s say a user asks Mitsuku if a banana is larger than X, but she doesn’t recognize what X is. She knows that a banana is a relatively small object so can deduce that X is probably larger.”

Even if a chatbot is utilitarian, providing spontaneous answers in a conversation — especially if unexpected — can delight and engage users. Poncho16 is a Messenger bot that gives you basic weather reports, but the creators gave the bot the personality of a Brooklyn cat. Poncho can conduct small talk and even recognizes other cats. “Weather is boring,” admits Poncho founder Kuan Huang. “We make it awesome.”

Poncho the Weather Cat17
Poncho’s personality differentiates the bot from boring weather apps. (Image: Mariya Yao, TOPBOTS6155514743403529252218148) (View large version19)

When You Should Add Conversation To Delight Users Link

Making a bot conversational takes tremendous effort, but if you are up to the challenge, here are the top situations in which conversation could distinguish your chatbot from competitors’ and truly delight users.

If You Need to Differentiate From Competition Link

As seen earlier, Poncho’s conversational personality distinguishes the chatty weather cat from boring, routine weather apps. Bots launch at a more rapid pace than mobile apps due to the lower technical barriers to entry. Dozens of bots already exist to service identical use cases, so winners need to stand out with a superior conversational UX.

Just like weather apps, public transit apps are soulless and boring. We use them out of necessity and not delight. Enter Bus Uncle20, a bot that can tell you anything you want to know about the Singaporean bus system in his quirky, broken English and suggest funny things to do while you wait.

Bus Uncle21
Bus Uncle’s quirky humor makes public transit bearable. (Image: Mariya Yao, TOPBOTS6155514743403529252218148) (View large version23)

Comprehensive, detailed guides and maps for the bus system exist on the Internet to help expats and locals find their way home, but Bus Uncle’s conversational interface both simplifies and adds joy to a routine task.

Beware that the bot is not all fun and games. Like any proper Asian uncle, Bus Uncle stays in character by occasionally forcing you to solve math problems.

Bus Uncle's Math Problems24
(Image: Mariya Yao, TOPBOTS6155514743403529252218148) (View large version26)

If You Need to Handle Edge Conditions Link

E-commerce is a challenging space for bots due to product diversity and language variability. Many conversational shopping bots malfunction when users use unrecognized vocabulary or suddenly switch contexts. Such failures are usually technical in nature, where a bot simply doesn’t have the requisite data set or intelligence to handle the edge input.

ShopBot27 from eBay avoids common e-commerce bot UX failures by combining limited option menus with the ability to handle unexpected user input. While many shopping bots hem users into a narrow series of menus, ShopBot was able to quickly adapt when I switched from shopping for jeans to shopping for blouses.

eBay's ShopBot28
eBay’s ShopBot handles unexpected user input better than other e-commerce bots. (Image: Mariya Yao, TOPBOTS6155514743403529252218148) (View large version30)

Shopping is a difficult use case for chatbots to master. Superior conversational experiences in e-commerce bots are a function not just of great copy, but of powerful technologies that process natural language, keep track of shoppers’ contexts and preferences, and anticipate diverse needs accurately.

RJ Pittman, chief product officer at eBay, explains31, “Shoppers have complex needs, which are often not fully met by traditional search engines. The science of AI provides contextual understanding, predictive modeling, and machine learning abilities. Combining AI with eBay’s breadth of inventory and unique selection will enable us to create a radically better and more personal shopping experience.”

If You Can Humanize a Brand Link

Chatting is an intimate act we do with close friends and family, which is why chatting with a “brand” is often an awkward and strange experience. Strong conversational skills in a chatbot can overcome this barrier and establish an authentic connection.

Maintaining a consistent and compelling brand voice in chatbots is not easy. PullString32, a conversational AI platform founded by ex-Pixar CTO Oren Jacob, employs an entire department of expert Hollywood screenwriters to bring brands like Mattel’s Barbie and Activision’s Call of Duty to life.

Its demo chatbot, Jessie Humani33, is powered by over 3,500 lines of carefully selected dialog to create the impression that she’s your messed-up millennial friend who can’t get her life together without your help.

Jessie Humani34
Jessie’s dialog is carefully written to inspire an emotional connection. (Image: Mariya Yao, TOPBOTS6155514743403529252218148) (View large version36)

The Case Against Chat Link

Many bot industry experts believe the word “chatbot” sets the wrong expectation among users that bots should have human-level conversational abilities. The hard reality is that natural-language processing and artificial intelligence still have much progress to make before bots will impress you with their gift of gab.

Ted Livingston, CEO of Kik, a popular messaging platform with a thriving bot store, is squarely on the side of no chatting. “The biggest misconception is that bots need to be about ‘chat.’ What we discovered is that bots that don’t have suggested responses simply don’t work. Users don’t know what to do with an empty input field and a blinking cursor,” he shared37 at a recent bot conference.

Kik started building a conversational platform two years ago38, long before bots suddenly became cool. In the beginning, its bots allowed freeform responses the same way Facebook Messenger bots do now. What resulted was user confusion and error, as well as complaints from developers about having to deal with the unnecessary complexity of processing open-ended conversation. Kik now restricts user responses to a limited set of predefined options and intentionally makes typing freeform text difficult.

For example, when Sephora’s Kik bot asks what type of beauty products a user would like to see, the bot follows the question with a menu of suggested responses to choose from. A user has to go out of their way to tap “Tap a message” in order to type normally.

Kik's Sephora Bot39
Kik’s Sephora bot offers users suggested responses. (Image: Mariya Yao, TOPBOTS6155514743403529252218148) (View large version41)

When You Should Restrict Chat For A Better Chatbot UX Link

There are many cases in which designers of chatbots should restrict conversation to provide a superior experience. Below are a few common situations in which letting users type freeform conversational text complicates development and decreases your bot’s usability.

If User Error Would Lead to a Failed Transaction Link

1–800-Flower’s bot for Facebook Messenger originally gave users three options for flower delivery dates: “Today,” “Tomorrow” or “Choose another date.” The third option allowed users to type in dates freeform, which often resulted in error, confusion and an abandoned or failed transaction.

By removing the third option for users to type in a date manually, 1–800-Flowers actually increased the number of transactions and overall customer satisfaction. Restricting conversation helped it focus on its most important users, the ones who want to send flowers urgently.

1-800-Flowers Messenger bot42
1-800-Flowers increased the number of successful transactions by limiting date options. (Image: Mariya Yao, TOPBOTS6155514743403529252218148) (View large version44)

If Your Competitive Advantage Is Simplicity Link

Chatbots should give users the key advantage of completing tasks with fewer taps and context switches than regular mobile apps. Enabling open-ended chat can undermine this simplicity and add development complexity related to handling variable input.

An example is the simple meditation bot Peaceful Habit45 for Amazon Echo and Facebook Messenger. The bot is designed to help regular meditators build a daily practice and should be quicker to use than meditation apps.

On the Amazon Echo, a user can start a 5-, 10- or 20-minute meditation completely hands-free, with voice alone. On Facebook Messenger, the bot sends a daily reminder with limited user options, so only a single tap is required to start a meditation practice.

Peaceful Habit: Meditation Bot For Amazon Echo46
Peaceful Habit simplifies daily meditation. (Image: Mariya Yao, TOPBOTS6155514743403529252218148) (View large version48)

If You Cannot Easily Handle Unbounded Input Link

Many user requests appear simple on the surface but are extremely complex to handle in an open-ended conversational interface due to variability of vocabulary, grammatical structures and cultural norms. For example, a user can ask to schedule a meeting by asking any of the following questions:

When’s Bob’s next open time slot?

Let me know the next three times Bob can chat.

Is Bob available at 4 PM PST today?

Turns out the complexity of handling seemingly simple meeting requests requires powerful artificial intelligence capabilities. Several well-funded companies have emerged just to solve narrow scheduling challenges with specialized technology.

When you consider more complex requests, such as asking for restaurant recommendations, limiting conversations often means less confusion for both your bot and your user. Sure49, a bot that offers local restaurant recommendations, asks users to type in what they are craving, but it often can’t understand the responses.

Sure bot on Facebook Messenger50
Sure promises to recommend local restaurants but gets confused as soon as you request unexpected dishes. (Image: Mariya Yao, TOPBOTS6155514743403529252218148) (View large version52)

By contrast, a similar bot named OrderNow53 finds local restaurants that deliver and offers a limited menu of cuisines to choose from.

OrderNow bot on Facebook Messenger54
OrderNew presents users with fixed cuisine options. (Image: Mariya Yao, TOPBOTS6155514743403529252218148) (View large version56)

These examples demonstrate that complex artificial intelligence, machine learning or natural-language processing is not required to create a great user experience using a chatbot. As Ted Livingston, CEO of Kik, warns57, “AI is not the killer app for bots. In fact, AI holds most bots back. Bots are just a better way to deliver a software experience. They should do one thing really well.”

How Much Should Your Chatbot Chat? Link

How “chatty” your chatbot should be will depend on your users’ mental models of chatbots and the goals and needs your chatbot fulfills for them. Bots on Kik that only offer limited responses can be just as successful and engaging as Mitsuku and Jessie Humani.

Problems occur when designers do not decide up front who their audience is, how the chatbot fits into their business or brand strategy, what domains the chatbot will and will not cover, and what a successful experience should look like.

When you are deciding how much “conversation” to design into your chatbot experience and are defining the right level of engagement, answer the following questions:

  • How are you setting user expectations?
    If you brand your chatbot as a character or a human replacement, users will expect a minimum level of conversational ability. If your bot’s functionality is utilitarian or limited, then guide conversations towards specific outcomes.
  • Is your chatbot utilitarian or entertainment-driven?
    Mitsuku is an artificial-intelligence companion, so she’s required to master the art of conversation. On the other hand, a Slackbot that performs SQL queries or pulls CRM data has no need to support chat.
  • Does your chatbot reflect your brand’s voice?
    Major brands such as Disney58 and Universal Studios59 use chatbots to engage audiences beyond simple ad clicks and video views. A chatbot working as a brand ambassador needs to authentically reflect the domain and voice of the company it represents.
  • Is your chatbot a familiar service or product?
    Businesses such as 1–800-Flowers and Domino’s Pizza already have millions of buyers who use their websites, mobile apps and phone numbers to order products. Users who already know what you offer and what they like won’t require as much explanation and hand-holding.
  • Does your chatbot need to differentiate itself in a competitive market?
    Weather apps are a dime a dozen. Poncho the Weather Cat differentiates itself by having a distinct personality and delightful reactions, making the bot stand out against other weather services.
  • How strong is your technical team and AI platform?
    Building an adaptable and user-friendly conversational AI is incredibly challenging. Worswick invested over a decade to make Mitsuku the award-winning chatbot she is today. Each conversational AI platform has strengths and weaknesses that will affect your chatbot’s UX.
  • How strong is your writing team?
    In the world of bots, writers are the new designers. Do your writers understand how to write engaging, emotional copy that draws users in? Bots reflect the communication skills of their makers.

As natural-language understanding, machine learning and artificial intelligence improve, chatbots will inevitably become smarter and more capable in interactions with humans.

For now, just be sure that your bot either sticks with utilitarian offerings or stays within a comfortable zone of conversational topics. Take a cue from how Mitsuku gracefully avoids confrontation by excusing herself from a potentially awkward political conversation.

Mitsuku, developed by Steve Worswick60
Mitsuku avoids topics outside of the bot’s domain of expertise. (Image: Mariya Yao, TOPBOTS6155514743403529252218148) (View large version62)


Footnotes Link

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  31. 31
  32. 32
  33. 33
  34. 34
  35. 35
  36. 36
  37. 37
  38. 38
  39. 39
  40. 40
  41. 41
  42. 42
  43. 43
  44. 44
  45. 45
  46. 46
  47. 47
  48. 48
  49. 49
  50. 50
  51. 51
  52. 52
  53. 53
  54. 54
  55. 55
  56. 56
  57. 57
  58. 58
  59. 59
  60. 60
  61. 61
  62. 62

↑ Back to top Tweet itShare on Facebook

Mariya is the Head of Design & Engagement at TOPBOTS, a leading branding & marketing firm specializing in bots, chatbots, and conversational artificial intelligence. Her mission is to help brands & businesses design delightful conversational experiences that simplify technology and streamline your life.

  1. 1

    I honestly find this all to be incredibly creepy. I don’t want a computer pretending to be a human, or pretending to be my friend. It’s extremely weird. I very rarely find myself in a situation where I think “this needs a chat bot”. Feels like another piece of tech being rammed down our throats, the insistence being that it’s useful to us and makes things better. The cynical side of me just thinks it’s a creepy, nasty idea designed to extract more information, more data from users and bind them more to tech to increase dependancy and make more money for the corporates.

    • 2

      Michał Sadowski

      November 24, 2016 5:04 pm

      So much this;if you want genuine human interaction, then there is no shortcut but getting real people to converse. This is just untruthful. It also obscures interactions way more than regular interfaces.
      I want to buy jeans, not chat about them. Give me a neatly organized menu so I don’t need to worry about being misunderstood – or just having to type all those longass words.
      I mean, it works as a cool demo and a way to spend boring afternoon, but it gets tiring quickly.

      • 3

        Hava a look at a apps or websites. At first many of them were dehumanized. But now more and more have their personas and talk to us in a human way. We may not realize that, but on second glance that’s what works better, what engages us more and provides better experience.

        BTW. Very interesting cases in the article!

  2. 4

    Abegail Louise Acosta

    November 25, 2016 7:02 am

    Great article, this really helps me a lot as a newbie in this industry working in a professional website development company.

  3. 5

    I think, before jumping into implementing a chat bot you should be sure, if it is the best solution for your use case. Comparison to real life situations might be a good help, because humans happen to be the most skilled chat bots you can think of.

    For example most people entering a shop prefer to check the range of goods on their own before talking to a sales assistent. Or if you take out an insurance policy you would be very uncertain if the whole process would take place just as a conversation.

    The biggest benefit of the graphical interface is, that it shows you directly the limted options it is capable to fulfill. Talking to a chatbot is a little like returning to the command line. So I think in most cases it is not very usefull to provide an unlimited interface to an absolute limited set of possible choices.

  4. 6

    Very insightful thoughts. I know many of us are still in denial about chatbots, but they are definitely here to stay and will change the course of design. We need to learn to embrace the new paradigm of conversational UX. I learned a lot.

  5. 7

    Michael AtMessage

    November 30, 2016 2:34 pm

    I agree with the CEO of Kik and wouldn’t recommend anyone build automated “chat” until they are really good at it. My take is that every brand’s first foray into messaging, should be a webform, that happens over a messaging app. These are simple questions, valuable for the brand and it frames the “bot” as something that marketing needs to promote, and something that users will engage with for a specific purpose (and not be as likely to wander off).

    From a technology perspective it seems like the channels have taken two separate things (Messaging Apps + AI) that are separately really valuable and they’ve combined them. The problem is that one of those things doesn’t exist (AI). Why do we think that happened? I’m setting up a mailchimp account right now. I don’t expect mailchimp to run AI functions when someone responds to my newsletter. When I’m connecting up my FB Messenger APIs, why is AI assumed in this arena?

    Nice article!

  6. 8

    However, the increasing popularity of chatbot messengers made people talk that now apps are doomed. Cleveroad guys wrote an article about How to Build a Chatbot, look at
    In their opinion nowadays users require products with the features that are easy to use and do not need a lot of actions to use. That’s why apps may look a bit clumsy and inconvenient compared to chatbots.
    But for now, it’s hard to say whether chatbots will replace apps because they still haven’t proved themselves as a universal service delivery medium.


↑ Back to top