7 Key Questions You Should Ask When Building a Chatbot

Conversational AI is positioned to be the next frontier of an engaging consumer experience.We have experienced Google Now and Siri in our day to day lives for some time now and I must say these are very promising products. These will keep getting better and better in coming years and you can imagine a smartphone without any app at all but just an assistant to help you do all your tasks. Really exciting!

Indian enterprises too have been warming up to the use of conversational AI products with Chatbots being the popular flavor. Prominent use cases have been

• Customer Service: You can serve your customer requests and provide access to the information at a fraction of cost and at larger scale compared to the human capital. We can find such use cases with Ecommerce, Financial Services companies already.
• Transactional Purpose: You can perform transactions like Book a cab, order pizza recharge your mobile or buy a product. There are different B2C startups providing such platforms.

Decision to use chatbot as another consumer touchpoint needs to be evaluated very carefully rather than just Me too! approach. Thing with such products is either they work well and provide great consumer experience or they become part of troll belittling the promise of AI. Consumers can be unforgiving when it comes to using such products.

What goes into making such products? AI, Machine learning, Natural language processing are words synonymous with the rise of chatbots.Chatbots really are a mix of keyword/phrase matching, NLP, machine learning. Here are some key questions you should ask while evaluating such applications

1. Enterprise chatbots need to take into account domain and related vocabulary so as to understand what a consumer is saying along inferring the intent. In case of financial services words like home loan, life insurance or transfer etc. How does your chatbot handle this? What are the possible limitations?
2. There are different ways a consumer can pose a question. Dialect, typos all come into the party. How does your chatbot handle this? You should put yourself in customer’s shoes and try simulating as many conversation scenarios as possible. May be design an internal test with different users before you go live.
3. Every machine learning product needs training data, enough training data really. How does your chatbot have access to this training data? Remember more exhaustive and varied training set is more accurate predictions/classifications can a model do. Example classifying an intent or discovering likely action.You should also try to understand how platform will improve over time once it gains access to more sets of diverse conversations.
4. Pronoun disambiguation! Example repeat my last recharge . Here your chatbot needs to match my to an user account. In a conversational workflow it is quite important to handle such states.
5. How does your chatbot handle out of the context questions? Does it handle such conversations gracefully or not?
6. How does your chatbot handle overall conversation/dialogue? How close does conversation resemble real world natural language interaction? Experience this for yourself as a consumer.
7. Lastly, how do you judge effectiveness of chatbot as a customer touchpoint? What metrics do you have in place apart from obvious cost benefits?

As you try answering these questions and evaluating such products do remember that there will be limitations and understanding natural language is still an area of research. You need to be realistic and work with your partner in creating right experience for your customer. I hope these questions will help you out in your quest.