Our previous article explored the Minimum Viable Architecture (MVA) concept. This article explores how the MVA concept can be applied using a fictional example of a chatbot that interacts with traditional insurance systems (such as policy administration systems) and additional data sources that may be external to the enterprise (reconstruction cost data, home valuation) to answer questions that a homeowner may have about their policy and coverages. For the purposes of illustration, we will focus on homeowner questions that relate to the replacement cost of a dwelling.

Implementing a chatbot using the MVA approach

We have chosen the chatbot example because it has a simple user interface that masks a set of complex problems relating to data integrity, concurrency, latency, and responsiveness. A chatbot is a software service that can act as a substitute for a live human agent, by providing an online chat conversation via text or text-to-speech.

This is a good candidate for inclusion in the scope of many software systems, such as home insurance systems used by insurance carriers. Home insurance specialists in insurance agencies and insurance companies often receive a high volume of queries by telephone calls and emails from their customers impacted by a natural disaster such as a wildfire. Homeowners affected by the natural disaster want timely answers to their questions, but the sheer volume of calls means that agents can’t respond in a timely manner to urgent homeowner questions. During more normal times, homeowners often have simple but non-urgent questions that they want to ask but don’t want to trouble an agent. A chatbot can improve customer satisfaction by providing more timely information while letting agents handle more complex tasks that require non-trivial decision-making abilities. 

Initial MVA: a simple menu-driven chatbot

Our first step in creating an MVA is to make a basic set of choices about how the chatbot will work, sufficient to implement the Minimum Viable Product (MVP). In our example, the MVP has just the minimum features necessary to test the hypothesis that the chatbot can achieve the product goals we have set for it. If no one wants to use it, or if it will not meet their needs, we don’t want to continue building it. Therefore, we intend to deploy the MVP to a limited user base, with a simple menu-based interface, and we assume that the latency delays that may be created by accessing external data sources to gather data are acceptable to customers. As a result, we want to avoid incorporating more requirements—both functional requirements and quality attribute requirements (QARs)—than we need to validate our assumptions about the problem we are trying to solve. This results in an initial design which is shown below. If our MVP proves valuable, we will add capabilities to it and incrementally build its architecture in later steps. An MVP is a useful component of product development strategies, and unlike mere s, an MVP is not intended to be “thrown away.”

An open-source, reusable framework (such as RASA) can be used to implement a range of customer service chatbots, from a simple menu-based bot to more advanced ones that use Natural Language Understanding (NLU). Using this framework, the initial MVA design supports the implementation of a menu-based single-purpose chatbot capable of handling straightforward queries. This simple chatbot presents a simple list of choices to its users, on smartphones, tablets, laptops, or desktop computers. Its architecture is depicted in the following diagram:

The chatbot interfaces with the following backend services:

• Insured coverages: This service is internal to the insurance carrier. It will be accessed via an API and its latency is expected to be low to moderate.
• Reconstruction costs: This service is external to the insurance carrier and will be accessed via an API. Its latency is unknown.
• Home valuation: This service is also external to the insurance carrier and is also accessed via an API. Its latency is also unknown.

As described in our previous article, we start the MVA process by making a small set of fundamental choices about the solution and use a simple checklist to ensure that we make appropriate architectural decisions. Our checklist includes the following items:

• Security—security requirements need to be taken into account for the MVP. A user should be authorized to access the information that the chatbot is retrieving, so the chatbot should capture user credentials and pass those credentials to the backend services for access validation.
• Monitoring—we believe that every application should provide basic monitoring capabilities to monitor performance and gather information about most potential system issues that may be encountered during the initial rollout.
• Platform—we have decided that the MVP will be hosted on a commercial cloud platform.
• User interface—we feel that a simple menu-based interface is adequate for the MVP, but this item may need to be revisited depending on the feedback from our initial users. The MVP U/I is available on smartphones, tablets, laptops, and desktop computers. 
• Latency and responsiveness—while we don’t have immediate concerns about latency and responsiveness, since the MVP deployment will be limited to a small user base, we know that if the MVP is successful we will need to expand its user base. Latency and responsiveness may become potential issues and would need to be addressed. Latency metrics will be included as part of the basic monitoring capabilities

The following choices are not of concern at the moment but may become concerns at a later time if the user base and usage grow substantially.

• Concurrency—no concerns at this time, assuming the user base doesn’t become too large.
• Throughput—no concerns at this time because the amount of data handled in queries is very small.
• Scalability—no concerns at this time but it may become an issue if the chatbot user base expands significantly.
• Persistency—no concerns at this time since very little data is stored by the chatbot.

For your own applications, consider this checklist a reasonable starting point that you may need to adapt or expand depending on the technical issues you are exploring. 

After the MVP is delivered, users seem relatively pleased with the capabilities of the product but express that they find a menu-based interface too limiting; even the simple menus used in the MVP are rather cumbersome, and expanding the menu options will only worsen the user’s experience, especially on smartphones and tablets. They would like to converse with the chatbot more familiarly, using natural language.

Implementing a Natural Language Interface

The open-source chatbot framework used for the MVP implementation includes support for Natural Language Understanding (NLU), so we will continue using it to add NLU to the capabilities of the chatbot. Using NLU transforms the simple chatbot into a Machine Learning (ML) application.

Switching to an NLU interface changes the chatbot’s architecture, as shown in the diagram below. Data ingestion and data preparation in the off-line mode for training data are two architecturally important steps, as well as model deployment and model performance monitoring. Model monitoring for language recognition accuracy as well as throughput and latency is especially important. Business users use certain “industry jargon” terms which the chatbot gets better at understanding over time.

The evolved architecture includes two models that need to be trained in a sandbox environment and deployed to a set of IT environments for eventual use in production. The two models can be thought of as an NLU model relating to users’ questions, and a Dialog Management model (DM) pertaining to the chatbot’s answers. More specifically, the NLU model is used by the chatbot to understand what users want to do, and the DM model is used to build the dialogues so that the chatbot can satisfactorily respond to the messages. Both the models and the data they use should be treated as first-class development artifacts that are versioned.

Addressing the backend services latency

Reconstruction costs and home valuation data are, at least initially, maintained and stored by other organizations; only insured coverage information is under the insurance company’s control. Even at low levels of usage, users may experience undesirable latency as the chatbot gathers necessary data from the two external data services. The assumption that latency delays are acceptable to customers can and should be tested early as part of the MVP phase using the initial menu-driven UI.

If latency, caused by accessing external services, appears to be undesirable, the architecture must be adapted to cache external service data locally (or at least at the same location as the insured coverage data) and periodically update cached data. Assuming that home valuations and reconstruction costs do not change much, if at all, over short time periods, caching that data seems like a reasonable trade-off. However, this assumption needs to be tested as well, and the cost of the impact of latency delays on customers’ experience should be compared to the cost of maintaining cache coherency to determine if caching is worth the time and effort. In addition, the MVA checklist should be revisited frequently to ensure that the assumptions made at the beginning of this process are still valid and that the architectural choices remain satisfactory as the MVP evolves into a full-fledged product. 

Conclusion

At first glance, a chatbot application doesn’t seem like something that really requires much architectural thought; frameworks abound that provide most of the building blocks, and developing the application appears to involve little more than training some NLU models and integrating some off-the-shelf components. But as anyone who has experienced the challenges involved with obtaining useful information from many chatbots will know, getting the chatbot application right isn’t easy and the cost of getting it wrong can dramatically affect customer satisfaction. Even a simple application like a chatbot needs an MVP and an MVA.

With more complex applications, the issues that the MVA needs to address will vary depending on the objectives of the MVP. While the MVP tests whether a product is worthwhile from the customer’s perspective, the MVA typically examines whether it is feasible to deliver that solution to customers, from a technical perspective, and whether that solution can be effectively supported. The MVA must also look beyond the MVP to at least provide the option of dealing with issues that must be handled if the MVP succeeds, for fear that a successful MVP leads to the situation where the organization can’t afford to sustain the product over the long run.