Select Page

Bot Training, creating a system feature to manage offers keyword tagging.

The project: create a single and unified platform for all digitised certified nationwide and non-nationwide seminars offered by a chamber of commerce entity through two main drivers:

  • innovation: through some sort of artificial intelligence;
  • performance: provide all the necessary information related to these seminars in the system as well as a the booking management;

Project Breakdown

For the innovation aspect, we developed a chatbot search interface as an artificial intelligence component, which required the implementation of the Bot Training Management feature, to enable the training and optimising of the chatbot models search interface results.

Duration

2019 — on-going

Project Type

PWA Back-office system

Responsabilities

Information Architecture
Wireframing
UI/UX Design
User-Testing

Tools

Atlassian: Jira
Atlassian: Confluence
Atlassian: BitBucket
Google Docs
Pencil & Paper
Figma
Invision

Initial Hypothesis

In an initial phase, the main hypothesis was that users would search via a “main keyword” which we categorised as the “main topic” which defined how we built the bot sequence flows.

A little over a year later with the chatbot feature being continuously improved under a Beta Testing Version for a year, me and the team finally received some important feedback on users insights towards the chatbot sequences which directly affected the bot training feature in the back-office as well.

The main issue identified in the chatbot was:

  • unrelated suggested results based on the users keywords input requests;

Whereas, the issues found in the bot training feature were:

  • uncertainty as to what keyword should be assigned as “main topic” and which could just be classified as “subtopics”;
  • confusion and frustration towards the restriction on assigning one “main topic” keyword.

Feedback Insights

“Either when searching for specific or more general offers, the bot sometimes retrieves unrelated offers to the specific users keywords request”

“It feels confusing what should be the main keyword assigned to “topic” and other keywords assigned to “subtopics” for an offer, when trying to optimise the search results of the bot through the Bot Training Management feature in the back-office”

“Why is it not possible to assign more than one keyword to a “topic” to optimise the bot retrieved results?”

Problem

From the insights and information gathered, what we concluded was that the employees responsible for the tagging of offers in the back-office bot training management, felt frustrated and confused by the current system at the time for multiple reasons, but the main one being — because we had created a chatbot model which required the user to provide information with a certain sequence (e.g. topic, date, location).

Through this sequence, we ended up restricting the user to always have to provide a “topic” keyword, which if not found or unrelated, would result in the process of failing at retrieving similar offer options when the specific keyword was not associated with multiple offers in the system.

Challenge

Re-design and optimise the bot training management model in a way that would no longer restrict the keywords assignment to an offer and therefore negatively influence the retrieved search results in the chatbot.

“How to improve the systems training model in order to provide a simpler, less restricted and results oriented experience?”

Concept Iteration

Together with the rest of the team, we started by creating a document in which we defined which functionalities were not working and how we could improve the model.

Throughout this process, one thing we all agreed on that became very clear was that there was no real benefit (neither for the employees using the back-office system, or the users searching for offers in the chatbot) in the categorisation of keywords by “topic” and “subtopics”, so instead, we decided:

  • we would no longer support the “topic” entity, by removing it from the systems model;
  • and sub levels of keywords would also no longer exist, therefore instead we would convert “topic” and “subtopic” keywords into general keywords.

Then we started by defining the new logic for this improved and optimised version of the bot training management system, the main goal being that every offer would now just have a set of keywords associated to it. The advantage of this system:

  • making the bot faster;
  • removing the limitation of assigning a single keyword (topic) to a selected group of offers;
  • allowing keywords allocation optional;
  • easier to identify offers with no associated keywords.

User-flow and Wireframes

Then I proceeded on to redesigning the flow to simplify the process.

Based on this new user flow roadmap, I quickly sketched some wireframes to help us define the development strategy for the refactoring of the feature, and proceed with the high-fidelity mockups redesign.

Design Iterations

Due to deadlines and phase of the project, no prototyping was required at this point, but I made sure to create hi-fidelity mockups of all functionalities portrayed in the user flow roadmap that I created in a previous stage of the process.

Stats Insights

There are currently over 80+ Legal Entities sending their offers to the system.

The system stores 2.800+ offers, provided by the 80+ Legal Entities suppliers.

Currently, there a more than 40+ users actively using and testing the platform.

Outcome

The new bot training management, now called keywords assignment is much simpler and easier to use. By removing the layers of tagging of the offers with a “main topic” and “subtopic” keywords, and instead grouping the two elements into one, the user is less likely to assign a specific categorising “main topic” keyword to multiple offers, which the system previously assumed as the main keyword to retrieve results.

Learnings

Having worked on the project since the very start, it has definitely given me a very deep understanding of the project, which when having to make these kinds of iterations which have a huge impact on the systems core functionality, I felt was really valuable and made it easier for me to iteratively find a solution that would meet the needs of the business and its customers.

Bonus Screens

Some designs of the other really cool features I’ve created for this project. Feel free to ask questions!