Case Study – How Does an LLM Chatbot Perform in Sales?

Before you get yourself an LLM chatbot, such as ours that is powered by OpenAI, you probably want to know what to expect.

In this case study, we will go through data and testing to see if an AI chatbot leveraging an LLM model powered by OpenAI indeed will increase sales.

How can we achieve more sales, and what steps will be necessary to leverage the chatbot to the fullest?

Contents

• 1.1 What is an LLM
• 1.2 About chatbots
• 1.3 About our experiment
• 2.1 The experiment
• 2.2 The implementation
• 3.1 Results

1.1 – What is an LLM?

Large Language Model (LLM): In the context of artificial intelligence and natural language processing, "LLM" is an acronym for Large Language Model. These are advanced machine learning models, like GPT (Generative Pre-trained Transformer) models, that are trained on vast amounts of text data and can generate human-like text in response to prompts.

1.2 – Chatbots

A chatbot is a computer program designed to simulate conversation with human users, especially over the Internet. These programs use natural language processing (NLP) and artificial intelligence (AI) techniques to understand and respond to user inputs in a conversational manner. Chatbots can be implemented in various platforms, such as messaging apps, websites, or standalone applications.

The primary goal of a chatbot is to provide information or perform tasks for users through a conversational interface.

There are two main types of chatbots:

Rule-Based Chatbots: These chatbots operate based on predefined rules and patterns. They follow a set of if-then statements to respond to user inputs. Rule-based chatbots are generally less flexible and rely on a structured conversation flow.

AI-Powered Chatbots: These chatbots leverage artificial intelligence and machine learning techniques, particularly natural language processing (NLP), to understand and respond to user inputs more dynamically. They can learn from interactions and improve over time.

Chatbots have become increasingly popular in various industries due to their ability to automate tasks, enhance user engagement, and provide instant responses. They are used in customer service, e-commerce, healthcare, finance, and many other sectors to streamline communication and improve user experiences.

Research from industry giants indicate that chatbots are responsible for an average of 67% increase in sales.

According to a report by IBM, businesses can reduce customer service costs by up to 30% by implementing chatbots.

A survey by Drift found that websites with chatbots experience an average 48% increase in revenue per chat hour and a 40% increase in conversion rate.

A study by HubSpot revealed that 82% of consumers expect an immediate response from businesses, and chatbots can provide round-the-clock support, meeting this demand.

According to a study by Intercom, 44% of online consumers say that having a chatbot available to answer their questions during an online purchase is one of the most important features a website can offer.

1.3 – The Experiment We’re Conducting

ClubDeMode is the business we will use to conduct the experiment. They are a high-end luxury clothing and accessories e-commerce website, that is meant to give great discounts on high-demand products.

The reason we are conducting the experiment on them, is because they have a business model where we can use several sales tactics, together with AI, to maximize the end results.

The tactics which I will go through in the next chapter.

2.1 – The experiment

Within our case study, we've integrated proven sales tactics to maximize the effectiveness of our chatbot. These tactics, encompassing special offers, scarcity, and personalized approaches, are further augmented by specific strategies that contribute to their success.

Personalized Approach: Gathering customer names at the outset allows us to create a more tailored experience. By utilizing these names throughout the conversation, we foster a sense of familiarity and engagement, and we establish rapport with the user.

Visual Engagement: Incorporating images when discussing products significantly enhances customer engagement and also increases conversions. Visual cues tend to captivate interest and evoke curiosity, nudging potential buyers toward a more detailed exploration of the offerings.

Special Offers: The introduction of special offers, particularly in the form of discounts, serves as a compelling incentive for customers. It not only encourages immediate action but also amplifies the perceived value of the products or services.

Scarcity Strategy: Incorporating scarcity into the conversation, such as indicating limited stock of a specific item, triggers a sense of urgency. This scarcity tactic prompts customers to act swiftly, amplifying their inclination to make a purchase before the item becomes unavailable.

The use of images and special offers plays a pivotal role in enhancing the chatbot's efficacy by not only capturing attention but also strategically influencing purchasing behavior.

2.2 – Implementation

The implementation of this chatbot is equally important as the configuration of its behavior. You can read the specifics under.

Automated Product Extraction: Utilizing Shopify's API, the chatbot automatically extracts product information from the Club de Mode Shopify store every day at 1 AM UTC. This automated process includes retrieving new products, removing discontinued items, and keeping accurate records of stock levels, prices, and other essential details. This ensures the chatbot operates with up-to-date information, preventing it from promoting products that are no longer available in the store.

Dynamic Sales Execution: The chatbot is configured to emulate the role of a proactive sales executive. Its primary focus is on maximizing product sales. For example, when only one item of a specific product remains in stock, the chatbot employs persuasive language to communicate a sense of urgency to the user. This tactic is designed to convey that the product is in high demand and encourage prompt decision-making, enhancing the likelihood of a successful sale.

3.1 Results

In our case study, the implementation of the chatbot in mid-August 2023 yielded remarkable results within the first month. By mid-September, the daily revenue experienced a substantial surge, increasing by an impressive 338%, from $17,557.08 to $44,968.61. Concurrently, the number of sessions saw a notable uptick of 125%, showcasing increased engagement with the website.

Sales exhibited a robust increase of 170%, while total orders demonstrated a significant rise of 150%. This surge in sales is further highlighted by a noteworthy 16% increase in the conversion rate, underlining the chatbot's impact on driving customer engagement and facilitating transactions.

An interesting aspect of our findings is the unintentional bug in our Shopify integration that was active from the 16th of October until 20th of October. This anomaly resulted in a temporary decline in sales compared to the preceding month's data, where the chatbot and Shopify integration were fully operational.

This emphasizes the pivotal role of the chatbot as a defining factor in driving these positive changes. Excluding the last four days of September and accounting for the bug, our analysis suggests a sustained and substantial increase of roughly 400% over time, affirming the enduring impact of the chatbot on overall performance.

Sources: IBM, Hubspot, Intercom, Juniperresearch