How does an LLM-based chatbot improve customer autonomy while enhancing service quality?
Faced with changing usage patterns and the need to respond ever more quickly and efficiently to customer requests, G7 (France’s leading taxi company) has made an ambitious commitment: to leverage generative artificial intelligence to improve its customer relationship. To meet this challenge, the company collaborated with Dydu, an expert in conversational AI, to deploy a chatbot based on a language model (LLM).
This approach is resolutely focused on innovation, but also on control, mastery, and quality. Let’s take a look back at this transformative project.
A clear objective: streamline customer relations without dehumanizing them
G7 is a company built on a dual promise: to improve mobility for passengers and simplify the daily lives of drivers. With 32 million passengers transported each year, 15,000 affiliated drivers, and more than 10,000 corporate customers, customer relations are a central and ongoing challenge.
The company already had an AI-enhanced help center, but wanted to go further with a tool capable of understanding natural language, adapting to context, and responding reliably, while managing risks with a “restricted” and configurable LLM.
It was in this context that G7 saw an opportunity in LLMs: not to replace its teams, but to better guide customers, offer them greater autonomy, and free up time for high-value requests.
Three objectives were identified:
1. Enable customers to quickly find answers to simple questions without having to contact an advisor.
2. Ensure consistent response quality when human contact is still necessary.
3. Leverage data from interactions to identify weak signals and adjust business and product priorities.
An LLM chatbot co-built with Dydu: a hybrid and pragmatic approach
G7 and Dydu opted for a hybrid approach, combining the power of LLM and business structure. The chatbot, now deployed on the G7 website, is structured around three response levels:
- Level 1: for simple, frequently asked questions, the LLM generates a natural response based on the data available on the website.
- Level 2: for more complex topics (e.g., refunds, legal framework), responses are constructed using more structured formats (decision trees), with intelligent AI support.
- Level 3: for sensitive cases (e.g., a person with a disability who needs a specific service), the request is forwarded to an advisor via a ticket automatically enriched by the LLM.
This architecture combines responsiveness, quality, and security in request handling, while ensuring a smooth experience for the user. Above all, the Dydu solution was designed to be used directly by business teams, without requiring technical skills.
Measuring performance: a new challenge for the LLM era
The chatbot testing phase began with the sending of hundreds of simulated questions, which were analyzed manually to assess the relevance of the responses. However, although valuable, this method quickly proved too time-consuming to be sustainable. G7 and Dydu therefore worked together to implement an automatic performance evaluation system tailored to the specific characteristics of LLM models.
The first challenge was to define the right indicators. Traditional chatbot KPIs (such as the non-comprehension rate) become obsolete because an LLM almost always provides an answer, even if it is not sure of its relevance.
It was therefore necessary to design new evaluation criteria, focused on the confidence of the model in its own answers. The LLM now self-evaluates each response using several techniques:
- A factual analysis, based on the number and diversity of sources used.
- A semantic analysis, which measures the gap between the generated response and the reference documents to identify any discrepancies.
- A probability score that reflects the relevance estimated by the LLM itself.
At the same time, the chatbot also uses AI to analyze the intentions and classify the topics addressed by users. This makes it possible to identify the most frequent or sensitive topics and prioritize improvements, particularly in cases of low confidence scores. This intelligent and scalable system is an essential foundation for continuously managing and improving the quality of the responses provided by the chatbot.
Improving the chatbot: ongoing, structured, and iterative work
The approach adopted is based on a continuous improvement cycle structured around three levers:
1. Improving the algorithm with a dedicated prompt: to frame responses and avoid digressions or hallucinations.
2. Improve the knowledge base to make it suitable for use by AI: add missing content, enrich existing articles, and clean up obsolete data.
3. Create decision trees: for specific, complex, or sensitive cases in order to control the responses provided.
This approach has not only improved the quality of responses, but also enhanced the reliability of the service for users.
All of this is part of an iterative process, carried out regularly with the G7 and Dydu teams, well beyond the initial production launch. The project does not stop when the chatbot goes live: it continues to evolve to remain effective and relevant.
Managing escalations: uncompromising efficiency
Another important aspect of the project was managing escalations to human teams. G7 has adopted a clear policy: it is better to escalate a request than risk giving an inaccurate answer.
Thanks to AI, sensitive tickets are now:
- Automatically detected, based on context or certain keywords.
- Pre-filled by the LLM, which saves time for advisors.
- Prioritized according to their complexity or criticality.
The result: more responsive, more efficient, and better informed customer service.
Very encouraging results from the very first weeks
G7’s structured and gradual approach quickly paid off. From the very first iteration, the results were satisfactory:
Before optimization:
- 50% of responses were very satisfactory
- 45% of responses needed improvement (often due to a lack of clear content)
- 5% of responses were unsatisfactory
After improvement:
- 95% of responses were very satisfactory (including cases that were escalated appropriately)
- 5% of responses needed improvement (sensitive topics that were not suitable for 100% automated processing, but possible with a hybrid approach)
- Less than 1% were unsatisfactory
What’s next?
The chatbot is just a starting point. G7 already plans to:
- Extend it to other channels (social media, apps, etc.)
- Connect it to its back office to broaden its scope
- Adapt it to other audiences, particularly drivers
- Explore new formats, such as the callbot
A human partnership, driving innovation
Beyond the technology, this project is also the story of a strong partnership between the teams at G7 and Dydu. Clear project organization, regular communication, and a shared commitment to continuous iteration and improvement have enabled us to overcome the challenges of generative AI.
“AI doesn’t replace humans. It enhances them. It helps us serve our customers better, faster, and with greater relevance.” –Samir Dilmi, CRO at Dydu
