Illustration of an AI receptionist connected to organized knowledge documents, safety-training information, and a human support representative.

XO Safety · Development case study

Building XO Safety’s AI Receptionist: What Changed When We Started Testing Real Conversations

How focused knowledge, clearer product distinctions, and controlled tests changed the way our receptionist answered callers—and exposed the limits we still needed to address.

Development and testing through October 8, 2026

A caller says, “I can’t log in.”

The receptionist tells them where to sign in.

The caller explains that signing in is the problem.

The receptionist repeats the website address.

That exchange captured one of the central challenges in developing XO Safety’s AI receptionist: having relevant information available does not guarantee that the AI will use it to answer the right question.

The project began with research, product information, and carefully reviewed answers. It evolved through repeated conversations that exposed a different set of problems: ambiguous requests, lost objectives, competing information sources, and instructions that sounded reasonable but did not reliably control behavior.

This case study documents that progression through October 8, 2026. The evidence comes from project files, revision records, and test-chat transcripts supplied during development. These are observations from one configuration, not a description of RingCentral’s internal algorithms or a measured production success rate.

A front desk with clear boundaries

XO Safety sells occupational safety training products, including individual online courses, Train-the-Trainer courses, compliance kits, and written safety programs.

Those categories can be confusing to callers. Someone asking for “forklift certification” might need individual instruction, materials to train employees, or help becoming an internal trainer. Similar wording can conceal very different purchasing decisions.

The intended receptionist therefore needed to do more than recite product descriptions. It needed to identify the customer’s objective, answer routine questions, and recognize when staff assistance was appropriate.

Accuracy mattered especially because course completion, employer certification, practical evaluation, and authorization to perform work are different things. A confident but imprecise answer could mislead someone about what a training purchase accomplishes.

We established a front-desk role: provide verified product and support information, respect human requests, and avoid making site-specific safety determinations.

All human routing ultimately used one operator destination. The project did not require a complicated departmental directory.

Research came before the answers

The initial research covered official RingCentral documentation, XO Safety’s public website, relevant Academy information, and authoritative regulatory sources.

Individual product-page FAQs received a separate audit. The recorded inventory covered 219 public product URLs and identified 74 displayed FAQ questions across 16 pages. Questions were grouped by their underlying intent while preserving product-specific answers.

That distinction mattered. Two pages asking whether a certificate is included might refer to different things: an issued online-course completion certificate or an editable document supplied in a training kit.

We also kept website repetition separate from actual customer frequency. A question appearing on many product pages showed that XO Safety considered it important; it did not prove that it was a frequent telephone question.

The owner reviewed answers individually, approving wording, requesting changes, and deleting unnecessary entries. By October 7, the original core package contained 147 approved active entries. That approval history remained separate from later experimental expansions.

A further design decision made the knowledge portable: maintain structured records and sources, then generate platform-specific files. RingCentral was the first application, but it would not be the only possible consumer of the knowledge.

Could a text file act like a decision tree?

Early experiments explored whether the knowledge base could control a branching conversation.

We supplied questions with several options and a distinct answer for each option. Some interactions followed the intended branches. That encouraged a second experiment: could a branch answer be restricted to situations where the parent question had already been asked?

The results were uneven.

Synthetic examples helped isolate the behavior from real product knowledge. In an “Amber Garden” test, a bare response of “green” produced a conversational reply about comparing colors. A more explicit question naming the test and the green option retrieved the intended hedgehog answer.

Other option-based tests were interpreted as transfer requests or produced unknown-information responses.

These experiments did not establish that AIR always ignores “IF” instructions. They did show that a text-based decision tree was not a dependable substitute for an application-controlled workflow.

We changed direction: branch questions could help callers identify their needs, but focused answers should remain directly retrievable. A caller should not have to complete a particular sequence before receiving information.

From five legacy files to caller goals

The uploaded legacy knowledge was organized into five files:

Compliance Kits.txt
Online Train the Trainer.txt
Online Safety Training.txt
Safety Plans.txt
Customer Service.txt

That structure reflected the business’s product categories. It was useful reference material, but callers often approached the business through a problem rather than a category.

We reorganized the shared information around five provisional caller goals: choosing training, getting course help, finding certificates, resolving purchase questions, and arranging group training.

Detailed product references remained available underneath those entry points. A price file was added later.

The resulting upload structure was:

01-Choose-Training.txt
02-Online-Course-Help.txt
03-Certificates.txt
04-Purchases-and-Orders.txt
05-Group-Training.txt
06-General-Information-and-Human-Help.txt

07-Compliance-Kit-Product-Answers.txt
08-Trainer-Course-Product-Answers.txt
09-Worker-Course-Product-Answers.txt
10-Written-Plan-Product-Answers.txt

11-Current-Product-Prices.txt

Each entry used a focused question, useful caller variations, and a short spoken answer. Sources, review notes, tests, and configuration instructions stayed outside customer-facing uploads.

The five goals were organizational choices, not rankings established by call analytics.

Remembering the subject was not enough

The receptionist could sometimes retain a subject across turns. But that did not mean it retained the customer’s objective or the selected product.

In one forklift conversation, AIR listed an operator course, a trainer course, and a compliance kit. When asked about price, it distinguished the two online courses. When asked, “How long is it?” it selected the operator-course duration without establishing which product the caller meant.

Another conversation began with:

“How many people can I train?”

AIR answered the unlimited-employee-use question for compliance kits.

The caller followed with:

“How about online?”

Instead of continuing the quantity discussion, AIR explained practical training and workplace evaluation requirements.

It retained a relationship to training, but lost the question about how many people were covered.

This prompted more explicit distinctions between products and intents. Ambiguous forklift duration questions received a clarification. Online enrollment quantities were separated from compliance-kit reuse and internal LMS permissions.

We could not make conversational memory reliable simply by adding more information.

Login problems needed separate answers

Course access became one of the clearest before-and-after examples.

Earlier tests mixed several problems:

  • Finding the course website.
  • Signing in at the wrong website.
  • Missing registration information.
  • Receiving an invalid-password error.
  • Finding an enrolled course after signing in.
  • Accessing purchased download files.

Broad sign-in questions sometimes retrieved menus or repeated the Academy address. Registration instructions appeared in one conversation but were reported unavailable in another.

The restructuring created separate entries for these states.

After the owner uploaded the revised files, “I can’t log in” produced:

“Are you trying to access an online course or files from a product purchase?”

When the caller explained they were trying to access a course at xosafety.com, AIR directed them to xosafety.academy.

When they then reported an invalid password at Academy, it offered operator assistance rather than repeating the website address.

That sequence showed a useful improvement. It did not establish that every login phrasing would work consistently: a later personality test again skipped the course-versus-download distinction.

Password recovery remained a proposed addition pending confirmation of the actual feature and instructions.

Better selection still needed better resolution

Employee-training requests also improved.

After restructuring, “I need to train my guys” prompted:

“Will your employees take individual online courses, or will you conduct the training using materials?”

When the caller specified silica training and said they wanted to conduct it themselves, AIR identified a construction compliance kit rather than defaulting immediately to an individual online course.

However, the conversation exposed another limitation. The caller worked in manufacturing.

AIR recognized the construction mismatch and suggested a general-industry written program. It subsequently explained that the program did not specifically list separate employee-training materials.

When asked what to buy for training manufacturing employees, it repeated the written-program option.

The system had become more precise about product scope, but the original need remained unresolved. The appropriate outcome was to acknowledge the gap and offer staff help, rather than keep presenting a product that had not been established as suitable.

Prices required their own evidence

Pricing initially failed because the available answers did not supply current product prices. We added a dated storefront capture covering 219 products and 582 variants.

Variant-level information mattered. Delivery choices and license options could change a kit’s price.

Tests retrieved specific forklift prices, including the $799 Online LMS option. But “What is the LMS Unlimited option?” followed by “How much is it?” produced a claim that pricing was unavailable.

Once the caller named the forklift kit and its license option, AIR retrieved the price.

The gap was therefore clarification, not simply missing data. “LMS Unlimited” identified an option shared by products; it did not identify which product to price.

The price file remained a snapshot, not a live quote. Bulk discounts, enrollment quantities, and Academy-account inclusions required separate evidence.

Saying “I’ll transfer you” did not mean transferring

One receipt test revealed an important boundary.

AIR said it would connect the caller for order help. The caller thanked it, and the conversation ended with a goodbye. No transfer announcement appeared in that exchange.

We therefore separated knowledge answers from application actions. The knowledge base could explain a process or offer assistance. Actual routing belonged in RingCentral’s transfer configuration.

Official documentation describes Transfer by context as rules that connect caller needs to configured destinations. We tested separate rules for human requests, frustration, and certificate corrections.

A certificate-correction comparison was promising: with the rule enabled, the exact request produced an immediate transfer announcement; with it disabled, AIR explained the email-and-operator correction process.

Equivalent wording did not always produce the same outcome, however. Another certificate-name request prompted confirmation.

Explicit human requests also continued receiving:

“Would you like me to transfer you?”

That happened even after the owner confirmed that Offer help before transfer was disabled.

The tests demonstrated that contextual rules could influence behavior. They did not establish reliable immediate transfer or successful connection to a person.

A transfer rule made troubleshooting worse

We tried putting a two-failure troubleshooting threshold into contextual routing.

The intended behavior was straightforward: give one solution, try a second if needed, and escalate after both failed.

But AIR sometimes transferred after the first reported failure—even when the caller explicitly said they had not tried the second suggestion.

The owner then identified a 500-character limit in the rule field. Earlier instructions might have been truncated, making some previous results harder to interpret. We shortened the rule and tested again.

Early transfer still occurred.

With the frustration rule disabled, an otherwise similar sequence progressed from logout/login to another browser, then offered human help after both failed.

We removed troubleshooting-count logic from the context rule. Focused failed-state answers had performed better in those observed sequences.

Later Balanced-detail testing again produced one step, a second step, and then a human offer. That was a successful conversation, not proof of a universal counter.

A competing skill disclosed unwanted information

The revised knowledge file specified Portland, Oregon as the location answer. Nevertheless, AIR disclosed the residential street address.

It correctly answered “no storefront” on the next turn, but the unwanted disclosure had already occurred.

The owner disabled the Location and business hours skill and repeated the tests. AIR then returned the city and state only, correctly explained that there was no storefront, and answered business hours from the knowledge files.

This was one of the strongest configuration comparisons in the session. It supported the conclusion that the location skill had supplied the unwanted detail.

It also showed why adding a better answer does not necessarily resolve a conflict. Competing skills and sources need inspection.

Personality changed the introductions more than the answers

We compared Formal, Neutral, and Friendly personalities, then Concise, Detailed, and Balanced response settings.

Personality mainly changed introductory language:

“Let me look into…”
“Let me check…”
“Alright, I’ll take a look…”

The repeated filler survived all three personalities.

For the same product-comparison question, Concise, Detailed, and Balanced produced essentially identical substantive answers. Troubleshooting remained similar, and explicit human requests still received an extra confirmation question.

These observations were limited to the questions tested. They did not prove that response settings never matter.

They did suggest that rewriting weak knowledge entries was more useful than expecting Detailed mode to expand them into better answers.

What we have learned—and what remains uncertain

The strongest practical finding was that focused questions and carefully scoped answers supported useful conversations. Separating course access from downloads, identifying employee-training methods, and preserving product-specific prices produced observable successes.

Several assumptions remained unproven:

  • Splitting identical content across multiple knowledge bases improves retrieval.
  • Alternative-question lists have special handling beyond ordinary meaning matching.
  • Text-file parent conditions reliably restrict branch answers.
  • Context rules accurately count troubleshooting failures.
  • General sentiment detection explains frustration responses.
  • A transfer announcement means a human answered.

Controlled tests for alternative-question lists and adding a large confined-space regulation remained queued.

Where the system stands

The latest restructuring produced eleven replacement text files and matching portable records, retaining the 106 product scopes from the legacy knowledge set.

The build recorded 74 changes and supplied 60 regression questions plus six multi-turn scenarios. Local checks verified consistency, product retention, and preservation of price answers. Those checks did not evaluate AIR’s actual retrieval.

The owner subsequently confirmed uploading the files and supplied post-upload results. This matters because preparation-time manifests still describe the package as uninstalled and untested; the conversation records document the later implementation and selected tests.

Remaining work includes resolving enrollment-unit questions, confirming account-recovery instructions, improving unresolved product-selection handoffs, and establishing approved emergency behavior.

The most revealing measure of progress was a small conversation. A caller said they could not log in, explained they were on the store website, and then reported an invalid Academy password. After restructuring, the receptionist handled each distinct problem instead of repeating one generic answer.

That is the standard this project is moving toward: understand what changed in the caller’s request, give the relevant verified answer, and recognize when the next useful step belongs to a person.

Sources and project notes

  • Project changelog: Development chronology, owner approvals, revisions, and preparation-time status, maintained in CHANGELOG.md.
  • Product-page FAQ audit: Captured website coverage, extraction method, and product-specific distinctions, maintained in research/product-page-faq-audit.md.
  • Restructuring records: File organization, source lineage, change maps, and local validation results in the October 8 restructuring package.
  • Testing evidence: User-supplied AIR test-chat transcripts and the project’s observed-test summaries. No production performance rate or confirmed phone-handoff rate is claimed.
  • RingCentral documentation: Managing skills for the AI Receptionist in the Admin Portal, reviewed in the project on October 8, 2026.
  • Business sources: XO Safety, XO Safety Academy, and owner-confirmed product and support information.

The supplied image-catalog workbook was unavailable in the execution workspace. Catalog images were omitted rather than substituted with unverified assets. No residential address, credentials, or caller contact details are reproduced.

Leave a comment