Building a Multi-Turn Intent Recognition Assistant from Scratch: Corpus, State, Routing, and Evaluation

The previous articles covered labels, corpora, models, state, task dependencies, and services separately. Now let’s put these components together to build a hotel assistant that can sustain continuous conversations: it can answer policy questions, look up mock orders, fill in missing booking details, handle modifications and conditional cancellations, and write mock results after confirmation.

This article explains how the modules divide the work, how to install and run the project, how to switch intent recognizers, and how to run the tests. Download the complete project, v3.0.

一、运行对话示例

After extracting the archive, enter the intent-recognition-lab directory. The default demo uses only Python’s standard library:

python3 batch3/assistant.py

It runs scenarios covering bookings, policy inquiries, conditional cancellations, order selection, modifications, and out-of-scope requests, printing each turn’s input, route, state, and result. The main state changes in the booking scenario are:

Here, “filling in the missing details” and “completing the booking” are two separate steps. When the user supplies dates, the program only forms a proposal; it performs the mock operation only after an explicit confirmation.

All order data lives in a local SQLite database. DEMO-A has a mock cancellation fee of 0, DEMO-B has one of 50, and DEMO-X belongs to a different demo user. These records make it possible to observe conditional and permission branches; none are connected to a real hotel or payment system.

二、模块分工与请求流程

Using “Cancel order DEMO-A” as an example, the HTTP or CLI entry point first supplies the session ID, request ID, state version, and text. The service reads the current state, checks whether the request is a retry of one that has already completed, and then passes the new request to the conversation handler.

The recognizer identifies cancellation as the target, and the slot extractor finds the order number. The state module binds the target to the object, while the routing layer queries the order’s ownership, status, fee, and version to construct a proposal awaiting confirmation. The service then returns confirm, while the order in the database remains booked.

On the next turn, “Confirm” is interpreted through the pending-confirmation state rather than being treated as a context-free classification problem all over again. Before execution, the system checks the order version and conditions again. On success, it updates the mock order, session state, and deduplication result within the same local transaction.

The component responsibilities map to these files:

The first batch’s recognition output contract and the second batch’s task graph remain separate. Single-task state and multi-task plans serve different purposes; connecting them through explicit entry points is easier to understand than packing every field into a single JSON object that keeps growing.

三、短回复与上下文处理

When the user says “Cancel the order” without giving an order number, the demo system lists two options: “1 DEMO-A; 2 DEMO-B.” On the next turn, “1” is interpreted against the currently valid options, resolves to DEMO-A, and the cancellation flow continues.

It does not fall back on a global rule saying that “1” always means “cancel the order.” Option records belong to the current session and task; a “1” in another session carries no such meaning.

Supplying a date or room type works much the same way. While a booking is in progress, “Beijing 2026-10-01 2026-10-03” can fill in the current details. If the user explicitly states a new booking goal, the system should switch tasks rather than mistake the new request for missing details in the old task.

The program therefore handles confirmation responses, option selections, and detail completions before calling the general-purpose intent recognizer. Reusing context that is already available can reduce unnecessary inference.

四、跨轮保留条件约束

Run “Cancel order DEMO-B if there’s no fee,” and the system queries the mock fee, finds that it is 50, and returns condition_false. It neither proceeds to confirmation nor cancels the order.

If the user initially says only “Cancel the order if there’s no fee,” the condition must be retained as part of the current task. After DEMO-B is selected on the next turn, the “no fee” constraint must still be checked; it cannot be dropped merely because that turn contains only a number.

Even if the condition holds when the proposal is created, it may change before confirmation. The attachment stores the order version used in the proposal, and the system rereads the fee and status before submission. If the version no longer matches or the condition has changed, it returns a conflict and lets the upper layer display the proposal again.

More complex multi-task requests can be run with:

python3 batch3/planner.py

The command accepts the task graph defined in the sixth article in the series. In an actual run, the order-lookup and room-lookup nodes complete, while the cancellation node stops at requires_business_validation. It shows how dependencies advance while retaining a confirmation entry point for write operations. Converting free-form expressions into arbitrary task graphs is still work for the recognition and planning modules; this command starts from an explicitly structured plan.

五、切换意图识别器

The default rules are easy to read and run and cover the expressions used in the tutorial. To use sentence-vector candidates, install the model experiment dependencies and start the service:

python -m pip install -r batch2/requirements.txt
python batch3/service.py --recognizer vector --db /tmp/intent-vector.sqlite3

The sentence-vector branch reuses the model and validation parameters fixed earlier and may send inputs with insufficient support to clarification. Switching candidate methods does not automatically relax order verification, parameter checks, or the confirmation flow.

When you select --recognizer llm, the adapter reads OPENAI_API_KEY and OPENAI_MODEL and calls the existing structured interface. The template is in batch3/.env.example; the service does not load it automatically, so the runtime environment must inject the variables. Even after the LLM returns a known single target, the result still passes through the same state and business constraints.

The current demo separates candidate recognition from simplified slot extraction and uses an explicit ISO format for dates. When extending it to dates expressed in natural language, complex references, or full LLM slot results, add normalization and field validation in the interface adapter layer instead of bypassing the state module and invoking tools directly. Rules, sentence vectors, and the optional LLM should not be assumed to provide the same recognition coverage.

This validation covers the local rule-based pipeline and sentence-vector integration. The LLM entry point retains a runnable configuration, but no online API calls were made. An eight-class banking student model is included to demonstrate the training workflow; its labels differ from the hotel domain and are not connected directly to the hotel router.

六、通过 HTTP 接口完成对话

On the first turn, submit a cancellation request:

After receiving the proposal awaiting confirmation at version 1, submit:

If the response is lost, resend the second request verbatim to retrieve the saved result. Do not generate a new request ID merely to retry. Reusing the same ID with different text returns an idempotency conflict; sending a new request with an old state version returns a state conflict.

When you run the example again, you can use a different session ID. Mock orders are persistent objects in the database, so a canceled order will not be restored automatically just because a new session starts. For a complete reset, use a new database file path.

七、运行测试与模型实验

Run these commands from the project root:

python3 -m unittest discover -s tests -v
python3 batch2/tests.py
python3 batch3/tests.py
python3 batch3/http_smoke.py

The first three check the contracts, state, task graph, and complete workflow. The last starts a real loopback HTTP server, sends requests, verifies the responses, and then shuts the server down.

Key test cases include booking after filling missing slots, an inquiry that does not trigger an operation, a condition retained across turns, an order change before confirmation, a fee change, duplicate requests that do not write twice, session isolation, rollback after a service timeout, and resuming a task after a process restart.

Run the model and metric experiments separately:

python batch3/evaluate.py
python batch3/train_small.py
python batch3/infer_small.py "My card was charged twice"

Functional tests check the program’s behavior, while classification experiments test the model’s judgment on the data. Both sets of results are stored in the project, so you can trace any failure back to the corresponding component while reading.

八、接入自己的业务

First replace the capability catalog and label definitions, then adapt the slots and business objects to your own domain. When replacing the mock order tool with a real service, explicitly define the identity used for authentication, the tool idempotency keys, how to query results after a timeout, how to handle fee changes, and how operations are confirmed.

The knowledge branch currently uses a small set of sourced policy entries to make the routing and returned evidence easy to inspect. After integrating document retrieval, you can continue to add chunking, hybrid search, reranking, and answer organization, but access to the current user’s orders should still go through permission-controlled business tools.

In a concurrent service, move time-consuming recognition outside the short transaction and then commit using a version check. Remote operations should recover through their own idempotency and reconciliation mechanisms. Store the model, prompt, labels, thresholds, and routing version together so that a production decision can be reproduced.

When integrating a new business, retain the user’s goal, context, and conditional constraints, and use them to choose the next step. Whether an operation is complete should be determined by the result persisted by the business system.

九、系列阅读导航

After reading and running the project, revisit the independent concepts in each article. It will be easier to understand why they need to remain separate and how they work together to shape the final result of a conversation.