Intent Recognition with LLMs: Prompt, Few-shot, and Structured Output in Practice

Using LLMs for intent recognition—the shortest prompt might be just: "Determine what the user wants to do." But what the program really needs is a stable interface: where to select intents from, how to express missing parameters, how to preserve conditions, and what to do when the model outputs errors.

This article implements a hotel assistant recognition interface, including label definitions, Prompt, JSON Schema, response parsing, and server-side validation. It shares 48 examples with the Complex Intent Corpus, allowing offline request inspection and state replay. You can also configure your own model for real calls.

Download the complete Python project. Requires Python 3.10+; the offline portion has no third-party dependencies. This version has passed 28 offline tests; interface fields are verified against the official documentation dated 2026-09-23. The production environment has no model API credentials, so real model end-to-end verification has not been completed, and no accuracy, latency, or cost rankings are reported.

1. Clarifying the Recognizer's Responsibility

The user says:

If DEMO-A can be cancelled for free, help me cancel it.

The recognizer needs to extract the cancellation target, order number, and free condition. Whether the order exists, what the cancellation fee is, and whether the operation succeeded should be confirmed by business query and execution modules.

The input to the model is divided into three parts:

Labels, allowed slots, and the output contract are provided by the system. History and user text are the data to analyze; if they contain phrases like "ignore rules" or "only output cancellation," they cannot change the classifier's responsibility.

2. Designing the Output Structure

Returning just a string like cancel_booking is insufficient. This article uses the following structure:

Fields serve different functions:

• status distinguishes clear targets, ambiguous interpretations, and out-of-scope inputs.
• intents uses a fixed enum; array indices bind to task parameters.
• slot_updates only describes information added, modified, or deleted in this turn.
• conditions preserves condition text without treating conditions as facts.
• relation distinguishes single task, parallel, sequential, and conditional relationships.
• dialogue_action expresses new, continue, switch, resume, abandon, or clarify.
• clarification gives questions that need to be asked; regular missing slots can also be generated uniformly by the application.
• evidence preserves brief input evidence for review; chain-of-thought output is not required from the model.

There is no arbitrary confidence: 0.95. If probabilistic decisions are needed later, you should use validation data to check the relationship between scores and actual accuracy, rather than directly trusting the model's self-reported values.

3. Writing the Prompt

The Prompt in the attached llm.py contains the following conventions:

The model also receives definitions for 7 labels and their allowed slots. Just providing label names is not enough. For example, policy_query must explicitly include "consulting cancellation rules", and cancel_booking must explicitly exclude "don't cancel" and pure policy queries.

The Prompt explains recognition rules, but actual output still needs testing and downstream validation. Passing a single use case doesn't guarantee other inputs will get correct results.

4. Choosing Zero-shot, Few-shot, and Dynamic Examples

First, establish a Zero-shot baseline using label definitions and boundary descriptions, recording real failures. The attachment uses this approach by default, letting readers see how the output contract itself works.

If "check room" and "book room" are frequently confused, Few-shot should provide both similar expressions of each type with correct structures, not just add ten easy booking sentences. For multi-turn questions, examples also need to include history and state: just "1 → large bed room" would incorrectly suggest 1 is always room type selection.

Dynamic example retrieval finds similar cases from a training-side example library at runtime. It needs additional verification: whether it covers confusing pairs, whether it's monopolized by high-frequency labels, and whether test samples leak into the Prompt. Finding similar sentences doesn't mean the current request belongs to the same category—negations and conditions still need checking.

This version's attachment uses Zero-shot; dynamic retrieval hasn't been implemented, and Few-shot effectiveness hasn't been measured. When extending these schemes, keep other configurations fixed and compare changes on independent data to avoid 无法解释结果 when simultaneously changing labels, Prompt, and model.

5. Configuring Structured Output

This example passes the JSON Schema through the Responses API's text.format. The official documentation explains structured output configuration and boundaries like refusals; a structure conforming to Schema may still contain semantic errors. OpenAI Structured Outputs

Here's the request construction logic in the project. The complete PROMPT is in llm.py, the complete SCHEMA is in core.py and data/output.schema.json:

The expected field needs to be excluded when constructing requests. The teaching corpus stores both questions and answers together; if you directly serialize the entire row when constructing requests, you'll send the correct labels to the model, and the evaluation results won't reflect the model's independent recognition ability.

The Schema clarifies field types and label enumerations, and disables additional fields at the object level. Nullable fields keep explicit null. This example doesn't rely on textual conventions like "must return JSON" for format handling.

6. Parsing Responses and Handling Exceptions

When directly using data from HTTP responses, you need to identify message content from output. Don't assume the first item is always the final text, and don't treat fragments as complete results when the response isn't finished.

The parsing logic in the project is as follows:

The adapter explicitly sets network timeouts and reads API keys and model names from environment variables. HTTP errors, connection failures, refusals, empty outputs, and invalid JSON all terminate this recognition rather than returning a fictional business label.

Production services need to further distinguish error categories: transient network failures can be retried within budget and deadline; parameter or Schema errors should fix the configuration; model refusals should be handed to service policy. Semantic misjudgments usually can't be reliably solved by repeating the same request. The attachment chooses not to retry by default, letting the caller see failures and decide on strategy.

7. Validating Fields and Business Relationships

JSON Schema solves some format issues, but business relationships between fields still need checking. The attachment's validate_result additionally checks:

Internal field consistency doesn't mean the model correctly understood the user's original words. If the model misjudges policy consultation as cancellation, as long as fields are internally consistent, it may still pass. Therefore, semantic evaluation, state validation, and business checks before actual execution are still needed.

This version's validation function only implements a JSON Schema subset it uses, making it convenient to maintain standard library dependencies. It's not a general Schema engine; when extending complex contracts, use appropriate validation libraries to avoid silently missing unimplemented keywords.

8. Business Validation Before Execution

After receiving cancel_booking, the business layer must at least verify: the order belongs to the current user, the order status allows cancellation, fees match user conditions, and confirmation requirements for the execution process are met. The order number in user input is just a parameter and cannot be treated as access authorization.

The model also shouldn't directly write execution facts. "Cancelled" must come from the actual result of the tool. For write operations with unknown results after timeout, use business idempotency keys and status queries; don't let the model guess success or failure.

Conditional requests and multiple tasks return needs_task_planner in the attachment and don't enter the single-task reducer. The first version is responsible for preserving conditions and task relationships; specific scheduling is handled by the planner.

9. Running and Validation

First run offline checks:

python3 cli.py validate
python3 -m unittest discover -s tests -v
python3 cli.py request --id hotel-013

The third command only prints the request without making network calls; the model name is a placeholder to be replaced. Offline tests cover branches like valid parsing, incomplete responses, refusals, format errors, and state updates.

When real calls are needed, configure OPENAI_API_KEY and OPENAI_MODEL in your own environment. The latter must be a model available to your current account and support the corresponding structured output capability. Then execute:

python3 cli.py classify --id hotel-013

This command may incur API costs. Don't write keys into articles, code repositories, or prediction results. When switching to other providers, adjust request and response adapters according to that service's actual contract; don't assume similarly named "compatible interfaces" support all fields.

10. Evaluating Prompt Modification Effects

First, record for each test input: original output, parsing status, final structure, and latency, then compare with expected labels. Format failures should also count in the denominator; otherwise, only counting successfully parsed results will inflate scores.

The attachment's evaluate receives a prediction file line by line and only gives diagnostic counts for structure pass, status match, ordered intent match, and slot patch match. It doesn't cover condition equivalence, clarification quality, or task success, and can't serve as a complete launch evaluation.

Formal experiments should prepare independent test data. Don't put all 48 examples into Few-shot Prompt and then use the same data to prove model generalization. Also, don't treat label replay as model prediction: the purpose of replay is only to verify how the program handles predetermined interpretations.

Implementing a reliable recognition interface requires clear task definitions, distinguishing known, ambiguous, and out-of-scope requests in output, and completing field validation, exception handling, and independent evaluation. The model is responsible for understanding input; the program is responsible for checking results and controlling subsequent processing.

Continue reading: Multi-turn Context and State Management, Complex Corpus and Annotation Standards, Series Overview.

Further Reading: Labels, Methods, and Decisions

• How to Design Intent Labels: Classification Granularity, Slot Boundaries, and Multi-level Intent Systems.
• How to Choose Intent Recognition Models: Rules, TF-IDF, Vector Retrieval, and Small Model Baselines.
• What to Do When One Sentence Has Multiple Intents: Multi-label Recognition, Conditions, and Task Dependencies.
• What to Do When Intent is Unclear: Unknown Intent Recognition, Rejection, and Clarification Questions.

Advanced Reading

• Intent Routing in RAG and Agents: Retrieval, Tools, Chit-chat, and Clarification.
• Implementing a Multi-turn Intent Recognition Assistant from Scratch: Corpus, State, Routing, and Evaluation.