AI Decision Models: Distinct from LLMs

Decision models have recently emerged as a distinct approach to AI. Jev, developed by TypeSafe AI, is a prominent example that generates structured choices, scores, and confidence estimates rather than generating free-form text.

Although traditional AI models have long been used for predictions and classifications, Jev is specialized in producing structured decisions that software can consume directly. This method provides an alternative pathway for building AI applications, particularly when a system must evaluate data and determine the subsequent action.

How does Jev function?

In a standard LLM workflow, you submit a prompt and receive generated text. Even when JSON is requested, the application must parse and validate the response before it can be utilized.

Jev operates differently. By providing context, defining specific questions, and specifying the expected answer type, you receive structured results that your application can process immediately.

Jev supports three primary question types:

• Choice: Selects an option from a defined list, such as billing, technical support, or sales.
• Score: Rates an item against an ordered scale, such as low, medium, or high priority.
• Yes or no: Estimates the probability of a statement's truth, such as whether a support ticket requires human intervention.

Multiple questions can be posed about the same input in a single request. For instance, a support application could classify a ticket, rate its urgency, and assess the need for human review. Jev returns these results in a structured format, enabling the application to determine the next step.

Decision models vs. LLMs

LLMs and decision models serve different functions. An LLM excels at explaining decisions, generating content, writing code, and reasoning through complex tasks. Jev is optimized for focused decisions with clearly defined outputs.

This does not mean LLMs are incapable of making decisions. They can classify information and return structured data. However, Jev is specifically designed for this task, eliminating the need for the application to interpret free-form responses.

This makes Jev highly useful for repetitive, well-defined decisions. For tasks requiring detailed explanations, creative output, or complex multi-step reasoning, an LLM remains the more appropriate tool.

What are confidence scores?

A key feature of Jev is that its outputs include probabilities and confidence metrics. These values assist developers in assessing how strongly the model favors a particular answer.

For example, a customer support application might query whether a ticket requires escalation. If the model indicates a high probability, the application could forward the ticket to a senior support team. If the result is uncertain, it could route the ticket for human review.

However, high confidence does not guarantee accuracy. A model can be highly confident yet incorrect. Developers should validate results against real-world examples and set thresholds based on the potential consequences of each decision.

It is also crucial to distinguish between the probability of an answer and a separate confidence score. The exact interpretation depends on the output type. Applications should adhere to the model's documentation rather than assuming all returned numerical values represent the same metric.

Where can decision models be useful?

Classification and routing

Many applications require sorting incoming information before processing. A support platform might classify requests by topic and assign them to the appropriate team. Similarly, business applications can categorize documents, emails, or customer feedback.

Jev can return a category from a predefined list, allowing the application to route items without parsing generated paragraphs.

Scoring and evaluation

Decision models can also evaluate information against a defined scale. For instance, an application could rate a support ticket's urgency, score a document's relevance, or assess whether a response meets specific criteria.

These results can be used to rank items, prioritize work, or identify cases requiring further review. The quality of these decisions depends on the model and how well the criteria align with the task.

AI agents and tool selection

AI agents utilize models to determine actions. An agent might select a tool, classify a request, or determine if a task needs additional information.

While a general-purpose LLM can make these decisions as part of its reasoning process, a decision model can handle narrower choices before the agent takes action.

For example, a custom workflow using Hermes Agent could leverage Jev to classify an incoming task before passing it to the agent. It could also help determine which workflow should handle the request. This requires custom integration rather than relying on assumed native support.

Decision models should not be treated as standalone security controls. If an agent can execute commands or modify files, permissions and deterministic rules must enforce allowed actions. A model's classification can inform this process but should not replace it.

Alternatives to Jev

Jev is part of a broader category of models designed to produce structured decisions. OpenJev and Laya are other notable projects to consider when comparing approaches.

OpenJev

OpenJev is an open project aligned with the Jev approach. It is worth exploring if you prefer an implementation outside a hosted service. Before adoption, verify its current availability, hardware requirements, supported features, and license.

Laya

Laya is an open-source decision model from Convai Innovations. It focuses on tasks such as selecting from predefined options, rating information, and answering yes-or-no questions. It may be relevant if you wish to evaluate a model that can run in your own environment.

These projects should be compared based on actual performance, not just stated capabilities. Use identical test data to measure accuracy, latency, resource requirements, and the alignment of confidence estimates with real outcomes.

Can you run decision models locally?

Running models locally offers greater control over the software environment and data processing location. However, deployment options vary by project. Jev is offered via an API, while open-source alternatives may allow inference on your own hardware.

Hardware requirements depend on model size, memory usage, and throughput. Some models run efficiently on a CPU, while others benefit from a GPU, especially when handling high request volumes.

If you wish to experiment with local AI models, DaDesktop provides cloud desktops with dedicated GPU options. This allows development and testing without purchasing a high-end GPU for your local machine.

Learn more about running local LLMs or explore AI agent environments.

When should you use a decision model?

Decision models are beneficial when an application requires a defined answer that triggers a specific next step. They complement LLMs by handling classification, scoring, and routing, while generative models handle language and complex reasoning.

The optimal choice depends on your application. Test the model against your own data, compare it with conventional classifiers and LLMs, and measure accuracy, latency, and cost. For critical decisions, include human review and safeguards where necessary.

Want to discuss your use case? Contact us and we will guide you in the right direction.