- Choose the decision boundary
- Protect the surrounding application
- Evaluate quality and operating cost
Choose the decision boundary
A drafting assistant, document search tool and autonomous action system have different risks. Specify what the model may suggest and what it may actually change. For consequential actions, define when a person must inspect and approve the result.
Write examples of acceptable outputs and unacceptable behavior. Include ambiguous requests and missing information. A demonstration that answers a friendly question correctly does not establish reliable behavior across the intended workload.
Protect the surrounding application
Model output should not automatically become trusted code, a database instruction or a command to another service. Keep permissions narrow and validate actions in the application. Treat instructions embedded in retrieved material as data rather than authority.
Identify which information can be sent to the selected provider and how that provider handles it under the chosen configuration. Avoid placing credentials or unnecessary personal records in prompts. The specific privacy and retention arrangement needs review for the project.
Evaluate quality and operating cost
Create a representative evaluation set and record the conditions under which the feature succeeds or fails. Include latency, vendor errors and usage limits. The product needs a useful response when the model is unavailable or uncertain.
Dappr uses AI-assisted development and can discuss AI-enabled applications. Bring the task, data sources and permitted actions. A scoped proposal should separate model integration, application controls, evaluation and ongoing operation rather than treating an API call as the entire product.
Choose a bounded task before choosing a model
An AI feature should have a clear job and a useful response when it cannot complete that job. A fictional equipment distributor might want staff to locate approved product-document passages and prepare a draft answer for review. That is different from allowing the system to make compatibility promises or change an order automatically.
This example is illustrative, not a delivered integration or a claim about model accuracy. The business would identify the documents, users, and decisions involved. Start with the task the feature should help complete, then evaluate whether a model-based approach is appropriate compared with ordinary search, structured filters, or a simpler workflow.
Define what source material can establish
A document can support an answer only within its actual scope and currency. The distributor may have several revisions of a product guide or different instructions for similar models. The application should preserve enough source context for a reviewer to understand which material informed a draft.
A citation is useful evidence to inspect, not proof that the generated interpretation is correct. Test questions that require distinguishing similar products, recognizing missing information, and declining to infer a fact that the documents do not establish. The feature should make uncertainty actionable instead of filling every gap with plausible language.
Keep retrieved instructions separate from authority
OWASP describes indirect prompt injection through external material such as documents and websites. It notes that retrieval techniques do not fully eliminate that risk. The application therefore needs boundaries around what retrieved content can cause, rather than treating every instruction found in a source document as permission to act.
For the fictional distributor, a product file should not be able to authorize sending private customer information elsewhere or changing access settings. The surrounding application must limit available actions and validate requests. The precise controls depend on the architecture and require review; this page does not claim that a prompt alone makes an AI system immune to manipulation.
Separate a suggested action from an executed action
OWASP's excessive-agency guidance highlights risks from unnecessary functionality, permissions, and autonomy. Apply that distinction when deciding what the feature may do. A tool that only needs to retrieve approved material should not automatically receive the ability to modify or delete the underlying records.
For the distributor, staff might review a draft and then use their normal process to send an answer. If later scope includes an action, define the user authority, required confirmation, and record of what happened. A fluent model response is not authorization to make a business commitment on someone else's behalf.
Evaluate with realistic and difficult questions
Build a set of representative tasks using approved or synthetic information. Include ordinary requests, ambiguous model names, conflicting document revisions, missing facts, and attempted instructions embedded in source material. Record what an acceptable answer or refusal looks like before comparing outputs.
For the fictional example, also test an unavailable document service, a model timeout, and a response that cites the wrong product. The interface needs a useful fallback and a way for staff to report errors. A handful of successful demonstration questions cannot establish reliability across the workload or justify an unsupported accuracy percentage.
Plan operation and review after launch
Define who maintains source material, evaluates changes, and reviews reported failures. Model versions, document collections, and application behavior can change independently. Keep enough configuration and evaluation history to understand which conditions produced an observed result, without retaining unnecessary private content.
Dappr can discuss AI-enabled application features around the task, data, and permitted actions. The scope should separate provider integration, application controls, evaluation, and ongoing operation. No particular external connector, perfect answer rate, security certification, or cost saving is promised by this page. A useful feature remains understandable and bounded when the model is wrong or unavailable.
Questions before you begin
Is adding an AI API the whole feature?
No. The product also needs a defined task, appropriate data handling, permission boundaries, evaluation, and useful failure behavior. The model call is one component. Scope the surrounding application and operating responsibilities so the business knows how the feature will be checked and maintained.
Do source citations guarantee an AI answer is correct?
No. A cited passage may be outdated, irrelevant, or interpreted incorrectly. Preserve context and test whether the answer matches the source and the question. The interface should support review and acknowledge missing information rather than treating a citation as automatic proof.
Can an AI feature take actions in other systems?
Only where the action, authorization, and technical connection have been explicitly scoped and verified. Limit permissions to what is needed and define review for consequential actions. This service page does not imply that every external system is supported or that model output itself grants authority.
How should AI quality be evaluated?
Use representative tasks and difficult cases with agreed expectations. Include ambiguity, missing sources, wrong interpretations, and service failures. Record limitations and review changes over time. A successful demo is useful, but it does not establish a universal accuracy level or eliminate the need for human oversight.