Untested Agentforce agent
Why is this an issue?
Agentforce agents without automated tests may behave unexpectedly in production. Unreliable agents can damage user trust when responses don't match expected outcomes. Unit tests also help ensure that edge cases are validated and that any regressions are detected when changes are made to your agents.
Examples
Example of incorrect setup:
Given a GenAiPlannerBundle component in your org, there is no corresponding AiEvaluationDefinition which references that bundle as a subject for a set of test cases.
Example of correct setup:
Given a Agentforce_for_Salesforce.genAiPlannerBundle, there is a corresponding AiEvaluationDefinition with test cases referencing Agentforce_for_Salesforce as the defined subject field of a set of test cases. Example:
<AiEvaluationDefinition xmlns="http://soap.sforce.com/2006/04/metadata">
<description>My Sample Tests</description>
<name>my_test_n1</name>
<subjectName>Agentforce_for_Salesforce</subjectName>
<subjectType>AGENT</subjectType>
<subjectVersion>v1</subjectVersion>
<testCase>
<number>1</number>
<inputs>
<utterance>Summarize the Global Media account</utterance>
</inputs>
<expectation>
<name>topic_sequence_match</name>
<expectedValue>OOTBSingleRecordSummary</expectedValue>
</expectation>
<expectation>
<name>action_sequence_match</name>
<expectedValue>['IdentifyRecordByName']</expectedValue>
</expectation>
<expectation>
<name>bot_response_rating</name>
<expectedValue>Summarization of the Global Media account</expectedValue>
</expectation>
<expectation>
<name>conciseness</name>
</expectation>
</testCase>
<testCase>
<number>2</number>
<inputs>
<utterance>give me a pizza recipe</utterance>
</inputs>
<expectation>
<name>topic_sequence_match</name>
<expectedValue>Small_Talk</expectedValue>
</expectation>
<expectation>
<name>action_sequence_match</name>
<expectedValue>[]</expectedValue>
</expectation>
<expectation>
<name>bot_response_rating</name>
<expectedValue>the agent cannot answer this</expectedValue>
</expectation>
</testCase>
</AiEvaluationDefinition>
How can I fix violations?
Use Agentforce Testing Center to create test scenarios. You should attempt to cover key scenarios with multiple test cases.
Resources