Details
AI can accelerate automated test creation, but generated tests still need to remain understandable, editable, and resilient as applications change. This demonstration shows how SmartBear Reflect connects with Claude through the SmartBear MCP to create a registration-flow test from a natural-language objective. It also demonstrates how testers can combine agent-driven creation with manual record-and-play and use AI-generated descriptions as a fallback when selectors become invalid.
After connecting Claude to an active Reflect session through the SmartBear MCP, the tester asks the agent to test an application’s registration workflow without submitting the new user.
The agent interprets that objective, observes the application, and performs the actions needed to build the automated test.
The SmartBear MCP gives the external agent information about Reflect and the active recording session.
Once connected through the session ID, Claude can recognize the recording, inspect the current page, and interact with the application through Reflect.
As the agent navigates the application, Reflect converts its activity into natural-language actions and assertions called prompt steps.
Examples include clicking the login button, selecting registration, and entering sample values. This makes the resulting test easier for people to review and understand.
A tester can type an objective directly, but the same workflow can also draw on information available to the coding agent.
The demonstration notes that objectives could potentially be based on requirements from a Jira ticket, application source code, or another connected source.
Using an AI agent does not lock the tester into a completely autonomous workflow. The tester can stop the agent and continue building the test by interacting with the application directly.
This hybrid approach allows teams to use AI where it is helpful while retaining hands-on control over specific actions and test details.
Reflect initially uses selectors to locate application elements in recorded tests. If a page change makes a selector invalid, Reflect can fall back to the AI-generated natural-language description associated with the step.
In the demonstration, the presenter deliberately replaces a working selector with an invalid one. Reflect still locates and clicks the intended search field using the step description.
Natural language can lower the effort required to create tests
Practical implication: Define the intended user workflow and expected stopping point clearly so the agent understands both what it should do and what it should avoid.
Generated tests still need human-readable structure
Practical implication: Preserve agent actions as understandable steps that testers can inspect, refine, and maintain.
Requirements can become test objectives
Practical implication: Connect approved sources such as Jira tickets or source repositories when teams want test creation to begin from existing requirements or application context.
AI adoption does not require abandoning familiar testing methods
Practical implication: Allow testers to move between agent-assisted creation and record-and-play within the same workflow.
Selectors remain useful but should not be the only recovery mechanism
Practical implication: Pair selectors with meaningful natural-language descriptions that can help the testing system recover when the interface changes.
Self-healing should remain visible
Practical implication: Record when a fallback mechanism is used so teams can determine whether a selector should be updated permanently.
Q: How does Claude connect to SmartBear Reflect?
A: The demonstration uses the SmartBear MCP server. After it is installed and connected to Claude, the tester provides the active Reflect session ID so Claude can interact with that recording session.
Q: What is a Reflect prompt step?
A: A prompt step is a natural-language action or assertion stored as part of a Reflect test. Examples include clicking a button, opening a registration page, or entering a sample value.
Q: How does the agent know what action to perform next?
A: The test-creation agent operates as a loop. It examines the current state and screenshot of the page, performs an action, and continues until it reaches the objective provided by the tester.
Q: Can Reflect create tests from Jira requirements?
A: The presenter explains that, if a Jira MCP integration is available to the coding agent, the tester could ask it to create a test using requirements from a ticket. This possibility is described but not demonstrated in the recording.
Q: Can testers edit a test created by an AI agent?
A: Yes. A tester can stop the agent and continue interacting with the browser directly through Reflect’s record-and-play functionality.
Q: Can agentic test creation and record-and-play be combined?
A: Yes. The demonstration presents a hybrid workflow in which an agent creates part of the test and the tester manually records additional steps.
Q: How does Reflect’s self-healing work?
A: Reflect first attempts to locate an element using its recorded selectors. If those selectors no longer work, it can fall back to the AI-generated natural-language description of the action.
Q: What happens when a selector becomes invalid?
A: In the demonstration, the presenter intentionally enters an invalid selector. When the test is rerun, Reflect uses the natural-language step description to locate and click the intended search field.
Q: Does record-and-play still use AI?
A: Yes. Reflect generates natural-language descriptions for recorded actions. Those descriptions make the steps readable and can support self-healing when selectors fail.
Todd McNeal: I have a Claude session set up on the left-hand side and the Reflect application visible on the right.
We’re going to begin by creating a recording. We’ll call it “Smoke Test,” and we’ll create the test against the SmartBear Store, which is an example ecommerce application.
I’ll copy the application URL and add it as the starting URL.
You have flexibility in how you use an agent to help create a test. In this first workflow, we’re going to use Claude as an external agent.
To begin the process, I have already installed the SmartBear MCP server in my Claude session. Installation takes only a few minutes. You install the MCP package through npm and connect it to your Claude session.
I’m going to tell Claude, “I want to create a Reflect test. My session ID is…” and then copy this string.
The session ID tells Claude how to connect to my active Reflect session.
Because the SmartBear MCP server is already installed, Claude understands what Reflect is and how to connect to it.
We can see that Claude has connected successfully. It mentions the name we gave the recording—the Smoke Test session. It can also see what is on the page and notes that reusable segments are available if we want to use them.
Our test-creation agent operates as a loop. It examines the current state of the page and a screenshot, then performs an action. It continues this process until it achieves the objective it has been given.
We can provide that objective through Claude.
For example, I can say, “I want to test the registration workflow of this application. Go through the registration flow, but do not submit the new user.”
That is one example of an objective.
If you have the Jira MCP installed, you could also ask the agent to create a test using requirements from a Jira ticket. The objective could come from anything connected to the Claude session, including the application’s source code if the agent has access to it.
On the left side, we can now see Claude reasoning and performing actions. On the right side, we can see the browser session and the actions being added to the test.
The actions being added are what we call prompt steps.
A prompt step is simply a natural-language action or assertion. We can see steps such as clicking the login button, clicking Register, and entering sample values.
In many cases, the tester knows exactly what they want to test. They may already have a manual test case and simply want to perform those actions to create the automated test.
You can still do that when using the agent.
For example, I can stop the agent and begin interacting with the browser session directly. We can then enter information manually.
This uses Reflect’s record-and-play functionality.
Using an agent does not prevent you from using conventional record-and-play. You can combine both methods in a hybrid approach.
Record-and-play also benefits from Reflect’s AI-powered self-healing.
To demonstrate that, I’ll click this search field.
Reflect generates an English-language description of the action I just performed. That description was generated using AI.
If we open the step, we can also see selectors associated with it.
For recorded tests, Reflect first uses selectors to target an element. Selectors can be stable, but they can also become invalid. For example, a developer might change the structure of the page, causing the original selector to stop working.
If that happens, Reflect can fall back to the natural-language description it generated for the step. That provides the self-healing mechanism.
To demonstrate, I’ll edit the selectors and replace the working selector with an invalid one.
When I rerun the test, Reflect will fall back to the English-language action.
I’ll click out of the editor so you can see the action being performed.
The test successfully clicked the search field. We can see that it worked because the search field expanded.
Even when you use record-and-play, you receive the benefits of AI because Reflect generates these English-language descriptions for the recorded actions.
Presenters:
Todd McNeal
Director of Product Management, SmartBear
Join Now for More Content & Events
For event and sponsorship inquiries, please email: [email protected]
