Tests

A test checks one behaviour of your app, for example “a user can reset their password”. You write it in plain language as a list of steps. Each step has an action and an expected result. Testifly’s AI agent then automates the test: it opens your app, follows the steps and writes the test code. After that, the test can run as often as you like.

Most tests are written for you during feature discovery. You can also write your own, or ask Tessa to draft one.

How do I create a test?

1. Open a feature and click ADD TEST. You can do this from the feature page or from the quick view on the Features page.
2. Fill in the form:
   • Test title – at least 5 characters.
   • Test Suites – search for an existing suite. To make a new one, type its name and choose Create ’…’.
   • Description – the purpose of the test, at least 5 characters.
   • Test Steps – click ADD STEP for each step. Write what to do in the first field and the expected result in the second. Each field needs at least 5 characters. Drag a step to change its order, or click the bin icon to remove it.
3. Choose how to save:
   • CREATE AS DRAFT – saves the test without code. Nothing runs yet.
   • CREATE AND AUTOMATE – saves the test and sends it to the agent for automation.

Tips for writing good steps

• Write one action per step, for example “Click the Save button”.
• Make the expected result something the agent can see on the page, for example “A green message Saved appears”.
• Mention every step a person would take, such as closing a pop-up or filling a required field. Missing steps are the most common reason for Automation Failed.
• Avoid exact text or spacing unless it matters. Styling can change how text looks.

What is on the test page?

Open a test from a feature or from the features list. The page shows:

• Status – above the title. See test statuses.
• Title and Description – click to edit.
• Alerts – for example “Your last run failed. Click to review.” or a link to a run that is awaiting your input.
• Errors – the explanation from the agent when automation failed or found a bug.
• Prerequisites – the tests that must pass first. Click ADD to add one, or the cross on a chip to remove it.
• Steps & expected results – click a step to edit it. Click ADD STEP to add one.
• Code – the test code the agent wrote. It is read-only.
• UI Components – the components the test uses, with screenshots.
• Share your experience? – give a thumbs up or down and a comment. This helps Testifly improve.
• Details – the Feature URL, the Test suites (click ADD to add one) and the Is Prerequisite switch.
• Runs – RUN TEST starts the test now. Below it is the history of earlier runs. Click a run to open its recording.
• DELETE – removes the test after you confirm.

What happens when I edit a test?

When you change a test, a bar appears at the bottom of the page with two buttons:

• Save as draft – saves your changes.
• Automate test – saves your changes and sends the test to the agent.

If you change the description or the steps, the old code no longer matches. Testifly sets the test to Draft and it will not run until you automate it again. Changes to the title, test suites, prerequisites or the Is Prerequisite switch do not reset the status.

When a test is in Automation Failed, Product Bug Detected or Not Applicable, the page explains the problem and the Automate test button is enabled. Fix the issue or improve the steps, then click it to try again.

What does the agent do when it automates a test?

Automation runs as an Automating Tests job. You can follow it on the Jobs page. The test shows Automating until it is done.

1. Wait in line. Tests are processed several at a time. Prerequisites are always automated before the tests that depend on them.
2. Open your app. The agent starts a real browser, signs in if needed, runs the prerequisites and opens the feature URL.
3. Work step by step. For each step, the agent looks at the page, finds the UI components the step mentions, writes the code for the step and runs it right away to check that it works.
4. Handle problems. If a step does not work, the agent studies the page and decides who is at fault:
   • The test description is unclear or misses a step. The agent may adjust the steps and try again.
   • The agent itself made a mistake, for example it used the wrong element. It corrects the code and tries again.
   • Your app behaves wrongly. The test is marked Product Bug Detected, with a short explanation of the bug.
   • The test cannot apply, for example because the data it needs does not exist. The test is marked Not Applicable, with a reason.
5. Save the result. When all steps work, the code is saved and the test becomes Ready To Run. Otherwise it gets one of the statuses below.
6. Run and report. Tests that were automated successfully are run straight away. When the whole job is done, you get an “automation finished” email.

Why did automation fail?

If a CAPTCHA or bot check blocks the browser, automation stops and a banner appears on the dashboard. See Disabling CAPTCHA.

How do prerequisites work?

A prerequisite is a test that prepares the ground for other tests, for example “Page Loaded Verification” or “Open the settings panel”.

• To make a test a prerequisite, turn on Is Prerequisite in its details. On the features list its title starts with “Prerequisite:”.
• To use it, open another test in the same feature and add it under Prerequisites.
• Prerequisites can have their own prerequisites. Testifly works out the right order.
• During automation and runs, prerequisites always go first. If a prerequisite fails, the tests that depend on it cannot continue. During automation they become Faulty Prerequisite; during a run they are Skipped.

What are the login and cleanup tests?

Every project has two special, hidden tests. You find them in Settings → Misc.:

• Login test (View login test) – created during setup from your login details. Every test that needs a login uses it. When you change the login page, the sign-in method or the test user, Testifly rebuilds and re-automates this test by itself.
• Cleanup test (View cleanup test) – empty at first. Add steps that reset your app, for example log out or delete test data. Automate it, then turn on Run cleanup after each run in Settings → Misc.

How do I restart automation for many tests at once?

Go to Settings → Actions and click Automate Tests. Testifly restarts automation for every test that shows Automating but is not being worked on, for example after an interruption. Tests in Draft are not included. Open each Draft test and click Automate test.

Is there a limit on automated tests?

During the free trial, a project can automate up to 30 tests in total. Login and cleanup tests do not count. When the limit is reached, remaining tests stay in Draft. After you choose a plan, you can automate them.