Whenever you have more than one ad per adgroup, Adalysis automatically creates an ad test, and once the statistically significant results are achieved, will notify you on the dashboard so you can take action (e.g., pause losing ads or pause and replace losing ads).
If you prefer less manual work, you can set up automated pausing of your loser ads after the test has been completed. By default, it will only pause 1 loser ads per adgroup at once, and only in adgroups where there are at least 2 ads, so that you never end up with an adgroup without active ads after pausing the bad performing one. Read more on that here...
In addition, you can set your own workflow to automatically replace your loser ads with another one that you will have prepared as a draft beforehand. Read more on that here...
By default, Adalysis will alert you about a completed A/B test whenever the confidence reaches the level of 80% (or more) for any of the metrics that are used for testing. You can overwrite this value as well as other settings by clicking on the Ad Testing Settings button (note, that it will also impact your settings for all your future A/B tests in Adalysis).
If you want to run an A/B test with a smaller confidence ratio or use the data from a shorter period of time that Adalysis considers necessary, you can do so in the Manual Tests section.