Using automated tests to test features only, or testing overall User Experience, Features, and Performance?

Since the pre-mobile application days of automated web app tests using Selenium, I have observed so many automated testers worrying about setting timeout on server response of the web app being tested, so that they can instruct Selenium to wait X number of seconds for response time of the browser to finish loading the content of the page such that features can be without a disruption from any delay in response from the backend.

My feeling always has been: why?

Selenium has the ability for the automated test code to wait until the entire content of the page has been loaded for any amount of time: using ajax or not.

But isn't testing a web/mobile app also wanting to test features + user experience + performance (which includes backend response times and component/page loading )?

If the automated tester wants to test features only, set a very generous timeout from the response to allow any potential performance issues to load to the remote client (web/mobile) to finish before testing and interacting with the page/screen.

Again, I ask: why?

Instead of using allowing a timeout before failing a test, why not just set a timeout before page/screen validation of what you think is an acceptable server response/page/screen load time before validating the page/screen that is acceptable for UX?

This will give the QA team insight of not just the product features that are working, but also working within an acceptable user experience and backend server performance response. Setting your own UX standards and acceptance criteria in the automated test code can provide a valuable insight into more aspects than features. I can tell you that I have found so many performance and UX related bugs with this testing strategy.

Appium WebElement assignment strategies for testing mobile apps

Almost on a daily basis, I am adding to our very solid and robust automated testing framework for my company's android and iOS mobile apps.

We use Appium to interact with the iOS (ViewController) and Android (Activity) mobile application screens.

Appium is the absolute best (and only product) that everyone should be using to write automated tests and interact with their company's mobile applications.

Using the Screen Object design pattern, where the complexity of screen object components are encapsulated, there are 2 basic ways to do this: By using xpath or by lookup by element type and assigning the element to the value (or name attribute value) that is seen by Appium during iteration of web elements (Webdriver API objects) of a particular type.

In this case, let's say that the login screen has a username/password set of components and that they are of type "text" as seen by Appium (using Java programming).

1) Strategy of looking up components by expected xpath locators that Appium will find on the screen:

public class LoginScreen extends BaseScreen {

    private final static String USERNAMEXPATH = "//linear/form/text[1]";
.
.
.
    public void enterUsername(String username) throws Exception {

        try { (getWebElementAtLocation(USERNAMEXPATH)).sendKeys(driver, username); }
        catch(Exception e) { throw e; }

    }

} 

In the base class there would be this method:

protected WebElement getWebElementAtLocation(WebDriver driver, String xpath) throws Exception {

        

        try { return driver.findElement(By.xpath(xpath)); }

        catch(Exception e) { throw e; }

        

    }

or, 2) Strategy of looking up and assigning by an element type and value (or attribute value, such as "name" value)

public class LoginScreen extends BaseScreen {

    private WebElement usernameField = null;
.
.
.
    public void enterUsername(WebDriver driver, String username) throws Exception {

          try {  

             if(this.usernameField == null) this.usernameField = getWebElementFromTextElementsValue(driver, "username"));

             this.username.sendKeys(username); 
        
        }
        catch(Exception e) { throw e; }

    }

} 

In a base class there would be this method:

public class BaseScreen {

.
.
.
    protected WebElement
getWebElementFromTextElementsValue(WebDriver driver, String value) throws Exception {

        try { 
   
              List<WebElement> elements = driver.findElements(By.tagName("text"));

            

              for (WebElement element:elements) {

                

                String data = element.getAttribute("name").trim();

                

                if((data == null) || (data.trim().length() == 0)) continue; // blank, so ignore

                   

                if(data.contains(text))return element;

                    

            }

            

            // if we get here, we could not find the element so throw an exception

            throw new Exception("Could not find text element in any screen element matching type: text and text value: " + value);

        }
        catch(Exception e) { throw e; }

}

The advantages of using strategy #1 are: providing Appium/WebDriver with the exact lookup xpath, which results if faster returning of the screen element for interaction. The disadvantages are: using this method may possibly make the maintenance of this class more "brittle" if the xpath for the element changes with screen updates/redesigns.

The advantages of using strategy #2 are: elements are looked up with attribute values (name or value attributes), and xpath location is not an issue. Assuming the getText() or getAttribute("name") will lookup the correct element regardless of any new components added to the screen. The disadvantage is that lookup might take a little longer since we are instructing Appium to iterate and check the values of the attributes of all components of the type "text" before returning the one we want. It may perhaps slow down the test execution.

Summary: If you have screens to test on your mobile app that don't change very much, use the xpath lookup strategy (#1). If the screen has alot of changing formats/design, use the web element lookup strategy (#2) to avoid less maintenance in needing the Inspector to find the updated xpath locator and thus update of the code.

Clari has a hot mobile app and is a great company to work for

www.clari.com

The hottest startup in the SF Bay Area (and perhaps the entire world) is at Clari.

There are some great job opportunities there for the top professionals in software. It's the number#1 rated mobile technology startup company.

Job listings are here: www.clari.com/#jobs