Using Selenified With Your Own Framework

So, earlier this year, when the Selenified press release was sent out, I got some great feedback from Alan Richardson. I was thrilled this had somehow popped up on his radar, and even more excited that he took the time to not only take a look at the framework, but also to share his thoughts. Several of things he commented on were things that were on my radar, but one thing in particular stood out. He noted that Selenified was built on top of TestNG, and as he preferred JUnit, this made it a ‘non-starter’ for him. Now, while I may disagree with his choice of testing frameworks, to each his own. And yet, this point was an important one.

When designing and developing Selenified, it was a conscious decision to build a framework, something more than just a Selenium extension. This came out of working with our clients. Many of them struggle to even setup their tests properly. The purpose of Selenified is to make writing Selenium tests simple from the start. You can immediately dive in and concentrate on writing test cases, not worry about how to run tests in parallel, setup browser drivers, etc.

That said, if someone is a good developer (which unfortunately I believe is undervalued as a tester attribute), they probably have done many of the same things I have done, and may not want to abandon their framework for this one. My initial thought to this idea then, was that this framework might not be for them. By making this a standalone tool, some of the benefits of using Selenified as an entire framework would get lost. Some of these benefits are directly tied to TestNG, such as running tests on multiple browsers in one run, or links to detailed test reports. Others are more custom to the framework, such as simply being able to run tests through a proxy or seamlessly connect out to a hub to execute the tests.

Time To Rethink

Well, wouldn’t you know it, not even 6 months after Alan had made that suggestion, I ran into the same issue. Working with my client, I got back into Behavior Driven Development, and starting writing tests with Cucumber. Cucumber is a framework, not a tool, and while Cucumber-JVM supports both TestNG and JUnit, those underlying frameworks are incompatible with the Selenified class. While all of the code existed to turn Selenified into a standalone tool, I had put off figuring out the best way to get this to work/run.

It’s wonderful to get to see your own words and actions come back to bite you in such a karmic way.

The New Implementation

It’s not here, in full at least…

I’ve spent the last year working diligently to get Selenified released as open source software, and much of that involved some intense refactoring to make things simple. I’m hesitant to split off a separate standalone version of Selenified at this point, as it will mean I will have two code bases to deal with. Selenified 3.0 was just recently released as a major refactor, and the more the framework is used, the more comments and suggestions I get. For this reason, I’m trying to keep just one project out there. My hope is in the next few months, things will settle down, and I’ll be able to put some more in depth thought into the correct approach.

For now, where does that leave us? Well, as I mentioned above, all of the code required to let Selenified run standalone exists, it just needs to be exported and modified slightly. So, that’s what I did, and now I have Selenified supporting my Cucumber test cases.

How To Do It Yourself

If this is something you are interested in doing, simply follow the below steps. These steps are specific to my implementation for Cucumber, but they should be simple enough to adapt use for any framework.

What Didn’t Change

I copied the com.coveros.selenified.application, com.coveros.selenified.element, com.coveros.selenified.services, and com.coveros.selenified.exceptions into my Cucumber project directory. I renamed the packages a bit, but the content of those files remained untouched.

This left only two packages left to look at migrating, or replacing: com.coveros.selenified and com.coveros.selenified.utilities. Not too shabby.

Utilities

For this particular implementation, I didn’t care about running things with HTMLUnit. So, I simply abandoned the CustomHtmlUnitDriver class. The Transformer class which is used to loop through the test cases on different browsers was abandoned, as Cucumber didn’t support it. The Point class I decided to relocate into com.coveros.selenified.element as it just seemed to fit, and be simpler. This left TestSetup and Listener. I will address TestSetup in the next section. Listener I unfortunately ended up rewriting. In order to get the detailed reports written into the TestNG output, we needed this custom listener, but I also wanted to include the custom Cucumber reports.

I overrode the default test listeners, and then wrote some custom methods to determine the current scenario and feature file, so make the output readable and friendly. The end result looks like the below

public class Listener extends TestListenerAdapter {

  private static final Logger log = Logger.getLogger(Listener.class);

  private static final String RUNNER_LOCATION = "target/generated-test-sources/cucumber";
  private static final String CUCUMBER_REPORT_LOCATION = "target/cucumber-parallel";
  private static final String CUSTOM_REPORT_LOCATION = "target/detailed-reports";
  private static final String OUTPUT_BREAK = " | ";
  private static final String LINK_START = "<a target='_blank' href='";
  private static final String LINK_MIDDLE = "'>";
  private static final String LINK_END = "</a>";
  private static final String BREAK = "<br />";
  private static final String TIME_UNIT = " seconds";

  /**
   * Runs the default TestNG onTestFailure, and adds additional information
   * into the testng reporter
  **/
  @Override
  public void onTestFailure(ITestResult test) {
    super.onTestFailure(test);
    output(test);
  }

  /**
   * Runs the default TestNG onTestSkipped, and adds additional information
   * into the testng reporter
  **/
  @Override
  public void onTestSkipped(ITestResult test) {
    super.onTestSkipped(test);
    output(test);
  }

  /**
   * Runs the default TestNG onTestSuccess, and adds additional information
   * into the testng reporter
  **/
  @Override
  public void onTestSuccess(ITestResult test) {
    super.onTestSuccess(test);
    output(test);
  }

  /**
   * Returns the scenarioID from the particular test that is running.
   * Used in conjunction with cucumber-jvm-parallel-plugin for parallel
   * execution of tests
  **/ 
  private int getScenarioID(ITestResult test) {
    String testRunner = test.getInstanceName();
    File runner = new File(RUNNER_LOCATION, testRunner + ".java");
    if (!runner.exists()) {
      log.error("Unable to locate scenario runner at " + runner.getAbsolutePath());
      return 0;
    }
    return Integer.parseInt(testRunner.replaceAll("[\\D]", ""));
  }

  /**
   * Returns the feature file name information from the particular test that is running
   * Used in conjunction with cucumber-jvm-parallel-plugin for parallel
   * execution of tests
  **/ 
  private String getFeatureInfo(ITestResult test) {
    String testRunner = test.getInstanceName();
    File runner = new File(RUNNER_LOCATION, testRunner + ".java");
    try (BufferedReader br = new BufferedReader(new FileReader(runner))) {
      while (br.ready()) {
        String line = br.readLine();
        if (line.contains("features = ")) {
          String[] parts = line.split("\"");
          return parts[1];
        }
      }
    } catch (FileNotFoundException e) {
      log.error("Unable to locate scenario runner at " + runner.getAbsolutePath());
      return null;
    } catch (IOException e) {
      log.error("Unable to read scenario runner at " + runner.getAbsolutePath());
      return null;
    }
    return null;
  }

  /**
   * Returns the feature name from the particular test that is running
   * Used in conjunction with cucumber-jvm-parallel-plugin for parallel
   * execution of tests
  **/ 
  private String getFeature(ITestResult test) {
    String featureInfo = getFeatureInfo(test);
    if (featureInfo == null) {
      return featureInfo;
    }
    File featureFile = new File(featureInfo.split(":")[0]);
    try (BufferedReader br = new BufferedReader(new FileReader(featureFile))) {
      while (br.ready()) {
        String line = br.readLine();
        if (line.contains("Feature:")) {
          String[] parts = line.split(": ");
          return parts[1];
        }
      }
    } catch (FileNotFoundException e) {
      log.error("Unable to locate feature file at " + featureFile.getAbsolutePath());
      return null;
    } catch (IOException e) {
      log.error("Unable to read feature file at " + featureFile.getAbsolutePath());
      return null;
    }
    return null;
  }

  /**
   * Returns the scenario name from the particular test that is running
   * Used in conjunction with cucumber-jvm-parallel-plugin for parallel
   * execution of tests
  **/ 
  private String getScenario(ITestResult test) {
    String featureInfo = getFeatureInfo(test);
    if (featureInfo == null) {
      return featureInfo;
    }
    File featureFile = new File(featureInfo.split(":")[0]);
    int scenarioLine = Integer.parseInt(featureInfo.split(":")[1]);
    try (BufferedReader br = new BufferedReader(new FileReader(featureFile))) {
      int count = 1;
      while (br.ready()) {
        String line = br.readLine();
        if (count == scenarioLine) {
          String[] parts = line.split(": ");
          return parts[1];
        }
        count++;
      }
    } catch (FileNotFoundException e) {
      log.error("Unable to locate feature file at " + featureFile.getAbsolutePath());
      return null;
    } catch (IOException e) {
      log.error("Unable to read feature file at " + featureFile.getAbsolutePath());
      return null;
    }
    return null;
  }

  /**
   * Writes out the status of the test into the TestNG reporting file
  **/ 
  private void output(ITestResult test) {
    File cucumberOutput = new File(
    CUCUMBER_REPORT_LOCATION + File.separator + getScenarioID(test) + File.separator + "index.html");
    File customOutput = new File(CUSTOM_REPORT_LOCATION + File.separator + getFeature(test).toLowerCase() + " - " + getScenario(test) + ".html");
    Reporter.log(getFeature(test) + " - " + getScenario(test));
    Reporter.log(BREAK + LINK_START + cucumberOutput.getAbsolutePath() + LINK_MIDDLE + "Brief Report" + LINK_END + OUTPUT_BREAK + LINK_START + customOutput.getAbsolutePath() + LINK_MIDDLE + "Detailed Report" + LINK_END);
    Reporter.log(BREAK + Result.values()[test.getStatus()] + OUTPUT_BREAK + (test.getEndMillis() - test.getStartMillis()) / 1000 + TIME_UNIT);
  }
}

Finally, in order to get this new listener picked up, I added the below line into my Test Runner

@Listeners({ some.package.Listener.class })

Selenified

I retained the Browser and Locator enumerations, and put those near the App and Element classes. Similarly, I kept OutputFile the same, and put it in it’s own package. The major re-write was porting the work from Selenified and TestSetup over so that the tests could be called simply.

Controller

I created a Controller class which controlled and managed the browser. This was very similar to my previous TestSetup class. This Controller class lived in the same package as my step definitions. It’s what took in the input information, and setup any browser information required, and could launch the browser itself. The code for it is below:

public class Controller {

  private Browser browser = Browser.FIREFOX; // default
  private WebDriver driver;
  private App app;
  private OutputFile file;

  public static final Logger log = Logger.getLogger(Controller.class);
  public static final String CUSTOM_REPORT_LOCATION = "target/detailed-reports";

  public static final String DEVICE_ORIENTATION = "deviceOrientation";

  public void setupController(Scenario scenario) {
    switch (browser) {
    case FIREFOX:
      FirefoxDriverManager.getInstance().forceCache().setup();
      driver = new FirefoxDriver(capabilities);
      break;
    case MARIONETTE:
      FirefoxDriverManager.getInstance().forceCache().setup();
      driver = new MarionetteDriver(capabilities);
      break;
    case CHROME:
      ChromeDriverManager.getInstance().forceCache().setup();
      driver = new ChromeDriver(capabilities);
      break;
    case INTERNETEXPLORER:
      InternetExplorerDriverManager.getInstance().forceCache().setup();
      driver = new InternetExplorerDriver(capabilities);
      break;
    case EDGE:
      EdgeDriverManager.getInstance().forceCache().setup();
      driver = new EdgeDriver(capabilities);
      break;
    case SAFARI:
      driver = new SafariDriver(capabilities);
      break;
    case OPERA:
      OperaDriverManager.getInstance().forceCache().setup();
      driver = new OperaDriver(capabilities);
      break;
    // if the browser is not listed, throw an error
    default:
      throw new InvalidBrowserException("The selected browser " + browser + " is not an applicable choice");
    }
    // start and setup our logging
    setupLogging(scenario);
  };

  public void setupRemoteController(Scenario scenario) throws MalformedURLException {
    DesiredCapabilities capabilities = getDeviceCapabilities();
    capabilities.setCapability("name", scenario.getId().replaceAll("-", " ").replaceAll(";", ": "));
    driver = new RemoteWebDriver(new URL(System.getProperty("hub") + "/wd/hub"), capabilities);
    // start and setup our logging
    setupLogging(scenario);
  }

  public DesiredCapabilities getDeviceCapabilities() {
    DesiredCapabilities capabilities = new DesiredCapabilities();
    switch (browser) { // check the browser
    case HTMLUNIT:
      capabilities = DesiredCapabilities.htmlUnitWithJs();
      break;
    case FIREFOX:
      capabilities = DesiredCapabilities.firefox();
      break;
    case MARIONETTE:
      capabilities = DesiredCapabilities.firefox();
      capabilities.setCapability("marionette", true);
      break;
    case CHROME:
      capabilities = DesiredCapabilities.chrome();
      break;
    case INTERNETEXPLORER:
      capabilities = DesiredCapabilities.internetExplorer();
      break;
    case EDGE:
      capabilities = DesiredCapabilities.edge();
      break;
    case ANDROID:
      capabilities = DesiredCapabilities.android();
      break;
    case IPHONE:
      capabilities = DesiredCapabilities.iphone();
      break;
    case IPAD:
      capabilities = DesiredCapabilities.ipad();
      break;
    case SAFARI:
      capabilities = DesiredCapabilities.safari();
      break;
    case OPERA:
      capabilities = DesiredCapabilities.operaBlink();
      break;
    case PHANTOMJS:
      capabilities = DesiredCapabilities.phantomjs();
      break;
    // if the browser is not listed, throw an error
    default:
      throw new InvalidBrowserException("The selected browser " + browser);
    }
    capabilities = getDeviceDetails(capabilities);
    return capabilities;
  }

  public DesiredCapabilities getDeviceDetails(DesiredCapabilities capabilities) {
    if (System.getProperty("deviceDetails") != null) {
      if (System.getProperty("deviceDetails").contains("=")) {
        Map<String, String> deviceDetails = Device.parseMap(System.getProperty("deviceDetails"));
        // needed for web browser testing using selenium
        if (deviceDetails.containsKey(CapabilityType.VERSION)) {
          capabilities.setCapability(CapabilityType.VERSION, deviceDetails.get(CapabilityType.VERSION));
        }
        if (deviceDetails.containsKey(CapabilityType.PLATFORM)) {
          capabilities.setCapability(CapabilityType.PLATFORM, deviceDetails.get(CapabilityType.PLATFORM));
        }
        // needed for web browser testing using appium
        if (deviceDetails.containsKey(CapabilityType.BROWSER_NAME)) {
          capabilities.setCapability(CapabilityType.BROWSER_NAME,
              deviceDetails.get(CapabilityType.BROWSER_NAME));
        }
        if (deviceDetails.containsKey(MobileCapabilityType.DEVICE_NAME)) {
          // by default, iPhone Simulator and Android GoogleAPI
          // Emulator are used, but these can be overridden
          capabilities.setCapability(MobileCapabilityType.DEVICE_NAME,
              deviceDetails.get(MobileCapabilityType.DEVICE_NAME));
        }
        if (deviceDetails.containsKey(DEVICE_ORIENTATION)) {
          // by default, portrait mode is tested on
          capabilities.setCapability(DEVICE_ORIENTATION, deviceDetails.get(DEVICE_ORIENTATION));
        }
        if (deviceDetails.containsKey(MobileCapabilityType.PLATFORM_VERSION)) {
          // by default, 10.3 and 7.1 are used, but these can be
          // overridden
          capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION,
              deviceDetails.get(MobileCapabilityType.PLATFORM_VERSION));
        }
      }
    }
    return capabilities;
  }

  public void setupLogging(Scenario scenario) {
    String feature = scenario.getId().split(";")[0].replaceAll("-", " ");
    file = new OutputFile(CUSTOM_REPORT_LOCATION, feature, scenario.getName(), scenario.getSourceTagNames(),
        browser);
    app = new App(driver, browser, file);
    file.setApp(app);
  }

  public void takeScreenshotAndTeardownController(Scenario scenario) {
    try {
      byte[] screenshot = ((TakesScreenshot) getDriver()).getScreenshotAs(OutputType.BYTES);
      scenario.embed(screenshot, "image/png");
    } catch (WebDriverException wde) {
      System.err.println(wde.getMessage());
    } catch (ClassCastException cce) {
      cce.printStackTrace();
    } finally {
      getOutputFile().finalizeOutputFile();
      getDriver().quit();
      Assert.assertEquals(getOutputFile().getErrors() + " Errors", 0 + " Errors");
    }
  }
}

Setup and Teardown

The last thing required, was to setup and teardown the application using Cucumber’s @Before and @After annotations. I wrote a separate Setup class for handling this, and also placed that in the Step Definition package. This mainly took the place of the Selenified class, and took care of managing the browser for my tests. For Cucumber, it also provided me the extra benefit of adding a final screenshot after each test was run.

public class Setup {

  private static final Logger log = Logger.getLogger(Setup.class);

  Controller controller;

  public Setup(Controller controller) {
    this.controller = controller;
  }

  @Before
  public void setup(Scenario scenario) throws MalformedURLException {
    if (System.getProperty("hub") != null) {
      controller.setupRemoteController(scenario);
    } else {
      controller.setupController(scenario);
    }
    controller.getApp().setSite(getEnvironmentURL(scenario));
  }

  @After
  public void teardown(Scenario scenario) {
    controller.takeScreenshotAndTeardownController(scenario);
  }
}

In case it wasn’t clear in the above code, I’m using Dependency Injectors to pass the controller into the Setup class. To handle this, I simply ensured that PicoContainer was included in my maven build as a dependency.

Writing Step Definitions

Then, in order to implement my Steps, it was as simple as referring to my controller. No having to launch a browser, or tear it down upon completion, and with PicoContainer managing my controller, every step has access to the same browser, even if they are in separate classes.

public class LoginSteps {

  private static final Logger log = Logger.getLogger(LoginSteps.class);

  Controller controller;
  LoginWorkflow loginPage;

  public LoginSteps(Controller controller) {
    this.controller = controller;
    loginPage = WorkflowFactory.getLoginPage(this.controller);
  }

  @When("^I navigate to the login page$")
  public void navigateToLogin() {
    this.loginPage.loadEnvironment();
  }
}

And we’re good to go

Conclusion

Not all of this code is on github, and I apologize for that. With Selenified changing at it’s current rate, it seems like more maintenance than I want to deal with, so please pull what you can from the code snippets in this post.

What are your thoughts? Are you jumping at the bit to have Selenified freed from TestNG? The more people asking for it, the more likely I am to get that working. Please let me know in the comments, or open a ticket over on github.

Until then, happy testing!