Introduction

Front end testing of web projects is a useful part of a testing strategy allowing you to test how the full stack works together. Automating this process has significant advantages over manual testing in terms of speed and reliability of testing and when included as part of a CI environment is a real asset for your testing strategy.

While .net based projects will often use Microsoft tools to implement continuous integration of automated front end tests, the open source world also provides a set of (free) tools to do this work.

Getting open source tools to work together is not always as easy as the Microsoft route. This article discusses the tips and tricks we learned when getting Selenium, Jenkins, ASP.NET MVC and SQL Server to integrate and provides a sample solution with web site and tests project to show how these tips might be implemented.

Sample Code

Example web and tests projects, plus scripts used in our Jenkins setup can be downloaded from GitHub here.

 

Technologies

The technologies discussed in this article are as follows:

• ASP.NET MVC – Microsoft web framework.
• Selenium – an open source automated testing framework.
• Jenkins – an open source continuous integration environment.
• SQL server Express – a free implementation of the SQL Server database.
• Code first migrations – the mechanism we chose to deploy our database.
• IIS – Microsoft application server.

 

The Website

The example website project is a simple MVC 5 site that allows you to list, create, edit, view and delete basic details about countries stored in a SQL Server database. The website uses standard .net forms authentication, Entity Framework to handle data connections and code first migrations to deploy the database.

Things we have learned

Separate the connection string and other configuration information.

Placing configuration information such as connection strings in a separate file is useful as it allows different settings on the Continuous Integration (Jenkins) machine. In our case we used the config source attribute on the connections strings element and provided a separate DB.config file – this allows the connection string to be easily changed based on the environment and means that connection strings can be easily excluded from source control.

 <connectionStrings configSource="DB.config" />

 

If taking this approach it is important to remember to specify that the included config file(s) should be copied to the output directory.

Visual Studio -> Solution Explorer -> Right click file -> Properties -> Copy to Output Directory -> Copy Always

 

Use a SQL Express instance rather than localDB.

The LocalDB database does not allow remote connections and so is fine when working in a development environment but does not work well when the website is run from full IIS (which is used for hosting when running integrations tests).

 

The Tests Project

The example tests project (SeleniumDemo.IntegrationTests) is generated from the standard UnitTestProject template. We then add the following nuget packages to enable Selenium testing:

Install-package Selenium.Support

Install-package Selenium.WebDriver

Install-package Selenium.WebDriver.ChromeDriver

We also use the FluentAssertions nuget package to improve the readability of our tests and add EntityFramework to the tests project:

Install-package FluentAssertions

Install-package EntityFramework

Test project structure

The tests project has the following structure:

SeleniumDemo.IntegrationTests

• Countries – contains test classes relating the Countries webpages.
  • PageObjects – contains PageObject classes for reading and manipulating the Countries webpages via the Selenium API.
• KnownData – classes which ensure the database is populated with expected data.
  • Model – classes used when populating the database with expected data.
• Login – classes relating to accessing and completing the login webpage to ensure the user is logged in to the webpage before other pages are requested.
• App.config – configuration settings related to the test project.
• DB.config – separate file used to store the database connection string.
• IntegrationSpecificationBase.cs – base class providing common functionality for test classes.
• Packages.config – Nuget configuration file which records packages associated with the project.
• PageObjectBase.cs – base class providing common functionality for reading and manipulating webpages via the Selenium API.
• UrlBuilder.cs – class that handles generating URLs for the website being tested.

 

 

Things we have learned

Ensure known data before each test is run.

It is critical that the database is in a consistent, known state so that you can assert the existence of expected data on the page. To this end create a set of classes that handle generating known data.

In the example test project the KnownData folder contains classes for generating known account information (AccountDataGenerator) and known Countries information (CountriesDataGenerator). Data generating methods on these classes are then called before each login is attempted (see LoginPageBase.PerformLogin) and before each test is run (see CountriesSpecification.TestInitialize). Note: These classes should also clear any data generated as part of the tests. It makes debugging easier if you make calls to clean up data BEFORE tests are run, rather than after they are run as this allows you to manually check the state of the database after a failed test.

 

Use the PageObjects pattern.

The Page Object pattern separates the operations for reading/interacting with the page being tested from the tests themselves. This makes the tests more readable, less brittle if the UI markup changes and allows you to move from page to page within your tests. This and a number of other general Selenium best practices are available in this informative article.

The example test project shows an implementation of this pattern. The Countries/PageObjects folder contains a class to expose the data and operations required by the tests on that page. These objects inherit from a standard base class that provides a number of methods to simplify these operations. Where possible these operations return the current page object to allow for a clearer, fluent interface. The tests in CountriesSpecification can then make simple, readable calls on the page objects.

 

Use the Login PageObject as an entry point for other pages.

In a system where a login is required, the login page object can be used as a mechanism for accessing other page objects (see example code in CountriesSpecification.cs). This ensures that the user is always logged in before trying to access a page that requires login.

In the example test project the CountriesSpecification class inherits from an IntegrationSpecificationBase class which initializes a LoginPage within the TestInitialize method. The tests can then make calls on the LoginPage to access any page object within the application.

The actual login is performed in the LoginPageBase.PerformLogin method. This is called each time a page object is requested and checks whether the user is logged in, ensures a suitable account exists in the database, then performs the actions on the webpage to log the user in.

 

Use Chrome developer tools to identify selectors.

The page objects rely on css, xpath or other selectors to identify DOM elements within the page under test. An easy way of getting a starting point for selectors is to use Chrome developer tools.

Load the page in Chrome -> F12 -> Find the element within the Elements tab -> Right click -> Copy Selector or XPath

It is best to use the output of these as a starting point as they can be verbose and brittle if you change your markup.

Use CSS selectors are preferable to XPath – here is a useful game that provides training on the subtleties of CSS selections.

 

Use IIS on your development machine to host the website when running tests.

We experimented with using IIS Express or Casini to host the website while running the automated tests, however both of these proved hard to configure appropriately and unreliable. Eventually we turned to installing IIS on our Windows development machines and this proved to be more successful. A couple of points that relate to this:

• When creating the websites within IIS select ‘All Unassigned’ IP addresses but specify a port. This allows you to host multiple websites on your development machine without needing to configure DNS.
• Use Visual Studio publish profiles. As you will need to redeploy the site to your local IIS instance frequently it is convenient to set up Publish Profiles with configuration transforms to make the publishing process as simple as possible. Publish profiles are available by right clicking the website project in Visual Studio Solution Explorer. If there are permissions problems with publishing your site to the local machine then run Visual Studio as an administrator. It is

 

Don’t be afraid to change your markup to simplify testing.

Some selectors may be complex and brittle (i.e. when the markup changes the selector no longer applies). It is OK to add ids or classes to your HTML markup to make Selenium selection easier.

 

Don’t hard code URLs within tests.

URLs can be hard to manage as site structure and environments change. Rather than hard code URLs within tests provide a class that manages URLs and make this available to your test classes.

In the example test project the URLBuilder class provides a mechanism for obtaining URLs. It gets the base URL and port of the project under test from a config file, and is passed around the test project wherever it is required e.g. Login/CountriesLoginPage.cs

 

 

 

The Jenkins Configuration

At a high level the key steps we include in our Jenkins project are as follows:

• Pull the solution from source control.
• Run a powershell or similar script to setup and build the project.
• Run unit tests with VSTest.console.exe

The scripts used in the example project are available in the CIBuildResourceFolder.

The Jenkinks CI will need the following elements installed:

• Jenkins
• Visual Studio – this is required as it provides command line build tools that are required. Plus it can be useful for debugging and identifying build problems.
• IIS – use IIS to host the website to be tested.
• SQL Server Express – the Express version of SQL Server is free and accepts external connections so it suitable for the Jenkins server.

Things we have learned

It’s easier to work on the command line than Jenkins.

By far the most important thing we learned about working with Jenkins is that it is relatively slow and can be hard to work with. This is because Jenkins is essentially a workflow and each step needs to complete before the next step must be run. This makes the debug/fix/test cycle slow and cumbersome. The consequence of this is:

• Write scripts to do the work where you can.
• Make sure all the steps work correctly before you integrate them with Jenkins.
• When everything is ready configure the Jenkins project.

 

When you work with handwritten scripts you can easily call them from the command line, tweak them, comment out sections etc. This massively increases the speed of getting each step to work correctly as you don’t need to wait for the source control pull to complete etc. before you check that some post build action has completed. Because many tweaks may be required to get an entire build process to work this approach can save days of frustration.

 

Use IIS on the Jenkins box to host the site to be tested.

As with the development phase it is easier to use a full IIS instance on the Jenkins box to host the website to be tested than use IIS Express. The website will need to be setup and tested.

 

Enable Nuget package restore on the Solution.

Nuget packages are now an essential part of .net. To ensure that all packages are downloaded when required on the Jenkins CI Server enable Nuget Package Restore from Solution Explorer.

Visual Studio -> Solution Explorer -> Right click Solution -> Enable Package Restore

 

Use Migrate.exe to deploy a code first migration based database.

Strategies for deploying the database will vary based on your data layer, however if using code first migrations you can use the migrate.exe file to perform the update-database action that would usually be called in Visual Studio Package Manager Console. Migrate.exe is included when Entity Framework is added as a Nuget package and can be found in the following folder:

Solution/packages/EntityFrameworkxxx/tools

Migrate.exe needs to be copied into the bin folder of the assembly that contains your migrations. See this article for more details.

 

 

Putting it all together

Once Jenkins, IIS and SQL Server Express are installed on the Jenkins server create a website within IIS for the project to be tested.

Next, create a script that will prepare, build and deploy the website and database each time the Jenkins job is run. The example code includes the folder CIBuildResources which would be included in the Jenkins workspace folder. This folder contains the script setup_and_build.ps1 – this performs the following steps:

• Removes old Test Results files
• Removes all files from the existing IIS website folder website.
• Drop the existing database
• Copy the DB.config files containing appropriate connection strings into the website and tests project ready for building.
• Copy the migrate.exe file to the website bin folder to allow for migrations to be run.
• Build the solution using MSBuild
• Recreate the database schema and seed the database by running the migrate.exe file in the website bin folder.
• Create necessary SQL Server user and update the database permissions by running the databse_permission.sql file.
• Copy the website from the workspace folder to the IIS folder.

 

Once each step is tested and you have confirmed that the website can be server by IIS this script can then be included in the Jenkins project.  Below is a screenshot of the Jenkins configuration for the example project to show how the steps combine.