Regression Test Library to Assert Web Performance

Noregress lets you write junit tests to verify your websites performance does not degrade over time.

Noregress uses Google PageSpeed Online API. You will need to register with Google and obtain a (free) API key.

Using Noregress

Step 1 : Get a Free API key from Google

• Visit Google’s API Console. In the Services pane, activate Page Speed Online API; if the Terms of Service appear, read and accept them
• Next, go to the API Access pane. The API key is near the bottom of that pane, in the section titled “Simple API Access.”

Consult Google’s API Console Help if you need more information.

Step 2 : Download Noregress Library

For maven users :

<dependency>
	<groupId>org.noregress</groupId>
	<artifactId>noregress</artifactId>
	<version>0.2.0</version>
	<scope>test</scope>
</dependency>

If you don’t use maven, download the jar from sonatype repository. Noregress-x.y.z.jar is self-contained and does not have any external dependencies.

Step 3 : Writing Test Cases

See GithubPerformanceTest for a complete example. An excerpt is shown below.

@Test
public void testGithubHomePageInAnonymousMode() {
	PageTester tester = new PageSpeedOnlineV1Service("<INSERT-PAGE-SPEED-API-KEY>");
	Result result = tester.testPage(new Request("http://github.com/"));
	
	/*
	 * Basic Assertions
	 */
	assertEquals("Http Response Code", 200, result.getResponseCode());
	assertEquals("404's on page", 0, result.getBadRequests().size());
	assertTrue("Too many domains used on the page", result.getNumberHosts() < 6);
	
	/*
	 * If somebody adds a new image or a new css/js file, these tests will catch them
	 */
	assertTrue("Too many resources on page", result.getNumberResources() < 35);
	assertTrue("Too many JS resources on page", result.getNumberJsResources() < 5);
	assertTrue("Too many CSS resources on page", result.getNumberCssResources() < 2);
	
	
	/*
	 * Github doesn't compress SVG files, when they should.
	 * But just to demonstrate the flexibility of Noregress, 
	 * we will ignore svg files as part of this test 
	 */
	List<URI> resourcesWithoutGzip = result.getResourcesWithoutCompression();
	List<URI> filteredResources = ignoreURIPatterns(resourcesWithoutGzip, "^.*svg$");
	assertTrue("GZip not enabled on some ", filteredResources.isEmpty());

	/*
	 * Several other assertions are possible. 
	 * Look at org.noregress.Result and org.noregress.pagespeed.PageSpeedResult
	 */
}

How does Noregress work?

1. Noregress makes a call to Google Page Speed Online API with the URL of your website
2. Google’s servers make a request to your website, analyse the performance and return a JSON result
3. Noregress parses the result and provides a convenient API to assert there are no regressions

Noregress can be made to work against WebPageTest.org ‘s REST API. That’s a TODO. See Issue #1

Best Practices

1. Use Noregress as part of your integration tests. Its a great idea to run these tests along with your Selenium tests
2. Make lots of small “actionable assertions”, instead of one big assertion. For example, asserting the overall page score will catch all regressions, but you won’t know what change caused that regression. Instead, checking for something like “Number of JS resources on a page” will tell you exactly what broke the test and how to fix it.
3. You have a limit of 250 requests a day. Assuming you do 10 commits per day, you can test 25 pages on your website. Choose wisely!

Anything else I should know?

1. This library wraps Google’s API, so you should read and comply with their terms and conditions
2. You can only test 250 pages per day. This limit is set by Google’s Page Speed API
3. The pages need to be accessible over public internet. You cannot use this library behind the firewall
4. You cannot use this library for password protected pages

License

Noregress is licensed under the MIT License

The MIT License

Copyright © 2011 Sripathi Krishnan

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the “Software”), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

Author

Sripathi Krishnan
Email : <FirstName><dot><LastName> at gmail.com