-
Notifications
You must be signed in to change notification settings - Fork 118
Writing Tests
At Discovery Garden, we're implementing Continuous Integration using Jenkins, an open source CI solution. This means that every time code is committed to certain GitHub repositories, automated tasks will be executed against the codebase. Included in these tasks will be a battery of tests executed in multiple environments. Once code passes the entirety of the test suite in all environments, it can be deployed into production. If any test fails, the developer will be notified so that the issue can be addressed immediately. Adopting this practice will lower the risk of development and maintenance, while increasing overall confidence in the product.
All tests will be written using Drupal's Simpletest module, which has been integrated into the Drupal Core as of version 7. The Simpletest module provides some conveniences for writing tests, most notably assertions, and can be used to develop unit tests for verifying basic functionality outside of a fully bootstrapped Drupal context. The Simpletest module also has the ability run what it calls "web" tests, providing a fully bootstrapped Drupal context for each test function.
- Download and install any external dependencies (most likely
php_curl). - Download, install, and enable the
develmodule for thedd()function. - Download and install the Simpletest module.
- Install the Drupal patch that is supplied with the Simpletest module
- (Optional, but likely a good idea) Update each of your Islandora module/libraries
- Run update.php
Simpletest is part of Drupal Core in Drupal 7, so you shouldn't have to do anything other than download, install, and enable the devel module for debugging with the dd() function.
D6: If you need to perform some Fedora functionality in your tests, which is most likely the case, you need to add a special user to your Fedora installation. Add the following XML snippet to your $FEDORA_HOME/server/config/fedora-users.xml file:
<user name="simpletestuser" password="41fe63c9636c6649f0a4747400f0f95e">
<attribute name="fedoraRole">
<value>administrator</value>
</attribute>
</user>
D7: See the readme in the test directory.
All test files end in the extension .test, and should reside alongside the code whose functionality is being tested. For example, if you have a module with a basic file structure such as
-
my_modulemy_module.infomy_module.installmy_module.module
In Drupal 6, all you have to do is create a my_module.test file and place it in the my_module directory. If you're in Drupal 7, there's the additional step of pushing the file name onto the files array in your .info file. Simply add a line like this to your .info file:
files[] = my_module.test
You are not limited to a single test file per module, and can have any number of test files so long as they all end in the .test extension. Let's say we've added an extra include file tomy_module, and our directory structure looks like this:
-
my_modulemy_module.infomy_module.installmy_module.modulemy_module.inc
If we wanted to write web tests for the module (to test its hooks, for example) and unit tests for the functions in the include file, we can create my_module.test and my_module.inc.test, respectively, in the my_module folder. If we're using Drupal 7, we'll also have to add the following two lines to my_module.info.
files[] = my_module.test
files[] = my_module.inc.testAll test files will contain a class definition for a class that ultimately extends either DrupalUnitTestCase or DrupalWebTestCase, but in either situation, the required boiler-plate code is essentially the same.
In order for tests to be used by Simpletest, four things must be done:
- Extend either DrupalUnitTestCase or DrupalWebTestCase (or a class that extends either, like IslandoraWebTestCase - more on that later).
- Implement a static
getInfo()function, which provides basic descriptive info for Simpletest's interface. - Implement a
setUp()method, which performs any neccessary initializations before running each test. - Implement one or more test methods. All methods starting with 'test' in lower-case will be automatically invoked by Simpletest when the tests are run. All test methods should perform at least one assertion.
The static getInfo() function is used by Simpletest's interface when choosing which tests to run. It returns an associative array of the form:
<?php
array(
'name' => 'Name of test case',
'description' => 'Describes what gets tested',
'group' => 'Group the test case belongs to',
)The setup function executes initializations needed for each test function. That is, before any test function is executed, the setUp() method is invoked. For unit tests, setUp() can be used to instantiate dummy objects for use in the tests. For example:
<?php
class MyUnitTestCase extends DrupalUnitTestCase {
protected $myTestObject;
public function setUp() {
$this->myTestObject = new TestObject();
}
}Now that the setUp() method has been defined, the myTestObject variable can be used within any of the test methods.
If you're writing a DrupalWebTestCase, the setUp() method becomes much more important. It is responsible for initializing the Drupal context created for each and every test, which includes enabling all modules the tested module is dependant on. It is also required to create a test user and log that user into the Drupal context. Manually doing this for every web test you write is tedious and time consuming, but fortunately most of the heavy lifting has already been done for you if you choose to extend IslandoraTestCase, which extends DrupalWebTestCase. Please see the section on IslandoraTestCase for usage.
The name of every test method must begin with a lowercase 'test'. In order to follow proper Drupal style guides, the method name must be in drinking camel case (e.g. testSomeFunctionality()). All test methods will perform one or more assertions. If any assertion fails, the test fails. By extending DrupalUnitTestCase or DrupalWebTestCase, you automatically have access to a great many assertions. Here's an example of a trivial test function
<?php
public function testSomething() {
$this->assertTrue(TRUE, "This test will always pass");
}Putting it all together, here's an example of a unit test file:
<?php
class MyUnitTestCase extends DrupalUnitTestCase {
protected $myTestObject;
public static function getInfo() {
return array(
'name' => 'My Unit Test Case',
'description' => 'Tests a test object',
'group' => 'Some Test Group',
);
}
public function setUp() {
parent::setUp();
$this->myTestObject = new TestObject();
}
public function testMyObject() {
$this->assertNotNull($this->myTestObject, "This test should always pass because we create 'myTestObject' in the setUp() method.");
}
}If you're writing web tests, a lot of the legwork is already done for you if you choose to extend IslandoraWebTestCase instead of DrupalWebTestCase. It is located in the main Islandora repository in the file islandora/fedora_repository.test.inc.
IslandoraTestCase handles the Drupal bootstrapping process for you, as well as creating and logging in a test user with Fedora permissions. The test user is available as the protected member variable $privileged_user. There's even a utility function for invoking module hooks, convienently named invoke().
All you have to do is provide an implementation for the abstract getModuleName() method that returns the name of the module you are testing as a string. You can optionally override the getUserPermissions() function to provide extra permissions for the test user.
If you choose to override the setUp() method, be sure to call parent::setUp(), or the Drupal bootstrapping process will not occur.
Assuming we have a module named 'example_module', here's an example IslandoraTestCase to test 'example_module'.
<?php
// For IslandoraTestCase
module_load_include('inc', 'fedora_repository', 'fedora_repository.test');
/**
* An example illustrating the usage of IslandoraTestCase. Performs tests on the imaginary 'example_module'
* module.
*/
class ExampleIslandoraTestCase extends IslandoraTestCase {
// Needed for invoke() and setUp(), amongst other things.
protected function getModuleName() {
return "example_module";
}
// Here we're adding an extra permission on top of the standard Fedora permissions.
protected function getUserPermissions {
$out = parent::getUserPermissions();
$out[] = 'some new permission';
return $out;
}
// Here we perform any additional setup on top of bootstrapping Drupal and logging in a test user.
public function setUp() {
parent::setUp();
// Do your extra setup here!
}
// Example test function.
// Asserts the output of a hook is not null
public function testSomeHook() {
$results = $this->invoke('some_hook');
$this->assertNotNull($results, "The results of 'some_hook' are not NULL.");
}
}While developing new features or revisiting old code, it is the task of the developer to provide tests demonstrating that the basic functionality of the code is working as expected. This can be a daunting task, especially when facing a large and established code base. Although the exercise of creating tests ultimately results in a judgement call of some sort, here are a few basic guidelines to help get you started.
If you don't need a full Drupal context in order to execute your tests, you should always use unit tests over web tests. They are significantly faster without the required overhead of bootstrapping a Drupal instance. Web test cases should only be used when a Drupal context is required.
Nobody's perfect. Code normally doesn't magically work the first time every time, including test code. Unfortunately, there's an extra hoop you'll have to jump through if you want to debug your test code. Since web test cases run in their own Drupal context, which dissipates as soon as the test function has finished execution, normal Watchdog logging will not work. You need to install and enable the devel module so that you can use dd() to log. Output from dd() appears in /tmp/drupal-debug.txt, which you can tail as you execute your test suite. In D7 (maybe D6 too) the line number of exceptions will only be printed for web test cases and not unit test cases. A quick and easy debug is to switch to web test case during the development of tests, but remember to switch back.
This approach to testing is known as Black Box Testing. Black box testing performs tests on an interface, without needing to know the implementation details of the module being tested. All public functions should have at least one test, and tests should verify that the function's output is appropriate given the input that you provide. If you are testing a module, each of the module's hooks should have at least one associated test method.
The reason why we write black box tests is so that as developers, we encounter significantly reduced risk when refactoring code. With a robust suite of tests in place for any given module, we can feel safe performing large refactors because we know that we will be alerted immediately if any new code interferes with the module's expected behavior.
At times, code should be expected to fail, especially if a function is provided faulty mangled data or incorrect arguments. These situations should be tested using the assertFalse() and assertNull() assertions.
You want your test functions to be as small as possible. When possible, you should only test one behavior or function per test method. There's nothing worse than a complicated and messy test function, especially when the function or behavior you are testing is also complicated and messy.
Tests, along with a variety of other useful tasks, are automatically executed by our Jenkins server using Apache Ant. In order for build automation to occur, a few housekeeping tasks must be taken care of.
Add a build directory to the root folder of your module. Your build directory will contain a single file, the Doxyfile. The Doxyfile is used for API documentation generation, and can easily be copied over from another Islandora module that's already been automated with Jenkins. Just be sure to change the 'PROJECT NAME' entry in the file to reflect your own module.
All tasks to be automated by Jenkins must be implemented in an Apache Ant build file. Each entry in the build file represents a task to be executed by Jenkins. The build file must be located in the root directory of your Islandora module and must be named 'build.xml'. Here's an example buildfile from one of the solution packs:
<?xml version="1.0" encoding="UTF-8"?>
<project name="islandora_solution_pack_image" default="build">
<target name="build" depends="clean,prepare,lint,phploc,code_sniff,phpcpd,pdepend,doxygen,phpcb,test" />
<target name="clean" description="Cleanup build artifacts">
<delete dir="${basedir}/build/test" />
<delete dir="${basedir}/build/logs" />
<delete dir="${basedir}/build/pdepend" />
<delete dir="${basedir}/build/api" />
<delete dir="${basedir}/build/code-browser" />
</target>
<target name="prepare" description="Prepares workspace for artifacts" >
<mkdir dir="${basedir}/build/test" />
<mkdir dir="${basedir}/build/logs" />
<mkdir dir="${basedir}/build/pdepend" />
<mkdir dir="${basedir}/build/api" />
<mkdir dir="${basedir}/build/code-browser" />
</target>
<target name="lint" description="Perform syntax check of sourcecode files">
<apply executable="php" failonerror="true">
<arg value="-l" />
<fileset dir="${basedir}">
<include name="**/*.php" />
<include name="**/*.inc" />
<include name="**/*.module" />
<include name="**/*.install" />
<include name="**/*.test" />
<modified />
</fileset>
</apply>
</target>
<target name="phploc" description="Measure project size using PHPLOC">
<exec executable="phploc">
<arg line="--log-csv ${basedir}/build/logs/phploc.csv --exclude build --exclude css --exclude images --exclude xml --names *.php,*.module,*.inc,*.test,*.install ${basedir}" />
</exec>
</target>
<target name="code_sniff" description="Checks the code for Drupal coding standard violations" >
<exec executable="phpcs">
<arg line="--standard=Drupal --report=checkstyle --report-file=${basedir}/build/logs/checkstyle.xml --extensions=php,inc,test,module,install --ignore=build/,css/,images/,xml/ ${basedir}" />
</exec>
</target>
<target name="phpcpd" description="Copy/Paste code detection">
<exec executable="phpcpd">
<arg line="--log-pmd ${basedir}/build/logs/pmd-cpd.xml --exclude build --exclude css --exclude images --exclude xml --names *.php,*.module,*.inc,*.test,*.install ${basedir}" />
</exec>
</target>
<target name="pdepend" description="Calculate software metrics using PHP_Depend">
<exec executable="pdepend">
<arg line="--jdepend-xml=${basedir}/build/logs/jdepend.xml --jdepend-chart=${basedir}/build/pdepend/dependencies.svg --overview-pyramid=${basedir}/build/pdepend/overview-pyramid.svg ${basedir}"/>
</exec>
</target>
<target name="doxygen" description="Generate API documentation with doxygen" depends="prepare">
<exec executable="bash">
<arg line='-c "sed -i s/PROJECT_NUMBER\ \ \ \ \ \ \ \ \ =/PROJECT_NUMBER\ \ \ \ \ \ \ \ \ =\ `git log -1 --pretty=format:%h`/ build/Doxyfile"'/>
</exec>
<exec executable="doxygen">
<arg line="${basedir}/build/Doxyfile" />
</exec>
<exec executable="git">
<arg line="checkout ${basedir}/build/Doxyfile"/>
</exec>
</target>
<target name="phpcb" description="Aggregate tool output with PHP_CodeBrowser">
<exec executable="phpcb">
<arg line="--log ${basedir}/build/logs --source ${basedir} --output ${basedir}/build/code-browser"/>
</exec>
</target>
<target name="test">
<exec executable="bash">
<arg line='-c "php ../../../../scripts/run-tests.sh --xml ${basedir}/build/test Islandora Solution Pack Image"' />
</exec>
</target>
</project>In order to use this for one of your own modules, a few minor modifications will need to be made. First, change the name attribute in the main project tag to reflect your own module's name. Second, add the appropriate directories to ignore/exclude in the phploc, code_sniff, and phpcpd tasks. Finally, change the name of the test suite to run in the 'test' task to be the test group of your own module. A list of all available test suite names is available by issuing a drush test-run from the command line while within the Drupal directory tree.
Make sure that the 'test' task execs bash to call php instead of using Drush. Using drush test-run to execute tests will work 90% of the time, but we've had strange permission issues that happen on a few of the test suites when using drush. If your tests run through simpletest's web interface but not through drush, chances are using php scripts/run-tests.sh will solve your problems.
Lastly, you'll need to make a proper Jenkins project through its web interface. If you don't have administrative privileges, contact your friendly neighborhood Jenkins admin and have him/her make one for you. If you're lucky enough to have the proper privileges, it can very easily be done. Click on the link to make a new job, name it, and select the "Copy existing job" option. Pick any of the other jobs on the box (the text field will even autocomplete it for you), and put the appropriate information in the following fields:
- GitHub project (3rd text field)
- Custom workspace and display name (under Advanced) - You'll need to have the module initially deployed on the server in order for this to work. Again, if you don't have the creds, contact your friendly neighborhood administrator to help you out.
- Git repository (under Source Code Management)
In order for Jenkins to execute its tasks upon a Github push, you'll need to add http://jenkins.discoverygarden.ca:8080/github-webhook/ as a webhook url in the Service Hooks section of the repository settings.
You may be looking for the islandora-community wiki · new to islandora? · community calendar · interest groups