Teakozi

Teakozi is a declarative REST API testing framework for testing micro-services. Here are the capabilities.

• Writing tests as readable YML files
• Externalize the configuration / environment specific parameters
• Dynamic Data - Collect Property and use it in subsequent steps.
• Scenario based testing. Externalize the test data and validation data from test specification
• Reuse API calls in multiple tests
• Validate response schema as per Swagger definitions
• Library extensions

How to use Teakozi

• include teakozi in your node application

$ npm install teakozi

create a directory in your project consisting of the following subdirectories

• tests
• tests/[testname].yml
• modules
• modules/[modulename].yml
• models
• models/[modelxx].js
• config
• config/index.js
• payload
• payload/[xxx].json

Directory	Purpose
tests	contains the yml files that describe the tests. You can arrange the test yml files in any suitable folder hiererchy. the framework will get all yml files in directory under this tree and ignore all the non .yml files
modules	reusable test steps that can be refered in the test yml file
models	externalize the data to drive the tests
config	just one index.js file
payload	the body of the content that can be called in a post step. This folder also contains the json payload against which you need to validate your responses.

config/index.js

This file contains

1. key value for replacing in the test yml files.
2. The index file can optionally have a tag called swagger pointing to the location of the swagger file from project home directory.
3. Also has extension functions that can be defined and used in the test definition. Take a look at section library functions below.
4. sync flag - whether the test should run sync / async. please refer to section sync / async as the behavior is dependent on test sync attribute and config sync attribute

see the example folder in the repo for reference. you need to update the config/index.js with your github auth to

Externalize data from tests

Teakozi allows you to externalize the data from test definition

Reusable modules

you can refer to the steps in the module as - {{module name}}

Configuration data

{{xxx}} marked content gets replaced from the configuration file keys-> value

Dynamic data

you can collect data in a step and reuse in the subsequent steps. so you can collect title and refer it in subsequent steps like ~title~

Library Functions

Add library functions in your config/index.js file as follows

Sync / Async calling

sync tag can be applied at the config level and also at file level. the framework will execute the test in the following manner

SYNC	Config true	Config false	Config undefined
Test - true	Sync	Sync	Sync
Test- false	Sync	Async	Async
Test - undefined	Sync	Async	Sync

This behavior is subject to change based on user feeback. Current phillosophy - default to sync when not explicit

module.exports = {
  swagger:'/config/swagger.yml',
  lib:{
    increment : function(i){return ++i},
    decrement : function(i){return --i}
  }
};

you can now use the increment and decrement functions while collecting the parameters from response. example

    collect:
     incremeted_id: increment->$.order[0].id
    print:
     - $.order[0].id
     - incremeted_id

Calling the tests from your node app

in a .js file call

require("teakozi").start("project/example")

Structure of a test yml file (tests directory)

Property	required	type	Purpose
name	required	string	Name of the Test Case
tags	optional	string	comma saperated list of tags for the test. you can use thse tags to filter / select the test you would like to take for a test run
sync	optional	bool	Whether the test should run sync / async. please refer to section sync / async as the behavior is dependent on this and config sync flag
iterate	optional	string	Name of the module file that returns an array of objects. The test steps defined will be repeated for all the elements of this array
steps	required	array	See Steps Section

Steps

Property	required	type	Purpose
get / post / put / delete / local / mongo	required	object	method to be called
name	required	string	Name of the Test Step
delay	optional	int[,int]	Delay in seconds to start this step. Optionally separated by comma, you can specify how may times should you loop before giving up. Example 3,5 means do the step after 3 seconds delay. if fails then retry for 5 times every 3 seconds. if passing move to next step
iterate	optional	string	name of the module that returns array of objects. the call will get repeated for each of the members of the returned array
check	required	object	what to do with the response received. What asserts to do and what properties to pick from the response to be used in subsequent calls
collect	optional	object	the jsonpaths values that should be collected for use in subsequent calls.
print	optional	array	the jsonpaths values that should be printed on the console output for debugging purposes
save	optional	object	the jsonpaths values that should be saved and the file name where it should be saved example : $: my_response_content The files are saved in the generated folder on the project
skip_on_error	optional	bool	Default: true. If entered false, it will execute the step even if the previous steps have failed. Can be used to do cleanup / teardown activities

get / post / put / delete

Property	required	type	Purpose
url	required	string	The API to be called
json	na -get, optional - for post, optional - put, na - delete	string	the name of file in modules that returns the json to be posted
file	na -get, optional - for post, optional - put, na - delete	string	Name of the file in modules (plain text) that needs to be posted
override	na -get, optional - for post, optional - put, na - delete	array	jsonpaths that need to be updated with value
headers	optional -get, optional - for post, optional - put, optional - delete	array	headers that go on the http requests. like Authorization content-type etc

example:

post:
     url: "https://api.github.com/gists/~gist_id~/comments"
     json: "create_comment.json"
     override:
      $.body: ~comment_text~
     headers:
      Authorization: Basic xxxx
      content-type: application/json
      User-Agent: Teakozi-test

local

Property	required	type	Purpose
file	required	string	name of the file in the payload folder

Example:

local:
    file: roads

mongo

Property	required	type	Purpose
server	required	string	connection string / endpoint of the mongo server like "mongodb://localhost:27017"
db	required	string	the name of the database
collection	required	string	collection name in the above db from where the records have to be fetched
query	required	object	the filter criteria of querying the records from collection

Example:

  - mongo:
     server: "mongodb://localhost:27017"
     db: 'test'
     collection: "test"
     query:
       name: j

collect

Collects the propoerties from the payload response that would be used in the subsequent steps

    collect:
     title: $..[0].name
     project_id: $..[0].id

collect also supports reading complete or subset of json recieved like

    collect:
     all_roads: $..Name

this will collect all the Name fields comming at any part of the json. using JSON path expression on the right hand side you can collect specific sections of json for comparision

print

Enables debugging of the call by printing the jsonpaths from the response

  - get:
     url: "http://localhost:3060/k.json"
    name: "Get Auth K"
    print:
     - status
     - $.mykey
    check:
     status: 200
    collect:
     auth_key: $.mykey

print also supports checking the objects / subset of json as defined by json path like

print:
 - status # status of the header
 - $..Name # All names appearing in the json
 - $[0].Name # Prints name value of the first record
 - $ # Prints complete JSON

check

Property	required	type	Purpose
status	optional	number	what status code is expected in response
schema	optional	string	you can add the name of the Model corresponding to the expected response. The swagger definition should be done in config/index.js
eq	optional	object	the jsonpaths that should be equal to
neq	optional	object	the jsonpaths that should notbe equal to
null	optional	array	the jsonpaths that should be null
deepEqual	optional	array	The jsonpaths and what collected objects to check against.
shallowEqual	optional	array	compare two collected json to match and validate such that each element of A is present in the B (A is a subset B is a superset)
regex	optional	array	The jsonpaths and their matching regex. The regex expression can be external mentioned as regex_for_currency or mentioned inline

example:

    check:
     status: 200
     schema: Domain
     body:
      eq:
       $.length: 20
       $..[0].name: "devops_best_practices"
      regex:
       $.domain: "^Ji[a-z]*"
      neq:
       $.length: 0
      null:
       - $.nonexistant
       - $.someother_nonexistant
      deepEqual:
        $: full_roads_payload
        $..Name: all_roads
      shallowEqual:
        - sub: sub
          super: super
          ar: empid
          permissionList: permissionId

where full_roads_payload is collected in a previous step

The check expression can be json path expression or can also represent a collected value from a previous or current step

shallowEqual

Shallow Equal is a way to compare two collected jsons to match and validate such that each element of A is present in the B (A is a subset B is a superset).

Property	required	type	Purpose
sub	required	string	the name of the object collected in this or previous steps that should be the subset
super	required	string	the name of the object collected in this or previous steps that should be the superset
[key]	optional / multi	string	key is the arrays in the object and value is the attribute of the object that should be used to compare and identify the object

example JSON A (subset)

{
    organization:"abc",
    address:"outer ring road",
    city:"bangalore",
    employees:[{
        emp: 123,
        name: "Ned Stark"
    },{
        emp: 234,
        name: "John Snow"
    }]
}

JSON B (superset)

{
    organization:"abc",
    address:"outer ring road",
    city:"bangalore",
    pin: 560102
    employees:[{
        emp: 123,
        name: "Ned Stark",
        cool: true
    },{
        emp: 234,
        name: "John Snow"
    },{
        emp: 345,
        name: "Sansa"
    }]
}

if test.yml

  shallowEqual:
    - sub: A
      super: B
      employees: empid

Result: Pass

Save

Save is a utility that allows the resoponse recieved from the step into a file. you have control to save the full payload or partial payload using jsonpath

example:

    save:
      $: "file_mongo_recs"
      $..name: "names_in_response"

More on Schema Validation

Define the location of the swagger file in your config/index.js with the name

 swagger: /config/swagger.yaml

In your check attribute you can add the name of the Model corresponding to the expected response.

 check:
   schema: Domain

The framework will add an assertion step to check if the schema of the response is matching with the schema as defined in the swagger. the assertion fails if the schema is not found or schema mismatch. the details of which all attribute are not matching schema is described in the test log JSON and viewable in the teakozi-viewer.

Running a subset of your test suits based on tags

in your index.js file

var tags_list = "a,b,c,d"
require("teakozi").start("project/github",working dir,{tag:tags_list})

run the test cases

$ node index.js

Chaining test execution

The start function returns a promise that is resolved when tests complete. you can chain further tests like following or handle the test end event for any other handling

var dir = "/home/jitendra/code/teakozi-examples"
var t = require("teakozi")
t.start("project/github",dir,{tag:"cars"})
  .then(()=>{
    t.start("project/github",dir,{tag:"swagger"})
  })

The test logs are shown on screen and the logs are written into the project directory/log with a timestamped folder containing all and individual test run information

Teakozi-Viewer

This supporting project enables you to view the HTML reports from the collected logs in the log directory (all.js). Please clone and install from the following github repository https://github.com/jpthethi/teakozi-viewer

Teakozi-Editor

Teakozi editor https://github.com/jpthethi/teakozi-editor is a companion to Teakozi framework. The editor helps in providing a GUI for creating and editing the yml files that describe the tests.

Feedback

Please submit your suggestions / bugs via Github. You are welcome to fork and submit back your pull requests

License

ISC