GUI Protocol
The Sprockets test driver will communicate with the Sprockets GUI page over a web socket as described here. The test driver must communicate the state transition graph’s structure to the GUI as well as provide updates when transitions are made. The document describes the protocol between the test driver and GUI in detail.
This document is not meant to:
-
Describe the GUI’s elements in details. The GUI will definitely be graph-based. But all the detailed elements of the GUI and how it exactly handles messages from the test driver is a separate discussion. This document only cares that the protocol covers a reasonable range of UI possibilities.
-
Decide what technology will be used to implement the GUI. This document is only focused on the protocol between the test driver and GUI.
For the sake of clarity/illustration, this document will work off of the following basic example:
| state sAppState(string appId) {
kNotRunning,
kRunning
}
state sMediaState(string appId) { kIdle, kPlaying } transition tAppLaunch(string appId) { pre_states = [ sAppState(appId).kNotRunning ] events { // Messages to launch the app. } post_states = [ sAppState(appId).kRunning ] } transition tAppStop(string appId) { pre_states = [ sAppState(appId).kRunning ] events { // Messages to stop the app. } post_states = [ sAppState(appId).kNotRunning, sMediaState(appId).kIdle ] } transition tMediaLoad(string appId) { pre_states = [ sAppState(appId).kRunning ] events { // Messages to load new content. } post_states = [ sMediaState(appId).kPlaying ] error_states = [ sMediaState(appId).kIdle ] } transition tMediaStop(string appId) { pre_states = [ sMediaState(appId).kPlaying ] events { // Messages to stop media session. } post_states = [ sMediaState(appId).kIdle ] } |
| State Id | sAppState | sMediaState |
| S0 | kNotRunning | kIdle |
| S1 | kRunning | kIdle |
| S2 | kRunning | kPlaying |
This is the graph structure that the test driver will communicated with the GUI. Note the diagram below is by no means what the GUI will actually look like. The graph structure is presented here for illustration purposes only.
The test driver and GUI will exchange messages in JSON format. The GUI will most likely be written in javascript, so JSON will be easy to use. It also doesn’t pin Sprockets to a message format that’s very specific to a certain JS library; JSON can be used no matter what GUI technology is chosen.
Describes the state transition graph, the current state, and the current transition for a single test. Note that this protocol accounts for the possibility that the test driver runs multiple tests in its lifetime or even runs multiple tests simultaneously, in which case the GUI would have to display a graph for each test. This is sent:
-
In response to GET_STATUS.
-
Whenever the test driver begins a new test. In the event that there is an existing test running, the STATUS message will only contain the new test’s data.
Coincidentally, the graph library used by the Sprockets test driver (NetworkX) has methods for serializing graph structures into JSON!
https://networkx.github.io/documentation/networkx-1.10/reference/readwrite.json_graph.html
Their JSON format is a nice/readable way to represent the graph, so it’s being reused here with a few additions:
| {
"type": STATUS,
// List of current state transition graphs (there is assumed to be one // graph per active test). This will be empty if there are no active // tests. "graphs": [ { // Graph Structure described below. }, ... ] } // Graph Structure. { // Unique identifier for this graph generated by the test driver. The test // driver is responsible for ensuring that all ids it generates for this // field are unique across the driver's lifetime. "id": "G0", // The name of the STL file corresponding to this graph. "stl_name": "application.stl", // Whether or not the test is running. See PAUSE and RESUME messages below // for context. "is_running": “<true|false>”, // List of nodes in the graph. // "id": Identifier for each node generated by the Sprockets test driver. // Each node identifier must be unique within the graph structure // it belongs to but must not be globally unique. // "values": Combination of state values represented by this node. "nodes": [ { "id": "S0", "values": { "sAppState": "kNotRunning", "sMediaState": "kIdle" } }, { "id": "S1", "values": { "sAppState": "kRunning", "sMediaState": "kIdle" } }, { "id": "S2", "values": { "sAppState": "kRunning", "sMediaState": "kPlaying" } } ], // List of edges in the graph. // "id": Identifier for each edge in the graph generated by the Sprockets // test driver. Each edge identifier must be unique within the graph // structure it belongs to but does not have to be globally unique. // // TODO(esum): Look into using the node id for "source" and "target" // instead. NetworkX uses the node's index instead by default. // "source": Zero-based index of the edge's source node in the "nodes" list. // "target": Zero-based index of the edge's target node in the "nodes" list. // "transition": Corresponding transition name in STL file. // "visit_count": Total number of times the transition has been taken/tested // (initially 0). // "error_node_id": (Optional) Id of the node to transition to if the // current transition is unsuccessful. Reflects the // "error_states" portion of an STL transition. If omitted, // the graph's current node does not change when an error // occurs. "links": [ { "id": "E0", "source": 0, "target": 1, "transition": "tAppLaunch", "visit_count": 0 }, { "id": "E1", "source": 1, "target": 0, "transition": "tAppStop", "visit_count": 0 }, { "id": "E2", "source": 1, "target": 2, "transition": "tMediaLoad", "visit_count": 0 }, { "id": "E3", "source": 2, "target": 0, "transition": "tAppStop", "visit_count": 0 }, { "id": "E4", "source": 2, "target": 1, "transition": "tMediaStop", "visit_count": 0 }, { "id": "E5", "source": 2, "target": 2, "transition": "tMediaLoad", "visit_count": 0, "error_node_id": "S1" } ], // Id of the test's current node in the "nodes" list. "current_node_id": 'S0', // Id of the current edge that is being tested by the test driver. "current_edge_id": 'E0', } |
Describes an update that has occurred in a state transition graph. This is sent whenever:
- An STL transition has completed (either successfully or unsuccessfully).
| {
"type": "TRANSITION_UPDATE",
// Id of the graph that this update is for. Matches "graph.id" in // "Graph Structure" "graph_id": "G0", // Id of the graph's previous node. "previous_node_id": "S0", // Id of the graph's new current node. "current_node_id": "S1", // Id of the graph's previous edge. "previous_edge_id": "E0", // Id of the graph's new current edge. "current_edge_id": "E2", // Whether or not the STL transition specified in the "previous_edge_id" // field succeeded. “success”: <true|false> } |
Notifies the GUI that a test has completed. This is sent whenever:
- A test finishes.
| {
"type": "TEST_RESULT",
// Id of the graph corresponding to this test result. "graph_id": "G0", // List of all transitions taken during the test. They shall be listed in // the order that they were taken. "transitions_taken": [ { // Each entry contains all of the fields in the "TRANSITION_UPDATE" // message except for the “type” field. }, ... ], // Time that the test started. “start_time_ms”: , // Time that the test ended. “end_time_ms”: } |
Requests a STATUS message from the Test Driver. This is sent whenever:
-
The GUI starts up and is ready to start rendering the UI. This is the first message sent over the web socket. The test driver should respond with a STATUS message.
-
The GUI feels like it… This really can be sent at any time for whatever reason. The use case above is the most prominent one though.
| {
"type": "GET_STATUS",
// (Optional) List of graph ids that the GUI wants to fetch the STATUS for. // The test driver's STATUS response should contain Graph Structures for // the requested graph ids. If a graph id does not exist, the Graph // Structure should be omitted from the STATUS response and an "ERROR" // message will be sent with “code” set to “UNKNOWN_GRAPH_ID”. If this // field is omitted, the test driver should respond with the STATUS for // all active graphs. "graphs_ids": [ "G0" ] } |
Pauses the tests with the specified "graph_ids". This means that the Test Driver will not continue to the graph’s next transition once the current transition completes. Note that this does not interrupt the current transition. The Test Driver will wait for the current transition to complete and remain in the current transition’s “post_state” until a “RESUME” message is sent. Note that the “is_running” flag will be false for this graph in future “STATUS” messages.
If an entry in the "graph_ids" field does not exist, it’s ignored and an “ERROR” message is sent from the Test Driver -> GUI with “code” set to “UNKNOWN_GRAPH_ID”. If the test is already paused, this is a no-op.
| { "type": "PAUSE", // STL transition graphs to pause. "graphs_ids": [ "G0" ] } |
Resumes the tests with the specified "graph_ids". The Test Driver will continue to the next planned state transition. Note that the “is_running” flag will be true for this graph in future “STATUS” messages.
If an entry in the "graph_ids" field does not exist, it’s ignored and an “ERROR” message is sent from the Test Driver -> GUI with “code” set to “UNKNOWN_GRAPH_ID”. If the test is already running, this is a no-op.
| { "type": "RESUME", // STL transition graphs to resume. "graphs_ids": [ "G0" ] } |
Communicates an ERROR message in either direction. Note this is not meant to communicate error transitions in the STL. These are unexpected run-time errors for debugging and possibly
UI purposes (ex: test driver hits internal unrecoverable error, sends INTERNAL_ERROR to GUI, and the GUI shows the user that the test has hit an error).
| { "type": "ERROR", "code": “”, // Holds any metadata/extra information associated with the error code. This // fields contents vary depending on the error code’s value. "custom_data": "" } |
Lists the different error codes that can be sent in the ERROR message.
(Table to be expanded as use cases arise)
| Code | Direction | Custom Data |
| UNKNOWN_GRAPH_ID | Test Driver -> GUI | ID of graph that’s unknown |
| INTERNAL_ERROR | Test Driver -> GUI | Error message describing what went wrong. (Can be displayed by the GUI). |