Skip to content

Latest commit

 

History

History
195 lines (150 loc) · 11 KB

README.md

File metadata and controls

195 lines (150 loc) · 11 KB
Raw


Screenshot

image-6

السلام عليكم

احسب ان هذه الفرصة هي خير و نعمة من الله فالحمدلله الذي رزقني هذا فما توفيقي الا به و ما توكلي الا عليه

Welcome to My GSoC Journey with Keploy 🐰!

image-7

👋 Hello and welcome to my project for Google Summer of Code (GSoC)! I'm excited to share with you the journey of building contract testing capabilities for Keploy, a powerful open-source tool designed for automated end-to-end test generation, mocking, and test management.

Through this project, I worked on enhancing Keploy by implementing contract testing features that automate the validation of interactions between different services. This ensures seamless communication through APIs, reducing the risk of integration errors and service downtimes.

In the sections below, I will walk you through the features of the project, how the schema matching and scoring systems work, and the value this project adds to the software development process. I'm also excited to share how this experience has helped me grow technically and professionally during GSoC.

📑 Table of Contents

My Proposal

You can find my proposal in this repo or ask me if you have any questions.

Key Features 🚀

1. Generate Contracts

The generate command allows users to create contracts for specified services. It auto-generates contracts based on the HTTP methods (like GET, POST, etc.), request body, response status, and body. This checks if the services behave according to expectations in terms of their input/output interactions.

Command Example:

keploy contract generate

When executed, this command will:

  • Capture HTTP requests and responses
  • Generate contracts in OpenAPI formats
  • Save these contracts for future use (validation, mocking, etc.)

Screenshot

2. Download Contracts

The download command fetches contracts for specific services from either remote repositories or local configurations. This helps maintain consistency across different environments (development, testing, production).

Command Example:

keploy contract download --driven "consumer" --path /local/path

When executed, this command will:

  • Download contracts from the specified path or repository
  • Ensure consistent versioning across testing and production environments
  • Make these contracts available for validation or mock generation

3. Validate Contracts

The validate command checks whether the service's behavior matches the generated contract. It ensures no unexpected changes occur in the service's communication patterns, reducing API-breaking changes.

Command Example:

keploy contract validate --driven "consumer" --path /local/path

When executed, this command will:

  • Compare the current service behavior against the contract
  • Log and report any discrepancies
  • Help identify if any API-breaking changes have occurred

Screenshot

4. Support for Multiple HTTP Methods

This project supports multiple HTTP methods, including GET, POST, PUT, DELETE, and PATCH. It automatically maps request parameters such as headers, query parameters, and path variables, validating the structure of the response.

5. OpenAPI Integration

The system integrates with OpenAPI for contract generation and validation. HTTP requests and responses are converted into OpenAPI documentation, ensuring standardized contract storage and validation in OpenAPI format.

6. Mock Generation for Testing

The mock feature generates mock data based on service contracts. This allows testing services in isolation without needing real service dependencies, improving the testing process.

Matching and Scoring 🎯

Matching Overview

The matching process compares mock and test services' request and response bodies, headers, parameters, and status codes. The goal is to ensure that both mock and test operations behave similarly, adhering to the expected contract.

The matching is carried out by:

  • Comparing the operation types (GET, POST, etc.) to ensure they match
  • Comparing the request bodies and validating JSON structures using schema comparison tools
  • Comparing parameters such as headers, query parameters, and path variables to ensure they align
  • Comparing response bodies, status codes, and content
  • Handling discrepancies using diff tools to highlight differences in the request/response payloads

Scoring System

The scoring system evaluates how closely the test service adheres to the contract. It assigns a score based on the similarity of the expected and actual outputs. Here's how the scoring is calculated:

  • Operation Matching: If the mock and test operations (GET, POST, etc.) match, a higher score is assigned.
  • Request Body Matching: The system compares the JSON structures of the request body. If the structures are identical, a higher score is given. The system uses JSON diff tools to calculate the differences between the expected and actual JSON.
  • Response Body Matching: The similarity between the response bodies (in terms of status code, body content, and headers) also impacts the score. A close match will result in a higher score, while differences will reduce the score.
  • Parameter Matching: Parameters such as headers, query parameters, and path variables are compared. Matching parameters contribute to a higher score.

Score Calculation

The score is normalized and expressed as a percentage. A score close to 100% indicates that the test service adheres closely to the contract, while lower scores indicate discrepancies.

For instance:

  • A match in operation types contributes a fixed percentage (e.g., 30%).
  • A match in request body structure adds another percentage (e.g., 40%).
  • A match in response body structure adds the remaining percentage (e.g., 30%).

Error Handling

In case of mismatches or errors during comparison, the system logs detailed reports and differences between the expected and actual values. This helps developers understand where the mismatches occurred and take appropriate action.

🎯 Sample

You can run a sample from this repo

Code 💻

You can take a look here

💡 Value Added

1. Automated Contract Verification

This project automates contract validation, reducing the manual effort required in integration testing. It automatically checks service compatibility against existing contracts.

2. Improved Service Integration

By ensuring services do not break integration points, contract testing reduces integration issues in microservice architectures.

3. Prevents Breaking Changes

The project highlights and prevents changes that violate the service contract, ensuring API stability and backward compatibility.

4. Faster Development Cycles

By validating contracts early, teams can resolve integration issues before they reach production, leading to faster and more reliable development cycles.

5. Mocking for Independent Testing

Mock data generation allows developers to simulate service interactions, enabling isolated testing without dependencies on real services.

6. OpenAPI Compliance

By using OpenAPI standards, this system ensures interoperability with various tools and services, automating the generation of OpenAPI documentation from service behaviors.

7. Cross-Environment Consistency

The system ensures that contract testing can be consistently applied across different environments (development, testing, production).

8. Clear Reporting

Detailed reports are generated during validation, highlighting passed, failed, and missed test cases for easy debugging and triaging.

Special Thanks

I would like to extend my deepest gratitude to my amazing mentors who guided me throughout this journey:

  • Gourav - Your wisdom and feedback were instrumental in helping me grow both technically and professionally.
  • Shivam - Thank you for your constant support and for always being available to answer my questions.

I truly appreciate all the time and effort each of you put into making this project a success. Your mentorship has been invaluable, and I am forever grateful!