This repository contains all the source-code and experiments related to the following paper:

Pouria Alikhanifard and Nikolaos Tsantalis, "A Novel Refactoring and Semantic Aware Abstract Syntax Tree Differencing Tool and a Benchmark for Evaluating the Accuracy of Diff Tools," ACM Transactions on Software Engineering and Methodology, 2024.

The oracle files are located in the oracle folder and the Experiment Results are located in the out folder.

Please check the Wiki for the detailed explanation. You can find the list of all supported ASTDiffTools here.

Table of Contents

• General info
• How to build DiffBenchmark
• Usage guidelines
  • With a locally cloned git repository
  • With two directories containing Java source code
  • With all information fetched directly from GitHub

General info

DiffBenchmark is a visualization and benchmarking tool for ASTDiff tools. It provides an infrastructure to compare the results of different ASTDiff tools with the oracle files.

It's extensible architecture allows you to add new ASTDiff tools and compare them with the existing ones.

You can visualize, compare, and evaluate the results of different ASTDiff tools using the DiffBenchmark.

How to build DiffBenchmark

First, clone the RefactoringMiner repository.

Then, you have to update the REFACTORING_MINER_PATH in one of the following ways:

Update the field REFACTORING_MINER_PATH in the benchmark.utils.Configuration.ExperimentFactory class to the RefactoringMiner cloned repository path in your hard drive or you can have it as env variable REFACTORING_MINER_PATH.

Import DiffBenchmark as a gradle project in your IDE, or run ./gradlew jar to build.

Usage guidelines

There are 3 different ways you can execute DiffBenchmark: You can find the examples here.

With a locally cloned git repository

Execute CompareWithLocallyClonedRepository.java

String repo = "https://github.com/Alluxio/alluxio.git";
String commit = "9aeefcd8120bb3b89cdb437d8c32d2ed84b8a825";
String pathToClonedRepository = "tmp/" + "Alluxio/Alluxio";
Repository repository = new GitServiceImpl().cloneIfNotExists(pathToClonedRepository, repo);
new BenchmarkWebDiffFactory().withLocallyClonedRepo(repository, commit).run();

With two directories containing Java source code

Execute CompareWithTwoDirectories.java

String folder1 = "PATH_TO_FOLDER1";
String folder2 = "PATH_TO_FOLDER2";
new BenchmarkWebDiffFactory().withTwoDirectories(folder1,folder2).run()

With all information fetched directly from GitHub

To use the following API, please provide a valid OAuth token in the github-oauth.properties file. You can generate an OAuth token in GitHub Settings -> Developer settings -> Personal access tokens.

Then add the token as OAuthToken env variable

Execute CompareWithGitHubAPI.java

String url = "https://github.com/Alluxio/alluxio/commit/9aeefcd8120bb3b89cdb437d8c32d2ed84b8a825";
new BenchmarkWebDiffFactory().withURL(url).run();