Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

initial version of visual CI suite [#61] #63

Merged
merged 2 commits into from
Jul 17, 2023
Merged

initial version of visual CI suite [#61] #63

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ea1f0ad
initial version of visual CI suite [#61]
bdon Jul 15, 2023
c14f566
add ci README
bdon Jul 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions ci/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Basemaps Visual Test Suite

## Artifacts

Recent git commits on `main` and pull requests each have a set of artifacts.

The test suite depends on the existence of a public HTTP endpoint with these paths:

* `ci-storage.protomaps.com/smalltestregion.osm.pbf`
* `ci-storage.protomaps.com/artifacts/ARTIFACT_SHA/smalltestregion_vector.pmtiles`
* `ci-storage.protomaps.com/artifacts/ARTIFACT_SHA/light.json`

`smalltestregion_vector.pmtiles` is the java tiler output at SHA run on `smalltestregion.osm.pbf`.
`light.json` is the generated `layers` of the GL JSON (not the full style).

*Later we will add more than just light.json*

## Test Examples

The file `examples.json` is a JSON array of named examples. Each example consists of:

* a `center` lon,lat
* a `zoom` level
* a `name` that must be a simple slug e.g. `null-island`
* a `description` to explain the cartographic feature under test.
* an array of string `tags` that group examples e.g. `buildings`, `national-parks`

## Test Runner

`index.html` is the single-file test runner, there is no build step. It takes query parameters:

Required query parameters:

* `?left=abc123&right=61`: The Artifact SHA or PR# to display on each side of the comparison.

Optional query parameters:

* `?name=null-island`: run only the named example.
* `?tag=national-parks`: run only one tag.
* `?showDifferencesOnly`: run the tests, but only display where the pixels don't match.

## Versions

The tile archive and named style layers are the only versioned artifacts. Non-versioned parts that affect the test run:

* The current `examples.json`
* The `smalltestregion.osm.pbf` covered areas and snapshot date from OSM.
* The `maplibre-gl-js` version.
* The font glyphs and sprite assets used by the style.
16 changes: 16 additions & 0 deletions ci/examples.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
[
{
"name":"taiwan-z8",
"description":"Compare the appearance of highways",
"tags":["highway", "cjk"],
"center":[121.333,24.320],
"zoom":8
},
{
"name":"taiwan-z16-buildings",
"description":"Compare the appearance of buildings",
"tags":["buildings","cjk"],
"center":[121.4818, 25.0271],
"zoom": 16
}
]
262 changes: 262 additions & 0 deletions ci/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,262 @@
<html>
<head>
<title>Visual Tests | Protomaps Basemaps</title>
<meta charset="utf-8" />
<link
rel="stylesheet"
href="https://unpkg.com/maplibre-gl@2.4.0/dist/maplibre-gl.css"
crossorigin="anonymous"
/>
<script
src="https://unpkg.com/maplibre-gl@2.4.0/dist/maplibre-gl.js"
crossorigin="anonymous"
></script>
<script src="https://unpkg.com/pmtiles@2.4.0/dist/index.js"></script>
<script src="https://unpkg.com/protomaps-themes-base@1.3.0/dist/index.js"></script>
<style>
body {
margin: 0;
background-color: #eee;
font-family: sans-serif;
font-weight:300;
}

#left-map,
#right-map {
height: 500px;
width: 500px;
display: inline-block;
}

canvas {
display: inline;
height: 500px;
width: 500px;
}

#example-results {
margin-left: auto;
margin-right:auto;
width: 1512px;
margin-top:100px;
}

.example-result, .example-result a {
width: 1512px;
color: rgba(0,0,0,0);
}

.example-result:hover, .example-result:hover a {
color: rgba(0.8,0.8,0.8,1);
}

.example-result img {
height: 500px;
width: 500px;
}

.example-result a, span {
margin-right: 0.5rem;
}
</style>
</head>
<body>
<div style="position: absolute; opacity: 0.0; z-index: -1">
<div id="left-map"></div>
<div id="right-map"></div>
<canvas id="left-canvas"></canvas>
<canvas id="right-canvas"></canvas>
<canvas id="diff-canvas"></canvas>
</div>
<div id="example-results"></div>
<script type="module">
import pixelmatch from "https://cdn.skypack.dev/pixelmatch@5.3.0";
let protocol = new pmtiles.Protocol();

maplibregl.setRTLTextPlugin(
"https://unpkg.com/@mapbox/mapbox-gl-rtl-text@0.2.3/mapbox-gl-rtl-text.js"
);
maplibregl.addProtocol("pmtiles", protocol.tile);

const DIM = 500 * window.devicePixelRatio;

document.getElementById("left-canvas").width = DIM;
document.getElementById("left-canvas").height = DIM;
document.getElementById("right-canvas").width = DIM;
document.getElementById("right-canvas").height = DIM;
document.getElementById("diff-canvas").width = DIM;
document.getElementById("diff-canvas").height = DIM;

const left_ctx = document
.getElementById("left-canvas")
.getContext("2d", { willReadFrequently: true });
const right_ctx = document
.getElementById("right-canvas")
.getContext("2d", { willReadFrequently: true });
const diff_canvas = document
.getElementById("diff-canvas");
const diff_ctx = diff_canvas.getContext("2d", { willReadFrequently: true });

const getJson = async (path) => {
let resp = await fetch(path);
return await resp.json();
};

const createMap = (container, url, layers, center, zoom) => {
return new maplibregl.Map({
container: container,
interactive: false,
style: {
version: 8,
center: center,
zoom: zoom,
glyphs: "https://cdn.protomaps.com/fonts/pbf/{fontstack}/{range}.pbf",
sources: {
protomaps: {
type: "vector",
url: "pmtiles://" + url,
attribution: "",
},
},
layers: layers,
},
});
};

const moveAndRenderMap = (map, center, zoom) => {
return new Promise((resolve, reject) => {
map.on("idle", (e) => {
let canvas = map.getCanvas();
resolve(canvas.toDataURL());
});
map.jumpTo({
center: center,
zoom: zoom,
});
});
};

const drawToCanvas = (ctx, data) => {
let img = new Image();
let promise = new Promise((resolve, reject) => {
img.onload = function () {
ctx.drawImage(img, 0, 0);
resolve();
};
});
img.src = data;
return promise;
};

const showExample = async (left_map, right_map, example, showDifferencesOnly) => {
let [left_data, right_data] = await Promise.all([
moveAndRenderMap(left_map, example.center, example.zoom),
moveAndRenderMap(right_map, example.center, example.zoom),
]);

const result = document.createElement('div');
result.className = 'example-result';

const row = document.createElement('div');
const left_img = document.createElement('img');
left_img.src = left_data;
row.appendChild(left_img);
const right_img = document.createElement('img');
right_img.src = right_data;
row.appendChild(right_img);

await drawToCanvas(left_ctx, left_data);
await drawToCanvas(right_ctx, right_data);

const diff = diff_ctx.createImageData(DIM, DIM);
const pixelsDifferent = pixelmatch(
left_ctx.getImageData(0, 0, DIM, DIM).data,
right_ctx.getImageData(0, 0, DIM, DIM).data,
diff.data,
DIM,
DIM,
{ threshold: 0.1 }
);
diff_ctx.putImageData(diff, 0, 0);

const diff_img = document.createElement('img');
diff_img.src = diff_canvas.toDataURL();
row.appendChild(diff_img);
result.appendChild(row);

// the text annotation
const annotation = document.createElement('span');

const nameLink = document.createElement('a');
nameLink.innerText = example.name;
nameLink.href = `?name=${example.name}`;
annotation.appendChild(nameLink);

const description = document.createElement('span');
description.innerText = example.description;
annotation.appendChild(description);

for (const tag of example.tags) {
const tagLink = document.createElement('a');
tagLink.innerText = tag;
tagLink.href = `?tag=${tag}`;
annotation.appendChild(tagLink);
}

result.appendChild(annotation);

if (showDifferencesOnly && pixelsDifferent === 0) {
// do nothing
} else {
document.getElementById("example-results").appendChild(result);
}
};

const queryParams = new URLSearchParams(
window.location.search
);
const left = queryParams.get("left");
const right = queryParams.get("right");
const name = queryParams.get("name");
const tag = queryParams.get("tag");
const showDifferencesOnly = queryParams.get("showDifferencesOnly");

const REMOTE_URL = "https://basemaps-ci.protomaps.com";

// get all JSONs first - we don't want to initialize the map without a starting position
let [examples, left_layers, right_layers] = await Promise.all([
getJson("examples.json"),
getJson(left ? `${REMOTE_URL}/${left}/light.json` : 'light.json'),
getJson(right ? `${REMOTE_URL}/${right}/light.json` : 'light.json'),
]);

if (name !== null) {
examples = examples.filter(e => e.name === name);
} else if (tag !== null) {
examples = examples.filter(e => e.tags.indexOf(tag) >= 0)
}
let example = examples[0];

// create two map instances:
// one for each version, so we don't have to re-initialize the map on changing view.
const left_map = createMap(
"left-map",
left ? `${REMOTE_URL}/${left}/tiles.pmtiles` : 'tiles.pmtiles',
left_layers,
example.center,
example.zoom
);
const right_map = createMap(
"right-map",
right ? `${REMOTE_URL}/${right}/tiles.pmtiles` : 'tiles.pmtiles',
right_layers,
example.center,
example.zoom
);

for (let example of examples) {
await showExample(left_map, right_map, example, showDifferencesOnly);
}
</script>
</body>
</html>
Loading