Disallowed hostnames configuration ignored by @percy/core

[agarzola]

The problem

We are using @percy/core within a custom Netlify plugin to kick off a new Percy build after each successful deployment (both for production deployments after a merge to main and on preview deployments for each PR). In all builds, we want to block certain third-party libraries (GTM, for example) from loading at all. In order to test whether GTM is being loaded or not, I created a temporary JS function which prints a large banner at the top of any page as soon as the GTM script tag has finished loading. No matter where we set the disallowed hostnames, the banner is always printed to the page, indicating that the library was indeed loaded.

We have tried configuring the disallowed hostnames option both directly in a disallowedHostnames array included in the object passed to the Percy() constructor in our code and —after reviewing the @percy/core docs, which seem to imply that certain options can only be configured via a config file— in a disallowed-hostnames array set in a .percy.yml file. However, everything we’ve tried has resulted in Percy seemingly ignoring this directive and loading GTM on the page regardless.

Environment

• Node version: v16 (latest)
• @percy/core version: v1.14.0
• Version of Percy SDK you’re using:
• If needed, a build or snapshot ID: https://percy.io/chromatic/chromatichq.com/builds/22915751/changed/1279130991
• OS version:
• Type of shell command-line [interface]:

Debug logs

Logs made this issue too long for GitHub, so I put them in this gist: https://gist.github.com/agarzola/f009e64e7c55fdfea576789680b1e704

Code to reproduce issue

HTML snippet near the top of every page’s HEAD element

<script async onload="gtagload()" src="https://www.googletagmanager.com/gtag/js?id=UA-REDACTED"></script>
<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());
  gtag('config', 'UA-REDACTED');
  // Create a large banner and place it at the top of the page after
  // GTM has loaded. This should result in no banner if GTM is blocked.
  function gtagload() {
    const div = document.createElement('div');
    div.style = 'background-color: #f3f2ef; color: black; font-size: 3rem; text-align: center;';
    div.innerText = 'GTM has finished loading.';
    document.body.prepend(div);
  };
</script>

Custom Netlify plugin

import Percy from '@percy/core';
import { readFileSync } from 'fs';
import YAML from 'yaml';

const paths = YAML.parse(readFileSync('./snapshots.yml', 'utf8'));

export const onSuccess = async function () {
  // Get build URL from environment.
  const baseUrl = process.env.DEPLOY_PRIME_URL;
  const clientInfo = 'netlify-plugin-visual-regression-testing';
  const environmentInfo = process.env.BRANCH;
  const loglevel = 'debug';
  // Percy will automagically detect this is a Netlify build and capture token
  // and branch information from the environment.
  const percy = new Percy({
    clientInfo,
    environmentInfo,
    loglevel,
  });
  // Start the Percy instance.
  await percy.start();
  // Fire off snapshots in sequence, passing in the `?reduce-motion` query
  // string to omit animations that can cause false positives.
  paths.forEach(async (path) => {
    await percy.snapshot({
      url: `${baseUrl}${path}?reduce-motion`,
      execute: loadLazyImages,
    });
  });
  // Wait until no more snapshots are processing.
  await percy.idle();
  // Stop Percy instance.
  await percy.stop();
};

/**
 * Load lazy-loaded images by switching them to be eager-loaded.
 *
 * Note: this function is passed to Percy to run in the context of the browser
 * before taking a snapshot. Hence the use of `document`.
 */
async function loadLazyImages() {
  // Find all lazy-loaded images that have not been loaded yet.
  const images = Array.from(document.querySelectorAll('img[loading="lazy"]'))
    .filter((image) => !image.complete || !image.naturalHeight);
  // Create new array of promises, each of which will resolve when its
  // corresponding image has finished loading.
  const promises = images.map((image) => new Promise((resolve, reject) => {
    image.addEventListener('load', resolve);
    image.loading = 'eager';
  }));
  // Wait for all image loading promises have resolved.
  await Promise.all(promises);
}

.percy.yml file

---
version: 2
discovery:
  disallowed-hostnames:
    - app.netlify.com
    - cdn.segment.com
    - forms.hsforms.com
    - js.hs-scripts.com
    - js.hsforms.net
    - api.segment.io
    - netlify-cdp-loader.netlify.app
    - px.ads.linkedin.com
    - track.hubspot.com
    - www.google-analytics.com
    - www.googletagmanager.com
  launch-options:
    # Increase timeout, as sometimes Netlify takes over 30s to respond.
    timeout: 60000,
snapshot:
  # Take snapshots with JavaScript enabled.
  enable-javascript: true

[itsjwala]

It seems there is a misunderstanding with the disallowed-hostnames option. All options nested under the discovery section are specific to local asset discovery. So disallowed-hostnames will only disallow hostnames from being captured by asset discovery. Their snapshot will still be re-rendered in our infrastructure when screenshots are taken, and assets that aren't captured by asset discovery will still be requested from within our browsers.

We re-render the snapshot as close to the original captured DOM as possible (after freezing animations and such). So if they wish to prevent network requests from happening completely, they'll need to do so from their own code or by removing those specific elements before the DOM snapshot is taken.

If you'd like to know more about the SDK, you can check a short breakdown at the top of this page: https://docs.percy.io/docs/debugging-sdks

[agarzola]

Thank you for this succinct, yet clear explanation, @itsjwala.

[agarzola]

@itsjwala Can I make a suggestion? The link doesn’t work because it was pasted twice consecutively. For the benefit of folks who might run across your answer in the future, can you fix the link?

Thanks again!

[itsjwala]

oops, updated!

[davew9999]

Sorry to resurrect an old post, but it seems relevant. I also thought disallowedHostnames would achieve the same as @agarzola thought. I think the documentation doesn't make this clear. The issue we were getting was timeouts requesting some resources, this FAQ article looks to describe our issue exactly.

However in this doc it says "We skip assets that are hosted on a different domain from what the snapshot came from." and so the example in the FAQ doesn't make sense, as it uses a different domain as an example.

It's also not clear what asset discovery actually does? The above doc is entitled "Screenshot rendering and asset discovery" and it doesn't seem to say what asset discovery is. It does say the below, which makes out that the asset discovery is part of the rendering.

...it captures DOM snapshots and page assets (like CSS and images). Percy handles all reconstruction and rendering on a dedicated infrastructure designed for scalable visual testing.

It looks like what we actually want is to remove the resource from the DOM.

Perhaps the docs could be improved to clarify all of this?