Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sanitize externalReferences URLs and exclude invalid ones + Strict JSON Schema validation #1130

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
+280 −9
Open

Sanitize externalReferences URLs and exclude invalid ones + Strict JSON Schema validation #1130

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f84aa82
Enable JSON Schema strict validation
marob May 30, 2024
40c8685
Add URL validation and URL conversion logic
timmyteo May 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ import {
getSwiftPackageMetadata,
getTimestamp,
includeMavenTestScope,
isValidIriReference,
parseBazelActionGraph,
parseBazelSkyframe,
parseBdistMetadata,
Expand Down Expand Up @@ -129,6 +130,7 @@ import {
parseSwiftResolved,
parseYarnLock,
readZipEntry,
sanitizeUrlAsIri,
splitOutputByGradleProjects,
} from "./utils.js";
let url = import.meta.url;
Expand Down Expand Up @@ -726,7 +728,9 @@ function addExternalReferences(opkg) {
}
}
}
return externalReferences;
return externalReferences
.map((ref) => ({ ...ref, url: sanitizeUrlAsIri(ref.url) }))
.filter((ref) => isValidIriReference(ref.url));
}

/**
Expand Down
101 changes: 101 additions & 0 deletions package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@
"@npmcli/arborist": "7.5.2",
"ajv": "^8.14.0",
"ajv-formats": "^3.0.1",
"ajv-formats-draft2019": "^1.6.1",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am having better success with validate-iri package.

import { validateIri, IriValidationStrategy } from 'validate-iri'
let yourIri = 'git@gitlab.com:behat-chrome/chrome-mink-driver.git'
console.log(validateIri(yourIri, IriValidationStrategy.Strict));

yourIri = '${repository.url}';
console.log(validateIri(yourIri, IriValidationStrategy.Strict));

yourIri = 'git+https://github.com/Alex-D/check-disk-space.git';
console.log(validateIri(yourIri, IriValidationStrategy.Strict));

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was looking for some validation library but didn't find this one 👍

"cheerio": "^1.0.0-rc.12",
"edn-data": "1.1.1",
"find-up": "7.0.0",
Expand All @@ -90,7 +91,8 @@
"tar": "^6.2.1",
"uuid": "^9.0.1",
"xml-js": "^1.6.11",
"yargs": "^17.7.2"
"yargs": "^17.7.2",
"hosted-git-info": "^7.0.2"
Copy link
Collaborator

@prabhu prabhu Jun 1, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not introduce this package and change the url format in this PR. We can drop any non-compliant urls, maybe with a log in DEBUG_MODE.

},
"optionalDependencies": {
"@appthreat/atom": "2.0.12",
Expand Down
2 changes: 1 addition & 1 deletion types/index.d.ts.map
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

31 changes: 31 additions & 0 deletions types/utils.d.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1119,6 +1119,37 @@ export function addEvidenceForDotnet(pkgList: any, slicesFile: any): any;
* @returns {Object} pkgFilesMap Object with package name and list of files
*/
export function parseMakeDFile(dfile: string): any;
/**
* Function to validate an externalReference URL for conforming to the JSON schema or bomLink
* https://github.com/CycloneDX/cyclonedx-core-java/blob/75575318b268dda9e2a290761d7db11b4f414255/src/main/resources/bom-1.5.schema.json#L1140
* https://datatracker.ietf.org/doc/html/rfc3987#section-2.2
* https://cyclonedx.org/capabilities/bomlink/
*
* @param {String} url URL to validate
*
* @returns {Boolean} Flag indicating whether the supplied URL is valid or not
*
*/
export function isValidIriReference(url: string): boolean;
/**
* Function to convert a Git repository URL to its HTTP URL counterpart
* Example: git@github.com:npm/hosted-git-info.git converts to https://github.com/npm/hosted-git-info.git
*
* @param {String} url URL to convert
*
* @returns {String} Converted URL, or original URL if conversion was not possible
*
*/
export function convertGitRepoUrlToHttpUrl(url: string): string;
/**
* Function that sanitizes an URL to try to make it conform to IRI RFC 3987.
*
* @param {String} url URL to sanitize
*
* @returns {String} Sanitized URL
*
*/
export function sanitizeUrlAsIri(url: string): string;
export const dirNameStr: string;
export const isWin: boolean;
export const isMac: boolean;
Expand Down
Loading
Loading