Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add test for combine tools script #3136

Open
wants to merge 63 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
7bbc0a8
tools test added
vishvamsinh28 Jul 28, 2024
527cf9f
tools test updated for errors
vishvamsinh28 Jul 29, 2024
878023a
tools test upadted
vishvamsinh28 Jul 31, 2024
aedbb7c
Merge branch 'master' into combineToolsTest
vishvamsinh28 Aug 9, 2024
86b914e
Merge branch 'master' into combineToolsTest
vishvamsinh28 Aug 31, 2024
be80885
refactor build-tools
vishvamsinh28 Aug 31, 2024
b04532e
Merge branch 'master' into combineToolsTest
vishvamsinh28 Sep 11, 2024
3ea1852
Merge branch 'master' into combineToolsTest
vishvamsinh28 Sep 20, 2024
1dfd188
87% coverage
vishvamsinh28 Sep 21, 2024
103d643
fewgw
vishvamsinh28 Sep 21, 2024
b3a87dd
fewgw
vishvamsinh28 Sep 21, 2024
8357f56
fewgwfwf
vishvamsinh28 Sep 21, 2024
db96f45
wf
vishvamsinh28 Sep 21, 2024
2b392d1
fwefgwe
vishvamsinh28 Sep 29, 2024
877990e
test update
vishvamsinh28 Sep 29, 2024
88b1491
test update
vishvamsinh28 Sep 29, 2024
dd90f3c
test update
vishvamsinh28 Sep 29, 2024
c347dbf
test update again
vishvamsinh28 Sep 29, 2024
71e4f13
coverage 98%
vishvamsinh28 Sep 29, 2024
03e0664
fefefge
vishvamsinh28 Sep 29, 2024
18e9d98
100% coverage
vishvamsinh28 Sep 29, 2024
b3ae94b
test cases for error added
vishvamsinh28 Sep 30, 2024
014fdb6
fixures
vishvamsinh28 Sep 30, 2024
e74251e
fixture update
vishvamsinh28 Sep 30, 2024
07a7dc3
fixutures again
vishvamsinh28 Sep 30, 2024
511c00f
Merge branch 'master' into combineToolsTest
vishvamsinh28 Sep 30, 2024
91ad534
Merge branch 'combineToolsTest' of https://github.com/vishvamsinh28/w…
vishvamsinh28 Sep 30, 2024
a78dfa1
idjwqf
vishvamsinh28 Sep 30, 2024
8577070
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 4, 2024
57f4dc3
fwqefqe
vishvamsinh28 Oct 4, 2024
0f9052e
tests updated
vishvamsinh28 Oct 4, 2024
a8ab07c
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 7, 2024
a454c8d
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 11, 2024
b7bf6c9
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 13, 2024
f95ecce
buildtools update
vishvamsinh28 Oct 13, 2024
acb1ec3
add empty line at last
vishvamsinh28 Oct 13, 2024
61ac558
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 14, 2024
906a873
Merge branch 'master' into combineToolsTest
anshgoyalevil Oct 15, 2024
5c4c192
update path
vishvamsinh28 Oct 16, 2024
6940c12
Merge branch 'master' into combineToolsTest
anshgoyalevil Oct 17, 2024
aeabab2
wfq
vishvamsinh28 Oct 18, 2024
db33c88
supress logs
vishvamsinh28 Oct 18, 2024
820de27
test and function update
vishvamsinh28 Oct 18, 2024
b16fca9
commit to re run the workflow
vishvamsinh28 Oct 18, 2024
995b9bd
commit to re run the workflow again
vishvamsinh28 Oct 18, 2024
4a4dcb7
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 19, 2024
bd6424d
added retry mechanicasm for enoent errors
vishvamsinh28 Oct 19, 2024
233efdd
use fsextra
vishvamsinh28 Oct 19, 2024
4b13fc8
update test
vishvamsinh28 Oct 19, 2024
0489f2d
fqewfewqf
vishvamsinh28 Oct 19, 2024
774da3f
fqewfewqf
vishvamsinh28 Oct 19, 2024
7c24517
fqwefge
vishvamsinh28 Oct 19, 2024
7279d79
commit to re run the workflow
vishvamsinh28 Oct 19, 2024
2eccf52
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 20, 2024
7601440
test upadted
vishvamsinh28 Oct 21, 2024
9afb39b
Merge branch 'combineToolsTest' of https://github.com/vishvamsinh28/w…
vishvamsinh28 Oct 21, 2024
464b1dc
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 27, 2024
496de09
Merge branch 'master' into combineToolsTest
vishvamsinh28 Nov 1, 2024
2419f06
Merge branch 'master' into combineToolsTest
vishvamsinh28 Nov 4, 2024
11fcfe6
test updated
vishvamsinh28 Nov 5, 2024
7536a96
add empty line at the end
vishvamsinh28 Nov 5, 2024
5164452
fege
vishvamsinh28 Nov 5, 2024
ca72a7f
fqefe
vishvamsinh28 Nov 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion scripts/build-tools.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ const fs = require('fs');
const { resolve } = require('path');

const buildTools = async (automatedToolsPath, manualToolsPath, toolsPath, tagsPath) => {

try {
let githubExtractData = await getData();
let automatedTools = await convertTools(githubExtractData);
Expand All @@ -13,8 +14,9 @@ const buildTools = async (automatedToolsPath, manualToolsPath, toolsPath, tagsPa
automatedToolsPath,
JSON.stringify(automatedTools, null, ' ')
);

await combineTools(automatedTools, require(manualToolsPath), toolsPath, tagsPath);

} catch (err) {
throw new Error(`An error occurred while building tools: ${err.message}`);
}
Expand Down
56 changes: 30 additions & 26 deletions scripts/tools/combine-tools.js
Original file line number Diff line number Diff line change
Expand Up @@ -106,37 +106,41 @@ const getFinalTool = async (toolObject) => {
// Combine the automated tools and manual tools list into single JSON object file, and
// lists down all the language and technology tags in one JSON file.
const combineTools = async (automatedTools, manualTools, toolsPath, tagsPath) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does tagspath signify??

for (const key in automatedTools) {
let finalToolsList = [];
if (automatedTools[key].toolsList.length) {
for (const tool of automatedTools[key].toolsList) {
finalToolsList.push(await getFinalTool(tool))
try {
for (const key in automatedTools) {
let finalToolsList = [];
if (automatedTools[key].toolsList.length) {
for (const tool of automatedTools[key].toolsList) {
finalToolsList.push(await getFinalTool(tool))
}
}
}
if (manualTools[key] && manualTools[key].toolsList.length) {
for (const tool of manualTools[key].toolsList) {
let isAsyncAPIrepo;
const isValid = await validate(tool)
if (isValid) {
if (tool?.links?.repoUrl) {
const url = new URL(tool.links.repoUrl)
isAsyncAPIrepo = url.href.startsWith("https://github.com/asyncapi/")
} else isAsyncAPIrepo = false
let toolObject = await createToolObject(tool, "", "", isAsyncAPIrepo)
finalToolsList.push(await getFinalTool(toolObject))
} else {
console.error('Script is not failing, it is just dropping errors for further investigation');
console.error(`Invalid ${tool.title} .asyncapi-tool file.`);
console.error(`Located in manual-tools.json file`);
console.error('Validation errors:', JSON.stringify(validate.errors, null, 2));
if (manualTools[key] && manualTools[key].toolsList.length) {
for (const tool of manualTools[key].toolsList) {
Copy link

@coderabbitai coderabbitai bot Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Use optional chaining for better null safety

The static analysis tool correctly suggests using optional chaining here.

-if (manualTools[key] && manualTools[key].toolsList.length) {
+if (manualTools[key]?.toolsList?.length) {
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (manualTools[key] && manualTools[key].toolsList.length) {
for (const tool of manualTools[key].toolsList) {
if (manualTools[key]?.toolsList?.length) {
for (const tool of manualTools[key].toolsList) {
🧰 Tools
🪛 Biome

[error] 117-117: Change to an optional chain.

Unsafe fix: Change to an optional chain.

(lint/complexity/useOptionalChain)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vishvamsinh28 Kindly apply this suggestion

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!

let isAsyncAPIrepo;
const isValid = await validate(tool)
if (isValid) {
if (tool?.links?.repoUrl) {
const url = new URL(tool.links.repoUrl)
isAsyncAPIrepo = url.href.startsWith("https://github.com/asyncapi/")
} else isAsyncAPIrepo = false
Comment on lines +122 to +125
Copy link

@coderabbitai coderabbitai bot Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Enhance URL validation robustness

The URL validation could be more robust by handling potential URL parsing errors.

 if (tool?.links?.repoUrl) {
-  const url = new URL(tool.links.repoUrl)
-  isAsyncAPIrepo = url.href.startsWith("https://github.com/asyncapi/")
+  try {
+    const url = new URL(tool.links.repoUrl)
+    isAsyncAPIrepo = url.href.startsWith("https://github.com/asyncapi/")
+  } catch (urlError) {
+    console.error(`Invalid repository URL for tool ${tool.title}: ${tool.links.repoUrl}`);
+    isAsyncAPIrepo = false;
+  }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (tool?.links?.repoUrl) {
const url = new URL(tool.links.repoUrl)
isAsyncAPIrepo = url.href.startsWith("https://github.com/asyncapi/")
} else isAsyncAPIrepo = false
if (tool?.links?.repoUrl) {
try {
const url = new URL(tool.links.repoUrl)
isAsyncAPIrepo = url.href.startsWith("https://github.com/asyncapi/")
} catch (urlError) {
console.error(`Invalid repository URL for tool ${tool.title}: ${tool.links.repoUrl}`);
isAsyncAPIrepo = false;
}
} else isAsyncAPIrepo = false

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't suggest this. It is not an error.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akshatnema Understood, thank you for the clarification.


✏️ Learnings added
Learnt from: akshatnema
PR: asyncapi/website#3136
File: scripts/tools/combine-tools.js:122-125
Timestamp: 2024-11-01T12:49:32.625Z
Learning: In `scripts/tools/combine-tools.js`, the existing URL parsing logic for `repoUrl` without additional error handling is acceptable and does not require changes.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

let toolObject = await createToolObject(tool, "", "", isAsyncAPIrepo)
finalToolsList.push(await getFinalTool(toolObject))
} else {
console.error('Script is not failing, it is just dropping errors for further investigation');
console.error(`Invalid ${tool.title} .asyncapi-tool file.`);
console.error(`Located in manual-tools.json file`);
console.error('Validation errors:', JSON.stringify(validate.errors, null, 2));
}
Comment on lines +129 to +133
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Improve validation error logging structure

The current error logging could be more structured and consistent.

-console.error('Script is not failing, it is just dropping errors for further investigation');
-console.error(`Invalid ${tool.title} .asyncapi-tool file.`);
-console.error(`Located in manual-tools.json file`);
-console.error('Validation errors:', JSON.stringify(validate.errors, null, 2));
+console.error({
+  message: 'Tool validation failed',
+  tool: tool.title,
+  source: 'manual-tools.json',
+  errors: validate.errors,
+  note: 'Script continues execution, error logged for investigation'
+});
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
console.error('Script is not failing, it is just dropping errors for further investigation');
console.error(`Invalid ${tool.title} .asyncapi-tool file.`);
console.error(`Located in manual-tools.json file`);
console.error('Validation errors:', JSON.stringify(validate.errors, null, 2));
}
console.error({
message: 'Tool validation failed',
tool: tool.title,
source: 'manual-tools.json',
errors: validate.errors,
note: 'Script continues execution, error logged for investigation'
});

}
}
finalToolsList.sort((tool, anotherTool) => tool.title.localeCompare(anotherTool.title));
finalTools[key].toolsList = finalToolsList
}
finalToolsList.sort((tool, anotherTool) => tool.title.localeCompare(anotherTool.title));
finalTools[key].toolsList = finalToolsList
fs.writeFileSync(toolsPath, JSON.stringify(finalTools));
fs.writeFileSync(tagsPath, JSON.stringify({ languages: languageList, technologies: technologyList }),)
Comment on lines +139 to +140
Copy link

@coderabbitai coderabbitai bot Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Implement safer file operations

The file writing operations should be more robust with atomic writes to prevent partial updates.

-fs.writeFileSync(toolsPath, JSON.stringify(finalTools));
-fs.writeFileSync(tagsPath, JSON.stringify({ languages: languageList, technologies: technologyList }),);
+const writeFileSafely = (path, content) => {
+  const tempPath = `${path}.tmp`;
+  fs.writeFileSync(tempPath, JSON.stringify(content, null, 2));
+  fs.renameSync(tempPath, path);
+};
+
+writeFileSafely(toolsPath, finalTools);
+writeFileSafely(tagsPath, { 
+  languages: languageList, 
+  technologies: technologyList 
+});
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
fs.writeFileSync(toolsPath, JSON.stringify(finalTools));
fs.writeFileSync(tagsPath, JSON.stringify({ languages: languageList, technologies: technologyList }),)
const writeFileSafely = (path, content) => {
const tempPath = `${path}.tmp`;
fs.writeFileSync(tempPath, JSON.stringify(content, null, 2));
fs.renameSync(tempPath, path);
};
writeFileSafely(toolsPath, finalTools);
writeFileSafely(tagsPath, {
languages: languageList,
technologies: technologyList
});

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does it preventing partial updates?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akshatnema, by writing the data to a temporary file first and then renaming it to the desired file, we prevent partial updates. If an error occurs or the process is interrupted during the write operation, the original file remains unchanged because the temporary file hasn't replaced it yet. The fs.renameSync operation is atomic on most file systems, ensuring that the target file is only updated if the write to the temporary file completes successfully. This approach helps maintain file integrity by avoiding situations where a file could be left incomplete or corrupted due to a failed write operation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the target file already exists, can there be possibility that a duplicate file is created with another name and the previous one is not removed?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akshatnema, you make a good point. If the process is interrupted after writing the temporary file but before the fs.renameSync operation, there's a possibility that the temporary file (${path}.tmp) remains, and the original file isn't updated. This could lead to leftover temporary files in the directory.

Additionally, if multiple instances of the script run simultaneously, they might use the same temporary file name, causing conflicts. To mitigate these issues, we can:

  • Use unique temporary file names: Incorporate a unique identifier in the temp file name, such as a timestamp or a random string.

    const writeFileSafely = (path, content) => {
      const tempPath = `${path}.${Date.now()}.tmp`;
      fs.writeFileSync(tempPath, JSON.stringify(content, null, 2));
      fs.renameSync(tempPath, path);
    };
  • Handle cleanup of temporary files: Implement a mechanism to clean up any leftover temp files on startup or before writing new files.

  • Use atomic write libraries: Consider using a library like write-file-atomic which handles these edge cases and ensures safer writes.

    const writeFileAtomic = require('write-file-atomic');
    
    const writeFileSafely = (path, content) => {
      writeFileAtomic.sync(path, JSON.stringify(content, null, 2));
    };

These approaches help prevent the possibility of duplicate files and ensure that the original file is safely updated. Let me know your thoughts on this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can there be a possibility that even scripts gets executed successfully but the temporary file is not able to replace the target file?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akshatnema, yes, there is a possibility that even if the script executes successfully, the temporary file may not be able to replace the target file. This could happen due to various reasons such as file system permissions, file locks, insufficient disk space, or other OS-level issues that prevent the fs.renameSync operation from completing successfully.

To mitigate this, we can:

  • Add Error Handling: Wrap the fs.renameSync operation in a try-catch block to handle any exceptions that might occur during the rename process.

    const writeFileSafely = (path, content) => {
      const tempPath = `${path}.tmp`;
      fs.writeFileSync(tempPath, JSON.stringify(content, null, 2));
      try {
        fs.renameSync(tempPath, path);
      } catch (error) {
        console.error(`Failed to rename temporary file: ${error.message}`);
        // Handle the error accordingly, possibly retry or abort
      }
    };
  • Check File System Constraints: Ensure that the script has the necessary permissions and that there are no locks on the target file.

  • Evaluate Necessity: If the risk of partial writes or corrupted files is minimal in our context, and the current implementation hasn't caused issues, we might opt to keep the existing code for simplicity.

Let me know your thoughts on whether you'd like to proceed with additional error handling or prefer to retain the current implementation.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!

} catch (err) {
throw new Error(`Error combining tools: ${err}`);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider enhancing error handling with structured errors

While the try-catch block is a good addition, consider creating specific error types for different failure scenarios (e.g., validation errors, file system errors) to help with error tracking and debugging.

+class ToolsCombineError extends Error {
+  constructor(operation, details, cause) {
+    super(`Failed to ${operation}: ${details}`);
+    this.name = 'ToolsCombineError';
+    this.cause = cause;
+  }
+}

 try {
   // ... existing code ...
 } catch (err) {
-  throw new Error(`Error combining tools: ${err}`);
+  throw new ToolsCombineError('combine tools', 
+    err.message, 
+    err);
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
try {
for (const key in automatedTools) {
let finalToolsList = [];
if (automatedTools[key].toolsList.length) {
for (const tool of automatedTools[key].toolsList) {
finalToolsList.push(await getFinalTool(tool))
}
}
}
if (manualTools[key] && manualTools[key].toolsList.length) {
for (const tool of manualTools[key].toolsList) {
let isAsyncAPIrepo;
const isValid = await validate(tool)
if (isValid) {
if (tool?.links?.repoUrl) {
const url = new URL(tool.links.repoUrl)
isAsyncAPIrepo = url.href.startsWith("https://github.com/asyncapi/")
} else isAsyncAPIrepo = false
let toolObject = await createToolObject(tool, "", "", isAsyncAPIrepo)
finalToolsList.push(await getFinalTool(toolObject))
} else {
console.error('Script is not failing, it is just dropping errors for further investigation');
console.error(`Invalid ${tool.title} .asyncapi-tool file.`);
console.error(`Located in manual-tools.json file`);
console.error('Validation errors:', JSON.stringify(validate.errors, null, 2));
if (manualTools[key] && manualTools[key].toolsList.length) {
for (const tool of manualTools[key].toolsList) {
let isAsyncAPIrepo;
const isValid = await validate(tool)
if (isValid) {
if (tool?.links?.repoUrl) {
const url = new URL(tool.links.repoUrl)
isAsyncAPIrepo = url.href.startsWith("https://github.com/asyncapi/")
} else isAsyncAPIrepo = false
let toolObject = await createToolObject(tool, "", "", isAsyncAPIrepo)
finalToolsList.push(await getFinalTool(toolObject))
} else {
console.error('Script is not failing, it is just dropping errors for further investigation');
console.error(`Invalid ${tool.title} .asyncapi-tool file.`);
console.error(`Located in manual-tools.json file`);
console.error('Validation errors:', JSON.stringify(validate.errors, null, 2));
}
}
}
finalToolsList.sort((tool, anotherTool) => tool.title.localeCompare(anotherTool.title));
finalTools[key].toolsList = finalToolsList
}
finalToolsList.sort((tool, anotherTool) => tool.title.localeCompare(anotherTool.title));
finalTools[key].toolsList = finalToolsList
fs.writeFileSync(toolsPath, JSON.stringify(finalTools));
fs.writeFileSync(tagsPath, JSON.stringify({ languages: languageList, technologies: technologyList }),)
} catch (err) {
throw new Error(`Error combining tools: ${err}`);
class ToolsCombineError extends Error {
constructor(operation, details, cause) {
super(`Failed to ${operation}: ${details}`);
this.name = 'ToolsCombineError';
this.cause = cause;
}
}
try {
for (const key in automatedTools) {
let finalToolsList = [];
if (automatedTools[key].toolsList.length) {
for (const tool of automatedTools[key].toolsList) {
finalToolsList.push(await getFinalTool(tool))
}
}
if (manualTools[key] && manualTools[key].toolsList.length) {
for (const tool of manualTools[key].toolsList) {
let isAsyncAPIrepo;
const isValid = await validate(tool)
if (isValid) {
if (tool?.links?.repoUrl) {
const url = new URL(tool.links.repoUrl)
isAsyncAPIrepo = url.href.startsWith("https://github.com/asyncapi/")
} else isAsyncAPIrepo = false
let toolObject = await createToolObject(tool, "", "", isAsyncAPIrepo)
finalToolsList.push(await getFinalTool(toolObject))
} else {
console.error('Script is not failing, it is just dropping errors for further investigation');
console.error(`Invalid ${tool.title} .asyncapi-tool file.`);
console.error(`Located in manual-tools.json file`);
console.error('Validation errors:', JSON.stringify(validate.errors, null, 2));
}
}
}
finalToolsList.sort((tool, anotherTool) => tool.title.localeCompare(anotherTool.title));
finalTools[key].toolsList = finalToolsList
}
fs.writeFileSync(toolsPath, JSON.stringify(finalTools));
fs.writeFileSync(tagsPath, JSON.stringify({ languages: languageList, technologies: technologyList }),)
} catch (err) {
throw new ToolsCombineError('combine tools',
err.message,
err);
🧰 Tools
🪛 Biome

[error] 117-117: Change to an optional chain.

Unsafe fix: Change to an optional chain.

(lint/complexity/useOptionalChain)

}
fs.writeFileSync(toolsPath,JSON.stringify(finalTools));
fs.writeFileSync(tagsPath,JSON.stringify({ languages: languageList, technologies: technologyList }),)
}

module.exports = { combineTools }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix this linter error

212 changes: 212 additions & 0 deletions tests/fixtures/combineToolsData.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
const expectedDataT1 = {
languages: [
{
name: 'JavaScript',
color: 'bg-[#57f281]',
borderColor: 'border-[#37f069]'
},
{
name: 'Python',
color: 'bg-[#3572A5]',
borderColor: 'border-[#3572A5]'
}
],
Comment on lines +2 to +13
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we directly use tagsData instead of creating this duplicate data?

technologies: [
{
name: 'Node.js',
color: 'bg-[#61d0f2]',
borderColor: 'border-[#40ccf7]'
},
{
name: 'Flask',
color: 'bg-[#000000]',
borderColor: 'border-[#FFFFFF]'
}
]
};
Comment on lines +1 to +26
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider reusing tagsData instead of duplicating data.

The color and border color data for languages and technologies appears to be duplicated from tagsData. To maintain consistency and reduce maintenance overhead, consider importing and reusing the existing data.

Example approach:

const { tagsData } = require('../path/to/tagsData');

const expectedDataT1 = {
    languages: tagsData.languages.map(({ name, color, borderColor }) => ({
        name,
        color,
        borderColor
    })).filter(lang => ['JavaScript', 'Python'].includes(lang.name)),
    technologies: tagsData.technologies.map(({ name, color, borderColor }) => ({
        name,
        color,
        borderColor
    })).filter(tech => ['Node.js', 'Flask'].includes(tech.name))
};
🧰 Tools
🪛 eslint

[error] 2-2: Delete ··

(prettier/prettier)


[error] 3-3: Delete ····

(prettier/prettier)


[error] 4-4: Replace ············ with ······

(prettier/prettier)


[error] 5-5: Replace ············ with ······

(prettier/prettier)


[error] 6-6: Delete ······

(prettier/prettier)


[error] 7-7: Replace ········ with ····

(prettier/prettier)


[error] 8-8: Replace ········ with ····

(prettier/prettier)


[error] 9-9: Replace ············ with ······

(prettier/prettier)


[error] 10-10: Replace ············ with ······

(prettier/prettier)


[error] 11-11: Delete ······

(prettier/prettier)


[error] 12-12: Delete ····

(prettier/prettier)


[error] 13-13: Delete ··

(prettier/prettier)


[error] 14-14: Delete ··

(prettier/prettier)


[error] 15-15: Delete ····

(prettier/prettier)


[error] 16-16: Replace ············ with ······

(prettier/prettier)


[error] 17-17: Delete ······

(prettier/prettier)


[error] 18-18: Replace ············ with ······

(prettier/prettier)


[error] 19-19: Replace ········ with ····

(prettier/prettier)


[error] 20-20: Replace ········ with ····

(prettier/prettier)


[error] 21-21: Replace ············ with ······

(prettier/prettier)


[error] 22-22: Replace ············ with ······

(prettier/prettier)


[error] 23-23: Delete ······

(prettier/prettier)


[error] 24-24: Replace ········ with ····

(prettier/prettier)


[error] 25-25: Delete ··

(prettier/prettier)


const manualToolsWithMissingData = [
{
title: 'Tool C',
filters: {},
links: { repoUrl: 'https://github.com/asyncapi/tool-c' }
}
];

const manualToolsToSort = {
category1: {
description: 'Sample Category',
toolsList: [
{
title: 'Tool Z',
filters: { language: 'JavaScript' },
links: { repoUrl: 'https://github.com/asyncapi/tool-z' }
},
{
title: 'Tool A',
filters: { language: 'Python' },
links: { repoUrl: 'https://github.com/asyncapi/tool-a' }
}
]
}
};

const toolWithMultipleLanguages = {
title: 'Multi-Language Tool',
filters: {
language: ['JavaScript', 'Python', 'NewLanguage'],
technology: ['Node.js']
},
links: { repoUrl: 'https://github.com/example/multi-language-tool' }
};

const automatedToolsT5 = {
'category1': {
description: 'Category 1 Description',
toolsList: [toolWithMultipleLanguages]
}
};

const invalidToolT4 = { title: 'Invalid Tool' };

const automatedToolsT4 = {
'category1': {
description: 'Category 1 Description',
toolsList: []
}
};
const manualToolsT4 = {
'category1': {
toolsList: [invalidToolT4]
}
};

const toolWithNewTagsT6 = {
title: 'New Tags Tool',
filters: {
language: 'NewLanguage',
technology: ['NewTechnology']
},
links: { repoUrl: 'https://github.com/example/new-tags-tool' }
};

const automatedToolsT6 = {
'category1': {
description: 'Category 1 Description',
toolsList: [toolWithNewTagsT6]
}
};

const toolWithNewLanguageT7 = {
title: 'New Language Tool',
filters: {
language: 'Go',
technology: ['Node.js']
},
links: { repoUrl: 'https://github.com/example/new-language-tool' }
};

const automatedToolsT7 = {
'category1': {
description: 'Category 1 Description',
toolsList: [toolWithNewLanguageT7]
}
};

const validToolT8 = {
title: 'Valid Tool',
filters: {
language: 'JavaScript',
technology: ['Node.js']
},
links: { repoUrl: 'https://github.com/asyncapi/valid-tool' }
};

const automatedToolsT8 = {
category1: {
description: 'Category 1 Description',
toolsList: []
}
};

const manualToolsT8 = {
category1: {
toolsList: [validToolT8]
}
};

const toolWithoutRepoUrlT9 = {
title: 'Tool Without Repo',
filters: {
language: 'Python',
technology: ['Flask']
},
links: {}
};

const automatedToolsT9 = {
category1: {
description: 'Category 1 Description',
toolsList: []
}
};

const manualToolsT9 = {
category1: {
toolsList: [toolWithoutRepoUrlT9]
}
};

const invalidAutomatedToolsT10 = {
invalidCategory: {
description: 'Invalid Category Description',
toolsList: []
}
};

const manualToolsWithInvalidURLT11 = {
category1: {
toolsList: [
{
title: 'Tool with Invalid URL',
filters: { language: 'JavaScript' },
links: { repoUrl: 'invalid-url' }
}
]
}
};

const circularTool = {
title: 'Circular Tool',
filters: {
language: 'JavaScript',
technology: ['Node.js']
},
links: { repoUrl: 'https://github.com/asyncapi/circular-tool' }
};

const automatedToolsT12 = {
category1: {
description: 'Category 1',
toolsList: [circularTool]
}
};

module.exports = {
expectedDataT1,
manualToolsWithMissingData,
manualToolsToSort,
automatedToolsT5,
automatedToolsT4,
manualToolsT4,
automatedToolsT6,
automatedToolsT7,
automatedToolsT8,
manualToolsT8,
automatedToolsT9,
manualToolsT9,
circularTool,
automatedToolsT12,
invalidAutomatedToolsT10,
manualToolsWithInvalidURLT11
}
17 changes: 17 additions & 0 deletions tests/fixtures/tools/automated-tools.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"category1": {
"description": "Sample Category",
"toolsList": [
{
"title": "Tool B",
"filters": {
"language": "Python",
"technology": ["Flask"]
Comment on lines +8 to +9
Copy link

@coderabbitai coderabbitai bot Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider using consistent array format for both language and technology.

For better consistency and future extensibility, consider making the language field an array like technology. This would allow tools to support multiple programming languages.

         "filters": {
-          "language": "Python",
+          "language": ["Python"],
           "technology": ["Flask"]
         },

Committable suggestion skipped: line range outside the PR's diff.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

language can take values in terms of both string and array.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akshatnema, understood. Thank you for the clarification.


✏️ Learnings added
Learnt from: akshatnema
PR: asyncapi/website#3136
File: tests/fixtures/tools/automated-tools.json:8-9
Timestamp: 2024-11-01T12:48:21.851Z
Learning: In the `tests/fixtures/tools/automated-tools.json` file, the `language` field within the `filters` object can be either a string or an array. Both formats are acceptable and supported in the codebase.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

},
"links": {
"repoUrl": "https://github.com/asyncapi/tool-b"
}
}
]
}
}
12 changes: 12 additions & 0 deletions tests/fixtures/tools/manual-tools.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[
{
"title": "Tool A",
"filters": {
"language": "JavaScript",
"technology": ["Node.js"]
},
"links": {
"repoUrl": "https://github.com/asyncapi/tool-a"
}
}
]
Loading
Loading