Add tabletojson dependency and implement HTMLTablesToJSON function

[pelikhan]

• The codebase has seen a big change where a functionality to convert HTML tables to JSON is implemented. 🔀🏗️

  • The implementation uses tabletojson library that is added in packages/core/package.json.
  • The function HTMLTablesToJSON doing the conversion is added into html.ts.
  • The tests for this function are added in html.test.ts.
  • The newly implemented function was also added to global HTML object in globals.ts.
  • As such, the HTML-JSON converter functionality is user-facing as it's part of the public API listed in prompt_template.d.ts.

• There are modifications around tracing and debugging, which improves coverage and quality of logs. 📝🔍

  • In promptrunner.ts, the model is now part of the heading used in tracing.

• The way the data is defined and stored is changed.

  • The function createDefDataNode is renamed to createDefData in promptdom.ts and runpromptcontext.ts.

• A minor change is made to the sample code in browse-text.genai.mts. Now, the code converts an HTML table to JSON using the newly implemented converter and removes the first column from the result before defining "DATA".

• A part of the code where verbose logs are made is removed reducing noise in the output. This happens in playwright.ts where certain log statements are removed providing a cleaner output for users. 🗑️ 🎯

• Minor formatting and typo corrections are carried out, for better code readability and understanding. 📘🔤

• Note that import lines and added comments are ignored as per the task instruction.

generated by pr-describe

[github-actions]

The method createDefDataNode was renamed to createDefData. Ensure that this change does not break any existing references to the old method name. 🔄

generated by pr-review-commit method_renaming

[github-actions]

The changes can be summarized as follows:

1. Deletion of two log verbose lines from the playwright.ts file which probably were meant for debugging.
2. Addition of a new function HTMLTablesToJSON in globals.ts and html.ts that converts HTML tables to JSON format.
3. Corresponding unit test for HTMLTablesToJSON function has been added in html.test.ts.
4. Renamed createDefDataNode to createDefData in promptdom.ts and modified all its references in other files.
5. Modified trace output in promptrunner.ts to give more descriptive information about the model and template id.

Concerns:

1. The convertTablesToJSON function uses external library tabletojson.convert but there is no error handling for it. The function could break if the library function doesn't work as expected.

Suggestion:

+export function HTMLTablesToJSON(html: string, options?: {}): object[][] {
+    try {
+        const res = tabletojson.convert(html, options)
+        return res
+    } catch(e) {
+        console.error("Error converting HTML Tables to JSON", e)
+    }
+    return []
+}

2. It would be more efficient to handle logging levels centrally, enabling and disabling debugging logs based on environment variables rather than manually commenting and uncommenting such logs.

LGTM with the above suggestions. 🚀

generated by pr-review