Does John Snow provide a way to generate narration based on the given points?

[AayushSameerShah]

Hello 👋
I will be quick, promise 😅

👉 Background

I have a program that takes a dataset as input and calculates some metrics based on the input and generates the points written in English. The points are templates. The main goal is to generate points that can be used as in interpretation of the given visualization in the dashboard/visualization.

👉 Example

Suppose there is a pie chart in the dashboard that shows which state of USA contributes to the gross sales and by how much. Thus there are 2 columns, one is numeric (for GrossSales) and second is the categorical (name of the state).

➡️ Then my program would produce the following based on the preset template:

- Florida (30.17%, 66158446), Arkansas (25.04%, 54913648) significantly contributed to the GrossSales, with Florida making the most significant contribution.

- Ohio (11.68%, 25601611), Arizona (11.45%, 25113742) did not significantly contribute to the GrossSales, with Arizona making the least significant contribution.

➡️ Now based on this I want to create a narration like the given below which takes all points created my program and generates a paragraph that has a smooth reading flow:

Florida and Arkansas are two states that significantly contributed to the GrossSales in the dataset. Florida made the most significant contribution, with 66158446 units sold and a GrossSales of 30.17%. Arkansas also made a significant contribution, with 54913648 units sold and a GrossSales of 25.04%.

Ohio and Arizona, on the other hand, did not significantly contribute to the GrossSales in the dataset. Ohio made the least significant contribution, with 25601611 units sold and a GrossSales of 11.68%. Arizona also made a relatively small contribution, with 25113742 units sold and a GrossSales of 11.45%.

In conclusion, Florida and Arkansas made the most significant contributions to the GrossSales in the dataset, while Ohio and Arizona did not significantly contribute.

👉 Problem

This can easily be achieved with GPT-3.5/4 or other LLMs like GPT-J/OpenAssit models which are >6B & instruction tuned. But I am wiling to find a simpler appraoch which can be light weight and simple.

I have tried GPT-2 and T5 (which are reasonably smaller) but their summaries are not approapriate (can't call these "summaries" right? "praraphrazing"?). Also paraphrasing models are not working.

Will you please suggest something? I have tried both of these model on JSL as well, but didn't work. And this templatized approach doesn't look good and feels more robotic. By using something I want to add some dynamic words in my narration.

---

Bigger and instruction tuned models have shown pretty good results but as asked here #13864 , JSL is adding the support of CausalLM. And as of now I am not sure if there are any good models to do this task. Also for fine tuning, the appropriate datasets are not available.

Please direct, if there is anything on this platform that I can use.
Thank you 🙏

[maziyarpanahi]

We are in the process of integrating ONNX Runtime in addition to TensorFlow in Spark NLP. Once that happens, we will bring CasualLM models that were fine-tuned over instructions and allow you to construct prompts to allow this.

Stay tuned, more updates will come! :)