This project analyses dialogues between an agent and a customer, discussing life insurance options in this example. The code processes text files containing dialogue, categorizes the dialogue based on sentiment, and extracts key topics from positive and concern-related statements for both the agent and the customer.
Sentiment Analysis: Uses a pre-trained DistilBERT model fine-tuned for sentiment analysis to classify each sentence in the dialogue as positive, neutral, or negative (concern).
Topic Extraction: Extracts key topics from the dialogue by identifying the most representative sentences within each sentiment category.
Dialogue Parsing: Handles the parsing of text files where dialogue is structured with speakers identified by names followed by a colon (:).
This project relies on the following Python libraries:
nltk: Used for sentence tokenization.
transformers: Provides pre-trained models for sentiment analysis and embedding generation.
torch: Required for running models from the transformers library.
numpy: Used for numerical operations, including computing cosine similarity.
scikit-learn: Used for calculating cosine similarity between sentence embeddings.
The script categorizes and extracts topics based on the dialogue sentiment:
Agent's Positive Topics: Sentences that reflect positive information or sentiment provided by the agent.
Agent's Concern Topics: Sentences that indicate potential concerns or issues addressed by the agent.
Customer's Positive Topics: Sentences reflecting the customer's positive reactions or appreciation.
Customer's Concern Topics: Sentences where the customer expresses concerns or asks about potential issues.
life_insurance_conversation.txt: Example input file containing the dialogue.
dialogue_analysis.py: Main Python script containing the code for dialogue analysis.