Qualitative Analysis with AI

Overview

Students explore how well AI tools can detect the emotional tone of online customer reviews for a product or business.

Why Use This?

Students gain hands-on experience using current AI tools for a common practice in the marketing field known as sentiment analysis, which examines online customer reviews of a product or business for a positive, negative, or neutral tone. In addition to practicing how to use AI tools for this purpose, students strengthen their critical thinking skills by comparing and contrasting these tools and the quality of their output.

How Does It Work?

Students are asked to use multiple AI tools (e.g., ChatGPT, Claude, or Gemini) to analyze customer sentiment for a specific product or business. They then write a single prompt for each tool that asks the AI to classify the reviews as positive, negative, or neutral; explain why; identify common themes; and suggest improvements.

Students then compare their results across tools and discuss each tool’s strengths, weaknesses, and differences. Finally, students are asked to think about how businesses might use AI tools for sentiment analysis and are invited to share their insights in a discussion forum.

Here are example instructions for the activity:

Sample Instructions: LLM Sentiment Analysis Discussion

Purpose

In a previous assignment, you conducted a sentiment analysis using traditional methods. Now, you’ll explore how AI language models perform sentiment analysis in comparison. You will gain hands-on experience with current AI tools and develop critical thinking skills by comparing and contrasting these tools and the quality of the sentiment analysis they provide.

Instructions

  1. Choose at least two AI language models from the following options:
    1. OpenAI’s ChatGPT
    2. Anthropic’s Claude
    3. Google’s Gemini
  2. Create a single, clear prompt that you will use consistently across all chosen AI tools. This prompt should include:
    1. Instructions for sentiment analysis (positive, negative, or neutral)
    2. Request for a brief explanation of each classification
    3. Request to identify recurring themes or issues
    4. Request for actionable recommendations based on the analysis

Example prompt: “Analyze the sentiment of the following customer reviews for a fitness club. For each review, classify the sentiment as positive, negative, or neutral, and provide a brief explanation for your classification. After analyzing all reviews, identify any recurring themes or issues mentioned. Finally, provide 3-5 actionable recommendations for the fitness club based on this analysis.”

  1. Use your prompt and the same preprocessed dataset reviews from Part 1 of the project to complete the analysis. Note: Ensure the data is in a format that can be easily input into the chosen AI tools.
  2. Save the results of your analysis.
Initial Post

Go to the discussion forum and share your perspective on the following:

  1. What are the strengths and weaknesses of each tool? Is one tool clearly better at a specific task (e.g., classification, recommendations, themes)?
  2. How did these tools’ recommendations compare to those you created in Part 1?
  3. Discuss potential implications for businesses using AI-assisted sentiment analysis with these tools.

Keep In Mind

Direct students to prepare their data in a way that is consistently formatted and compatible with their chosen AI tools.

Testimonial