The Pudding explored if an AI like Claude can make a data-driven, visual story similar to what they do. They broke the process into four steps and evaluated Claude's work at each stage.
1. Idea Generation
- Claude was given background on The Pudding's storytelling and asked to generate ideas. It came up with ideas on video game female characters, fast fashion's environmental impact, and song lyrics' emotional content.
- The idea about song lyrics was chosen. While the ideas were okay, they lacked the depth and intrigue of a fully developed story. Grade: C+
2. Data Collection & Analysis
- Claude wrote Python scripts to collect data on Billboard Hot 100 songs from 1960 - 2020. It had to scrape lyrics from Genius.com and perform sentiment analysis using NLTK.
- It made mistakes like not handling multiple artists properly but could correct them when guided. It also analyzed the data, finding trends in sentiment over time, differences among genres and artists. Grade: B-
3. Storyboarding & Prototyping
- Claude created a storyboard on the rise of negative sentiment in popular music. It included sections on the trend, genre impact, cultural shifts, artist case studies, and future trends.
- It also prototyped visual ideas, writing Python code for charts. The visuals were mostly good, but the storyboard was a bit safe. Grade: B+
4. Development & Writing
- Claude was asked to write Svelte components for the story. It was good at simple tasks like creating a line chart but struggled with more complex ones like a scatterplot.
- It couldn't generate images as described, and the code it wrote had issues. Grade: D
Conclusion
- Claude's overall grade was C+. While it was helpful in some aspects, it couldn't handle a full project or complex tasks like a human would.
- The Pudding doesn't feel replaceable for now as their work has more craftsmanship. However, they will use AI as a tool, like for writing helper scripts or generating headline options.