AI Music Video Showdown: Claude vs GPT in Autonomous Filmmaking

The Challenge: Autonomous AI Filmmaking

We set out to answer a simple question: Can frontier AI models autonomously direct a music video from scratch? To find out, we built a tool-calling harness that gave Claude Fable 5 and GPT-5.6 Sol the same song—Bruno Mars and Mark Ronson's 'Uptown Funk'—along with a $100 budget, web search, and local ffmpeg access. The models had to research video generation tools, create clips, watch their own footage, edit it, and produce a final cut.

Each model ran at two budget levels ($25 and $100), for four total runs. The entire process was logged, including every tool call, error, and generation charge. The goal was to see how these models handle open-ended, long-horizon creative tasks when left to their own devices.

The Setup: Tools and Constraints

The harness provided six tools: a planning tool for thinking (no cost), web search for researching models and APIs, a budget checker, image and video generation tools (the only budget-consuming actions), and a local shell with ffmpeg/ffprobe for audio analysis and video editing. The models could choose any generation model from FAL or Replicate, and had to manage their budget carefully.

Once the budget hit zero, paid generation stopped, but the models could continue editing. This setup forced them to prioritize and plan their spending, mimicking real-world production constraints.

The Results: Divergent Approaches

The four runs produced markedly different results. Claude Fable 5 at both budgets stuck to a text-to-video pipeline, using Wan 2.5 at $25 and Seedance 1.0 Pro at $100. GPT-5.6 Sol was more experimental: at $25, it used an image-to-video approach, generating keyframes with FLUX schnell and animating them with Wan 2.2. At $100, it mixed three different video models—Wan 2.5, Veo 3.1 Lite, and Hailuo 2.3 Standard—in a single run.

This divergence in strategy reflects each model's inherent strengths. Claude Fable 5 favored consistency and stability, while GPT-5.6 Sol prioritized creative exploration, even if it meant higher risk of failure.

Cost and Efficiency Analysis

The financial breakdown reveals significant differences. Claude Fable 5's $100 run spent $48.60 on generation and $25.05 on LLM tokens, totaling $73.65. GPT-5.6 Sol's $100 run spent only $36.57 on generation and $3.25 on tokens, totaling $39.82. Despite similar token volumes, GPT-5.6 Sol's token costs were dramatically lower—about 85% less than Claude's.

This cost disparity is crucial for developers and content creators. GPT-5.6 Sol offers comparable or better creative output at a fraction of the operational cost, making it more accessible for budget-constrained projects.

Creative Quality and Technical Flaws

None of the videos were cinematic masterpieces, but the creative differences were telling. GPT-5.6 Sol at $25 proved the most inventive editor, overlaying text and animating still images with video effects—techniques none of the other runs attempted. Claude Fable 5's $100 video was subjectively preferred for its coherent output, but it lacked the experimental flair of GPT's approach.

Common weaknesses included character and story inconsistency, overly literal interpretations of lyrics (e.g., generating a dragon for 'make a dragon wanna retire, man'), and poor tempo matching between cuts and music. Notably, none of the models iterated on their edits or self-reviewed their clips, suggesting a fundamental limitation in current AI's ability to refine creative work.

Tool Usage and Error Handling

The models' tool usage patterns revealed their operational philosophies. Claude Fable 5 made fewer but more deliberate tool calls, with zero failed generation calls in its $100 run. GPT-5.6 Sol made more calls but experienced more errors—61 failed calls at $25 and 10 at $100—though these were mostly transient network issues.

Neither model touched Replicate, despite both FAL and Replicate keys being available. This suggests that FAL's pricing and model selection were more attractive, or that the models' training data biased them toward FAL's ecosystem.

Broader Implications for AI Creativity

This experiment highlights a critical gap in current AI capabilities: while frontier models can execute complex, multi-step tasks, they struggle with subjective, stylistic, and iterative refinement. The lack of self-review and editing iteration is particularly notable, as it suggests that even the most advanced models lack the meta-cognitive ability to assess and improve their own output.

However, the fact that these models can autonomously produce a full music video—from research to final cut—is a significant milestone. As models continue to improve, we can expect better character consistency, more nuanced creative decisions, and lower costs. For now, GPT-5.6 Sol offers the best balance of cost and creativity, while Claude Fable 5 provides stability and coherence.

The full transcripts and code are available open source, allowing developers to replicate and extend these experiments. The question is no longer whether AI can create, but how we can guide it to create better.