# How Do AI Models Perform on the Famous Pelican Test?

AI drawing capabilities evaluated through the iconic pelican-on-a-bicycle benchmark.

- **Source:** [Hacker News](https://news.ycombinator.com)
- **Published:** 2026-07-22
- **Topic:** [Data & AI](https://brifing.si/c/data-ai)
- **Original article:** <https://dylancastillo.co/posts/pelicanmaxxing.html>
- **On Brifing:** https://brifing.si/s/29ec63e1-69f8-411e-af1e-25fde271eeca

## Origin of the Pelican Test

Simon Willison has tested every major LLM release with the same prompt: 'Generate an SVG of a pelican riding a bicycle.' What began as a joke has become one of AI's most famous informal benchmarks.

## Test's Popularity

The pelican test is now so famous that there's debate about whether AI labs might be 'pelicanmaxxing' - specifically optimizing for this benchmark. Results often become top comments on Hacker News release threads.

## Research Methodology

The study created 48 prompts combining 8 animals and 6 vehicles. 1,008 SVG images were generated across 7 leading models. Each image was scored 1-5 for animal, vehicle, and scene coherence.

## Models Tested

GPT-5.6 Terra, Claude Sonnet 5, Gemini 3.5 Flash, Grok 4.5, Qwen3.7-Max, GLM-5.2 and DeepSeek V4 Pro were compared. Three samples per prompt were generated at temperature 1.0.

## Evaluation Process

Generated SVGs were rendered to PNGs. GPT-5.6 Luna scored each image. Gemini 3.1 Flash-Lite analyzed scene elements. Only 11 regenerations were needed across all 1,008 images.

## Pelicans Underperformed

Pelican drawings ranked 6th out of 8 animals across all models. Cats, whales and raccoons were drawn better, suggesting labs aren't specifically training for pelicans.

## Bicycles Also Struggled

Bicycle drawings ranked 5th out of 6 vehicles. Simple vehicles like skateboards scored higher. 2/3 of bicycle images had missing or disconnected parts.

## Famous Combination Failed

The pelican-bicycle combination ranked 42nd out of 48. Even after difficulty adjustment, it showed no special performance, indicating no specific optimization.

## Model Comparison

Claude Sonnet 5 led overall performance, while GLM-5.2 did best on pelican-bicycle. However, its strong performance across other combinations suggested general capability rather than specific optimization.

## Future of Testing

The research shows informal benchmarks shouldn't drive model development. More comprehensive tests evaluating general capabilities are needed for proper AI assessment.

## Conclusions

While the pelican test remains a fun benchmark, there's no evidence labs are specifically optimizing for it. The study recommends more diverse and comprehensive evaluation methods for AI systems.

## Sources covering this story

- https://dylancastillo.co/posts/pelicanmaxxing.html — 

---

_Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._