# Pelican Bicycle Drawing: AI Models' Performance Analyzed The performance of AI models in drawing pelicans on bicycles has been analyzed. - **Source:** [Hacker News (news.ycombinator.com)](https://brifing.si/source/13c923cd-6d99-442b-9b8b-1181f4d8cd91) - **Published:** 2026-07-22 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/ff9bfab6-8d45-4701-89d5-39174c4c439b ## Pelican Bicycle Drawing: A Benchmark Story Simon Willison has been testing every major language model release with the prompt 'Generate an SVG of a pelican riding a bicycle' for the past few years. What started as a joke has become one of the most famous informal benchmarks in AI. ## Popularity on Hacker News Simon's pelican bicycle drawing results are often among the most upvoted comments on Hacker News threads announcing new AI model releases. This test has now become a standard for AI labs. ## Pelicanmaxxing: Benchmark Manipulation? With billions or even trillions of dollars at stake, a strong result could persuade users. Could AI labs be manipulating this benchmark? An experiment was conducted to answer this question. ## Experiment Details 1,008 SVGs were generated across 7 different AI models. Each image was scored by an LLM judge and analyzed using Claude Fable 5. All code is available on Github. ## Animal and Vehicle Selection The experiment used 8 animals and 6 vehicles. The pelican and bicycle combination was chosen as a variation of Simon's famous prompt. Other animals and vehicles varied in similarity and difficulty. ## Models and Testing Process 7 different models, including GPT-5.6 Terra, Claude Sonnet 5, and Gemini 3.5 Flash, were tested. Three samples were generated for each prompt, and each image went through a three-stage pipeline. ## Visual Evaluation Each SVG was rendered to PNG and scored by GPT-5.6 Luna for the animal, vehicle, and coherence of the action. These scores were used to compare the performance of the models. ## Pelicans' Performance Pelicans ranked 6th out of 8 animals. If labs were working on the benchmark, pelicans would be expected to rank higher. However, this was not the case. ## Bicycles' Performance Bicycles ranked second to last out of 6 vehicles. This suggests that labs did not specifically train on the benchmark. Drawing bicycles proved to be more challenging than other vehicles. ## Pelican Bicycle Combination The pelican bicycle combination ranked 42nd out of 48 combinations. This result supports the idea that labs did not specifically train on the benchmark. ## Conclusion and Implications The experiment shows that AI labs did not manipulate the pelican bicycle drawing benchmark. This increases the reliability of informal benchmarks in evaluating AI model performance. ## Sources covering this story - https://dylancastillo.co/posts/pelicanmaxxing.html — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._