Why LLM Pelicans Look So Unusual
| Source: HN | Original article
A recent discussion examines the unusual appearance of LLM “pelican” visualizations, probing the reasons behind their distinctive shape.
A discussion that began on Hacker News this week has turned into a de‑facto benchmark for large language models (LLMs). The thread titled “Why do LLM pelicans look like this?” sparked a flood of comments after users posted SVG drawings of a pelican on a bicycle generated by various models. The visual oddities – crooked beaks, tangled spokes and malformed wings – prompted a deeper look at what the test actually measures.
The “pelican test” was first described in an August 2026 post by Andrej Karpathy, who showed a side‑by‑side comparison of two‑hour, 1 million‑token runs that produced markedly different SVG structures. The evaluation asks three questions: does the image resemble a pelican riding a bicycle, is the SVG code clean and maintainable, and can the model explain its design choices. Because SVG output combines natural‑language reasoning with precise code syntax, the benchmark exposes gaps that pure text tests miss.
Why it matters is twofold. First, the test highlights how far LLMs have progressed in generating structured, visual artefacts, a capability increasingly relevant for design tools, automated documentation and UI generation. Second, the recurring failures – tangled paths, missing elements – reveal lingering weaknesses in spatial reasoning and program synthesis that could affect safety‑critical applications.
The community is already looking ahead. Researchers plan to formalise the pelican benchmark, adding larger token budgets and more complex scenes to stress‑test model reasoning. Expect upcoming conference talks and possibly a shared leaderboard that will let developers track improvements across model families. As the conversation evolves, the humble pelican may become a standard yardstick for the next generation of multimodal AI.
Sources
Back to AIPULSEN