Show HN Real Design Benchmark: Show HN: Real Design Benchmark
In this story
Hi HN,<p>I'm Adam Ginzberg, one of the cofounders of Subframe, a code-first design tool.<p>One of the things I'm really curious about is understanding LLM performance on subjective domains like design.<p>As an experiment I built this game which presents two designs of the same screen, one of which was generated by AI. First you pick which one you like better and then which one you think was made by AI.<p>Most design work happens in an existing product, so we set up this game to test that. As context, the AI gets 4 screenshots from the existing product, the product's icons + fonts, and a written brief about the screen it is asked to build. It then writes HTML which is screenshotted identically to the real screen. To get the data, we signed up for the products with test accounts and added dummy data. All of the inputs were manually checked to make sure nothing about the target screen was leaked in the prompt. You can
Hi HN,<p>I'm Adam Ginzberg, one of the cofounders of Subframe, a code-first design tool.<p>One of the things I'm really curious about is understanding LLM performance on subjective domains like design.<p>As an experiment I built this game which presents two designs of the same screen, one of which was generated by AI. First you pick which one you like better and then which one you think was made by AI.<p>Most design work happens in an existing product, so we set up this game to test that. As context, the AI gets 4 screenshots from the existing product, the product's icons + fonts, and a written brief about the screen it is asked to build. It then writes HTML which is screenshotted identically to the real screen. To get the data, we signed up for the products with test accounts and added dummy data. All of the inputs were manually checked to make sure nothing about the target screen was leaked in the prompt. You can