Repolode
Submit
leigest519

leigest519/ScreenCoder

Converts UI screenshots and design mockups into editable HTML/CSS code using a modular multi-agent pipeline.

Open-source alternative to:Figma to code pluginsLocofyAnima

View on GitHub
Stars
3,006
Forks
320
Open issues
16
Last commit
last month

What is it?

ScreenCoder is a visual-to-code system that takes a screenshot or design mockup and generates HTML/CSS front-end code. It splits the work across modules: block detection, UI element detection via the UIED engine, placeholder generation, and image replacement that crops original regions from the screenshot back into the page.

The pipeline lets developers and designers adjust the generated layout and styling after generation, rather than producing a fixed output. Generation models can be selected among Doubao (default), Qwen, GPT, and Gemini, each configured with a local API key file. A Hugging Face demo is available, and the repository includes post-training code (SFT and RL) and the ScreenBench benchmark of web screenshots paired with HTML source.

The project is a research codebase tied to an EMNLP 2026 paper, and it is run through a sequence of Python scripts rather than a packaged library.

Key features

  • 01Turns UI screenshots or design mockups into HTML/CSS code.
  • 02Detects UI components with the UIED engine and maps them to page regions.
  • 03Replaces placeholder blocks with cropped images from the original screenshot.
  • 04Supports Doubao, Qwen, GPT, and Gemini as generation models.
  • 05Includes SFT and RL post-training code and the ScreenBench dataset.

Use cases

  • Prototyping a web page quickly from an existing screenshot or mockup.
  • Generating an editable starting point for front-end code from a design draft.
  • Building a benchmark or experiments on visual-to-code generation.

Topics

#ui-to-code#html-css#multimodal-llm#computer-vision#front-end#code-generation#screenshot-to-code

FAQ

What is ScreenCoder?
Converts UI screenshots and design mockups into editable HTML/CSS code using a modular multi-agent pipeline. ScreenCoder is a visual-to-code system that takes a screenshot or design mockup and generates HTML/CSS front-end code. It splits the work across modules: block detection, UI element detection via the UIED engine, placeholder generation, and image replacement that crops original regions from the screenshot back into the page.
Is ScreenCoder free to use? What license does it have?
ScreenCoder is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is ScreenCoder still maintained?
Updated within 6 months. The last commit to ScreenCoder was on Sep 15, 2026.
What can I use ScreenCoder for?
Prototyping a web page quickly from an existing screenshot or mockup. Generating an editable starting point for front-end code from a design draft. Building a benchmark or experiments on visual-to-code generation.
Where can I get ScreenCoder?
The source code and installation instructions are on GitHub: https://github.com/leigest519/ScreenCoder

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects