Midscene.js: Betting on Pure Vision for Cross-Platform UI Automation
An MIT-licensed open-source UI automation framework from ByteDance. UI actions rely solely on feeding screenshots to a vision-language model, with no DOM parsing. A single JS API works across Web / Android / iOS / desktop. The trade-offs: each step is slower and more token-expensive, and everything hinges on the model's grounding ability. Note that Midscene retired MCP after 1.9.8 in favour of Skills + CLI.