Vision models are great at "click the green button" and bad at "click div[role="button"].btn-primary:nth-child(3)". vulpine-mark generates annotated screenshots with durable object IDs so agents can click by label instead of selector.
Renders a screenshot of the current page, overlays numbered labels on every interactive element, and returns the annotated image plus a stable objectId per label. The agent picks an element by its @N label; vulpine-mark resolves that back to a DOM target on the next interaction tool.
VulpineOS exposes vulpine-mark through the MCP toolbelt as vulpine_annotated_screenshot and vulpine_click_label. It also works standalone over CDP via Foxbridge for any agent stack that wants Set-of-Mark behaviour.
Labels survive minor DOM churn — re-annotation reuses the same id when the element is the same target.
Agents reference @1, @2, @3 instead of selectors. Lower hallucination rate, higher hit rate.
Works with VulpineOS, Foxbridge, or any CDP-driven browser. No VulpineOS dependency.
All four open-source pieces — VulpineOS, Foxbridge, vulpine-mark, mobilebridge — are MPL 2.0 and ship together.
