Control a C+ desktop app from a language model

A C+ desktop application can expose its live native interface as a semantic agent surface. In 0.0.28 the same model is implemented for AppKit on macOS, GTK on Linux, and Win32 on Windows. Facet supplies the integration layer; direct users can open the platform backend themselves.

The stack

• agent_core owns stable node identities, exposure, roles, capability grants, sensitivity tiers, authorization outcomes, events, and the platform-neutral backend vtable.
• agent_appkit, agent_gtk, and agent_win32 walk live native trees and execute approved operations. The mobile siblings are agent_uikit and agent_android.
• agent_mcp exposes the backend over JSON-RPC 2.0 and MCP. The transport does not create authority; it passes the policy's grant to every backend call.
• agent_inapp calls the same backend without a socket. Each session carries a fixed grant.
• facet_agent installs the correct platform surface for a Facet application.

Describe, act, observe

1. Call describe_ui. The default exposed view is the small, keyed tree an assistant should reason about. The opt-in full view adds structural nodes, classes, and frames for diagnostics but does not bypass authorization.
2. Act by stable id with verbs such as click, set_text, and scroll_to. Text writes include the version last observed; VersionConflict means the UI changed and must be described again before retrying.
3. Consume semantic events filtered by node, verb, and role. Event queues are bounded and do not block the UI emitter.

The application controls two separate boundaries. Exposure determines which nodes belong to the surface. The caller's Grant determines whether it can read values, operate controls, cross Protected or Private sensitivity tiers, or edit the UI tree. A disabled, observe-only, or unwired control is still bounded by its native affordance ceiling even when a grant contains act.

Embedded driver

A Facet application enables the optional layer before App::run, then opens a session for the task:

import "facet_agent/agent" as agent;
import "agent_core/auth" as auth;

agent::enable();

let reader = agent::in_app_with_grant(auth::reader());
let operator = agent::in_app_with_grant(auth::operator());

reader can describe and read ordinary exposed values but cannot act. operator can also operate ordinary controls. Protected access requires a new session with a deliberately wider grant; the standard convenience grants never include Private content or structural editing.

External driver

For MCP, install an application policy with the shape fn(auth::Request) -> auth::Grant, enable facet_agent, and register an application id through facet_runtime. Request carries the caller's channel, client name, requested method, and opaque token. A client name makes a prompt legible but is not a credential; verify the token when identity must be unforgeable.

fn policy(req: auth::Request) -> auth::Grant {
    if req.method == "describe_ui" { return auth::reader(); }
    return auth::nothing();
}

agent::set_policy(policy);
agent::enable();
runtime::agent_mcp("myapp");

An empty grant refuses the request. A Protected node can answer NeedsGrant, which means the application may ask the user and mint a wider grant. A Private node answers Forbidden when policy deliberately withholds the corresponding bit.

Use describe_ui, act by id, and treat the returned outcome as authoritative. The full architecture and platform list are in Agent surface.