#### Remote Protocol Renderer and Serialization The remote protocol renderer side receives protocol messages from the core side and translates them into native window operations and graphics rendering. The serialization and channel infrastructure provides composable layers that convert between typed protocol calls and Parser2 JSON node packages, enabling GacUI applications to run across process boundaries over any user-provided transport (named pipe, HTTP, WebSocket, etc.). ## GuiRemoteRendererSingle `GuiRemoteRendererSingle` (in `Source/PlatformProviders/RemoteRenderer/`) is the renderer-side implementation that bridges `IGuiRemoteProtocol` to a real native window with actual graphics rendering. It is not a full `INativeController` — it relies on a real platform provider (e.g., Windows Direct2D via `SetupRawWindowsDirect2DRenderer()`) already running, and registers itself as a listener on a native window. ### Class Hierarchy `GuiRemoteRendererSingle` inherits from: - `Object` — standard base - `IGuiRemoteProtocol` (virtual) — receives all protocol messages from the core side - `INativeWindowListener` (protected virtual) — listens to native window events (mouse, keyboard, window lifecycle) - `INativeControllerListener` (protected virtual) — listens to global timer and shortcut key events ### Key Members - `window` / `screen` / `events`: The native window handle, current screen, and `IGuiRemoteProtocolEvents` callback for firing events to the core side. - `availableElements`: `Dictionary>` mapping element IDs to actual graphics elements created by the real rendering engine (e.g., `GuiSolidBorderElement`, `GuiSolidLabelElement`). - `availableImages`: `Dictionary>` mapping image IDs to loaded native images. - `renderingDom` / `renderingDomIndex`: The rendering DOM tree received from the core side, used for on-screen rendering and hit testing. - `solidLabelMeasurings` / `fontHeightMeasurings`: Caches and measurement tracking for text elements, supporting the measurement feedback loop. - `pendingMouseMove`, `pendingHWheel`, `pendingVWheel`, `pendingKeyAutoDown`, `pendingWindowBoundsUpdate`: Accumulation fields for coalescing high-frequency events before sending. - `disconnectingFromCore`: Prevents further protocol requests and events after the renderer disconnects from the core. - `stoppedByFatalError` / `fatalError` / `fatalMaskElement` / `fatalTextElement`: Retain-mode state and graphics for displaying a fatal-error overlay instead of immediately closing the renderer. ### Source File Organization - `GuiRemoteRendererSingle.cpp`: Construction, destruction, main window registration, connection lifecycle (`Opened`, `BeforeClosing`, `AfterClosing`, `Closed`), screen/config management, core disconnection, and fatal-error retention. - `GuiRemoteRendererSingle_Controller.cpp`: Controller-level requests — `RequestControllerGetFontConfig`, `RequestControllerGetScreenConfig`, `RequestControllerConnectionEstablished`, `RequestControllerConnectionStopped`. - `GuiRemoteRendererSingle_MainWindow.cpp`: Window style notifications — `RequestWindowNotifySetBounds`, `RequestWindowNotifySetTitle`, `RequestWindowNotifySetEnabled`, `RequestWindowNotifyShow`, etc. - `GuiRemoteRendererSingle_IO.cpp`: IO requests (global shortcuts, mouse capture, key state queries) and native-to-protocol input event conversion. Contains `SendAccumulatedMessages()` for batching high-frequency events. - `GuiRemoteRendererSingle_Rendering.cpp`: Core rendering pipeline — element creation/destruction (`RequestRendererCreated`, `RequestRendererDestroyed`), begin/end rendering, DOM rendering (`RenderDom` recursive traversal), fatal-overlay rendering, hit testing, and `GlobalTimer`/`Paint` driven refresh. - `GuiRemoteRendererSingle_Rendering_Elements.cpp`: Property updates on ordinary graphics elements (solid border, sink border, splitter, background, gradient, inner shadow, polygon). - `GuiRemoteRendererSingle_Rendering_Label.cpp`: Solid label measurement and property updates. - `GuiRemoteRendererSingle_Rendering_Image.cpp`: Image creation, metadata, and image frame element updates. - `GuiRemoteRendererSingle_Rendering_Document.cpp`: Complex document paragraph element rendering via `GuiRemoteDocumentParagraphElement` inner class. ### Rendering Pipeline When the core sends `RequestRendererBeginRendering(ElementBeginRendering)`, the `updatedElements` field contains a list of `OrdinaryElementDescVariant`. Each variant is dispatched via `Apply(Overloading(...))` to the corresponding `RequestRendererUpdateElement_*` helper, which looks up the element by ID in `availableElements` and sets its properties. The core sends `RequestRendererRenderDom` or `RequestRendererRenderDomDiff` to update the rendering DOM tree. The legacy command-based rendering path (`RequestRendererBeginBoundary`/`RequestRendererEndBoundary`/`RequestRendererRenderElement`) is disabled with `CHECK_FAIL` — the DOM-diff approach is required. `RequestRendererRenderDom` installs a full `RenderingDom`, rebuilds `renderingDomIndex`, and sets `needRefresh = true`. `RequestRendererRenderDomDiff` requires an existing DOM, applies `UpdateDomInplace(renderingDom, renderingDomIndex, diffs)`, and also sets `needRefresh = true`. Actual painting happens in either `RequestRendererEndRendering` or `GlobalTimer()`: when `needRefresh` is true and refresh is no longer suppressed, `ForceRender()` calls `RenderDom` to recursively traverse `renderingDom`, renders each element in order with clipping, optionally draws the fatal-error overlay, redraws the native window, and handles render-target resize/lost-device failures by resizing/recreating the target and scheduling another refresh. `RequestRendererEndRendering` clears suppression and can force the completed frame to render before returning `RespondRendererEndRendering(id, elementMeasurings)`. While connected, `GlobalTimer()` also flushes accumulated IO events and drives caret blinking; after disconnection it clears pending core events but can continue rendering a retained fatal-error overlay. `RequestRendererIdle()` is an idle hint and is intentionally ignored by this renderer. ### Event Forwarding `GuiRemoteRendererSingle` implements all `INativeWindowListener` mouse/keyboard callbacks to forward OS events as protocol events: - **Discrete events** (button clicks, key presses): Sent immediately via `events->OnIOButtonDown`, `events->OnIOButtonUp`, `events->OnIOButtonDoubleClick`, `events->OnIOKeyDown`, etc. Mouse-button events carry an `IOMouseInfoWithButton` whose `button` is the mapped `NativeMouseButton` value (`Left`, `Middle`, `Right`, `Mouse4`, or `Mouse5`) and whose `info` is the complete `NativeWindowMouseInfo`, including independently captured `alt` and `osSuper` state. - **High-frequency events** (mouse move, wheel, key auto-repeat): Accumulated and coalesced: - `pendingMouseMove`: Only the latest `NativeWindowMouseInfo` is kept and sent directly as `IOMouseInfo`. - `pendingHWheel` / `pendingVWheel`: `NativeWindowMouseInfo` wheel deltas are summed across frames and sent directly as `IOMouseInfo`; an `alt` or `osSuper` transition flushes the pending delta before accumulating the next event. - `pendingKeyAutoDown`: Only the latest auto-repeat key is kept. - `SendAccumulatedMessages()` is called from `GlobalTimer()` to flush these accumulated events. - **Window lifecycle events** (`Opened`, `BeforeClosing`, `Moved`, `DpiChanged`): Translated to protocol events like `OnControllerConnect`, `OnControllerRequestExit`, `OnWindowBoundsUpdated`. All outgoing callbacks and responses are guarded by `CanSendEvents()`. `DisconnectFromCore()` marks the renderer as disconnecting, releases mouse capture, unregisters global shortcuts, and clears all accumulated events so no stale input is sent afterward. ### Super-Key Label and Environment Refresh When the native renderer opens, it sends its resource service's canonical Super-key label in `ControllerGlobalConfig::osSuperKeyName`. The remote core returns that label from `GuiRemoteController::GetOSSuperKeyName()`; before the first renderer supplies a label it returns `osSuper`, and disconnecting does not clear the last received label. Each renderer connection invokes the controller environment-change path. `GuiApplication` forwards the change through each live `GuiWindow`, which refreshes its display font and calls `GuiComponent::EnvironmentChanged()` on installed components. `GuiToolstripCommand` overrides that hook to raise `DescriptionChanged`, so bound shortcut text adopts a replacement renderer's label. Detached commands are intentionally not notified, and commands do not register individual `INativeControllerListener` instances. ### Hit Testing Hit testing is performed locally in the renderer by traversing the rendering DOM tree via `HitTestInternal`. Each DOM node may have `hitTestResult` and `cursor` attributes set by the core side. The renderer walks the tree and finds the matching node for a given point. This avoids round-trips — hit testing stays entirely renderer-side. ### Document Paragraph Rendering Document paragraph elements are represented by the renderer-local `GuiRemoteDocumentParagraphElement` wrapper. The wrapper is both an `IGuiGraphicsElement` and its own renderer/factory, but it delegates actual text layout to an `IGuiGraphicsParagraph` created from the active render target. The wrapper caches full paragraph state, including text, wrapping/alignment/max-width, caret state, text runs, inline-object runs, merged runs, inline-object bounds, inline-object properties, and callback ranges. This cache is required because `SetRenderTarget(nullptr)` destroys the native paragraph object; when a new render target arrives, `TryRecreateParagraph()` creates a fresh paragraph and reapplies properties, runs, and caret state. `ApplyUpdateAndFillResponse(arguments, response)` accepts incremental `runsDiff` updates from the core side. The first update must contain `text`; later updates must omit `text` and only update properties/runs. Removed inline objects clear cached bounds/properties/ranges and reset native paragraph ranges when the paragraph exists. Inline-object hit testing calls `TryGetInlineObjectRunProperty(callbackId, outProp)` instead of directly reading the wrapper dictionaries. ## Protocol Serialization and Channel Architecture The channel system converts typed protocol calls to Parser2 JSON node packages and moves those packages through `VlppOS` inter-process channels. It no longer uses a separate JSON-string transformer stack for GacUI remoting. ### Core Interfaces - `IGuiRemoteProtocol`: The high-level typed protocol interface with named methods (`RequestNAME`, `OnNAME`, `RespondNAME`). - `channeling::JsonPackage`: Alias of `Ptr`. - `channeling::IJsonChannel`: Alias of `inter_process::IChannel`, with `IJsonChannelReader`, `IJsonChannelClient`, and `IJsonChannelServer` aliases for the corresponding channel roles. - `GuiRemoteProtocolNetworkChannelServer`: Network channel server alias based on `inter_process::NetworkProtocolChannelServer`. - `GuiRemoteProtocolChannelClient` and `GuiRemoteProtocolLocalChannelClient`: Client helpers that expose the `GacUIRemoteProtocol` JSON channel through `GetProtocolChannel()`. ### Typed Protocol to JSON Channel Two channel adapters bridge typed protocol calls and JSON packages: - `GuiRemoteProtocolCoreChannel`: Implements `IGuiRemoteProtocol` and reads from an `IJsonChannel`. `RequestNAME(...)` serializes arguments with `ConvertCustomTypeToJson()`, packs an envelope with `JsonChannelPack()`, and sends it to the current renderer client. Incoming packages are unpacked with `JsonChannelUnpack()`, dispatched by name, deserialized with `ConvertJsonToCustomType()`, and delivered as `events->OnNAME(...)` or `events->RespondNAME(...)`. It also queues outgoing packages before a renderer is known and exposes `DetachRenderer(clientId)` for renderer replacement. - `GuiRemoteProtocolRendererChannel`: Reads renderer-side JSON packages and calls the wrapped `IGuiRemoteProtocol`. Its constructor takes the renderer-side `IJsonChannel` and the `IGuiRemoteProtocol`; the channel sends renderer events and responses back to `GacUIRemoteProtocolCoreClientId`. ### Transport Layer Named-pipe and HTTP remoting use `VlppOS` network protocol clients/servers underneath the JSON channel. The channel serializer for transport package lists is `glr::json::JsonNodeListSerializer`, shared with Parser2 JSON infrastructure. GacUI remoting code therefore passes `JsonPackage` values through channel clients and servers instead of converting each package through a dedicated `WString` protocol layer. ### JSON Envelope Format Every package is a JSON object with fields: - `"semantic"`: One of `"Message"`, `"Request"`, `"Response"`, `"Event"`. - `"id"`: (optional) An integer request/response ID, present only for `Request` and `Response`. - `"name"`: The message/event name (e.g., `"ControllerGetFontConfig"`, `"IOMouseMoving"`). - `"arguments"`: (optional) The serialized arguments as a JSON value. Messages (no response expected) use `"Message"`. Requests (response expected) use `"Request"` with an ID. Responses use `"Response"` with the matching ID. Events use `"Event"`. ### JSON Serialization of Protocol Types Protocol types are code-generated from `Protocol/*.txt` files into `GuiRemoteProtocolSchema.h`/`.cpp`. Each struct gets a `JsonHelper` specialization with `ToJson` and `FromJson` methods. The shared infrastructure in `GuiRemoteProtocolSchemaShared.h` provides: - Primitive type serializers: `bool`, `vint`, `float`, `double`, `WString`, `wchar_t`, `VKEY`, `Color`, `Ptr` (Base64-encoded). - Generic container serializers: `Nullable`, `Ptr`, `List`, `ArrayMap`, `Dictionary`. - `Variant` types are serialized with a type discriminator field. - `ConvertCustomTypeToJsonField` adds a named field to a JSON object. ### Async Channel `GuiRemoteProtocolAsyncJsonChannel` is the core-side async wrapper around an `IJsonChannel`. It queues outgoing packages, queues incoming events for `ProcessRemoteEvents()`, stores incoming responses by request id, and blocks `BatchWrite(disconnected)` until the current `PendingRequestGroup` is satisfied or disconnected. `connectionCounter` and `connectionClientId` protect pending requests when channel events arrive after disconnect/reconnect boundaries. `GuiRemoteProtocolAsyncJsonChannelRenderer` is the renderer-side async wrapper. It queues received packages and main-thread tasks, then schedules `ProcessPendingMessages()` through an `IGuiRemoteProtocolAsyncRendererInvoker`. Before `SetInvokeInMainThread(...)` is called by renderer `GuiMain`, work is cached. After the invoker is installed, it is drained in FIFO order on the renderer UI thread. `Initialize(reader)` requires a non-null reader; `Detach()` explicitly clears it, increments `messageVersion`, and drops queued work. The version check prevents callbacks queued before `Detach()` from running after detachment or a later reader installation. ## Demo Project Pair Two projects in `Test/GacUISrc/` demonstrate a full remote protocol deployment. They are paired: one is the core side (console application) and the other is the renderer side (Windows application). Remote-view-model remoting support is organized in `Test/RemotingHelpers/Rvmt/`. `ViewModelShared.h` owns only generic channel aliases, fixed RVM constants, and inline Ready-message helpers; `ViewModelHostClient.*` owns the generic network host client; and `ViewModelHostServer.*` owns the protected generic `RpcServerHelpers` implementation and the application-facing `RemoteViewModelChannelServer`. Concrete generated-RPC composition lives instead in `Generated_RemoteViewModelTest/RemoteViewModelTestInitialize.*` and is invoked directly by each consuming application. The consolidated `Source_RemotingHelpers.vcxitems` inventory lists only generic helpers under its `Rvmt` filter and is imported only by remoting applications; standalone applications receive reusable automation endpoints through GacUI Core instead. ### RemotingTest_Core (Console Application) Located at `Test/GacUISrc/RemotingTest_Core/`. Accepts `/Pipe`, `/Http`, or `/MiniHttp` arguments to start a named-pipe, Windows HTTP, or portable MiniHTTP server. **Protocol stack setup** (`StartServer` in `GuiMain.cpp`): 1. `StartNamedPipeServer`, `StartHttpServer`, or `StartMiniHttpServer` constructs the matching `RemotingChannelServer`. `/RVMT` constructs `RemoteViewModelChannelServer`, which adds exact RVM-host admission while preserving the base renderer behavior. 2. The shared run function starts the server, creates a local core client with `GuiRemoteProtocolLocalChannelClient`, and connects it to the server. Local clients are accepted directly because only the owning process can call `ConnectLocalClient`. 3. It wraps the core client's protocol channel in `GuiRemoteProtocolAsyncJsonChannel` and creates `SwitchableRenderersCoreChannel`. This `GuiRemoteProtocolCoreChannel` subclass verifies at submission time that its protocol renderer is still the transport server's current renderer. `Submit` evaluates this virtual guard before applying the base channel's default no-renderer behavior, so the switchable channel can report disconnection for both a stale renderer and renderer id `-1` while the default channel still treats `-1` as an idle state. After `BatchWrite`, it checks both renderer identity and the transport guard again: admission can detach a renderer while the batch delivers its responses. A cancelled batch clears both outstanding IDs and buffered responses in `GuiRemoteProtocolFilter`, so a replacement renderer cannot inherit partial results from its predecessor. 4. It builds `GuiRemoteProtocolFilter` -> `GuiRemoteProtocolDomDiffConverter` -> `SetupRemoteNativeController`. The Core remains usable before a renderer connects. 5. In `/RVMT`, the shared server owns the requester session. The application calls `Connect` once with the complete required-service-name list, initializes generated RPC objects with the exposed generic dispatcher and assigned client ID, then calls repeatable `RequestService`. The exact RVM host is admitted before service acquisition, while renderer admission remains closed until the first service acquisition moves the requester into its running phase. 6. On shutdown, Core clears the server's stored JSON/protocol channel pointers, finalizes the optional requester session, and stops the channel server before stack-owned wrappers are destroyed. `RemotingChannelServer::OnRemoteClientConnected` accepts replacement renderers. If a different renderer is already current, it calls `GuiRemoteProtocolCoreChannel::DetachRenderer(oldClientId)`, tries to send a raw `ControllerConnectionStopped` package to the old renderer, and disconnects the old transport only when notification fails. `RemoteViewModelChannelServer` overrides this single remote-admission operation to accept the RVM host and to check the concrete requester phase before delegating renderer admission. If the RVM host disconnects, it broadcasts the business error when renderers are enabled and then terminates the requester. ### RemotingTest_Rendering_Win32 (Windows Application) Located at `Test/GacUISrc/RemotingTest_Rendering_Win32/`. Accepts `/Pipe`, `/Http`, or `/MiniHttp` arguments to start as a named-pipe, Windows HTTP, or portable MiniHTTP client. **Protocol stack setup** (`StartClient` in `GuiMain.cpp`; this function is not a template): 1. Receives a named-pipe, Windows HTTP, or MiniHTTP `INetworkProtocolClient` and creates the shared `RemoteProtocolRendererClient`, derived from `GuiRemoteProtocolChannelClient`, over it. 2. Creates `GuiRemoteProtocolAsyncJsonChannelRenderer` over the client's protocol channel. 3. Creates `GuiRemoteRendererSingle` and `GuiRemoteProtocolRendererChannel(&asyncRendererChannel, &remoteRenderer)`. 4. Waits for the server, then calls `SetupRawWindowsDirect2DRenderer()` to run the native window event loop. 5. In `GuiMain()`, creates the native window, registers it with `GuiRemoteRendererSingle`, creates a retained `Ptr`, installs it through `asyncChannel->SetInvokeInMainThread(invoker)`, drains startup work with `ProcessPendingMessages()`, and runs the window service. 6. On exit, clears the invoker, unregisters the main window, stops the network connection, and clears stack-owned renderer/channel pointers. `RemoteProtocolRendererClient` queues both protocol packages and terminal actions through the async renderer's ordered main-thread FIFO. A Core-authored `!Error` arrives through `OnReadError`, claims the first fatal error, and opens the native Yes/No prompt. Choosing Yes calls `ForceExitByFatelError()`; choosing No calls `RetainByFatalError(message)`, keeps the native renderer window open with a `[STOPPED]` title and fatal overlay, and exposes the error through renderer automation. A fatal local channel error has different UI semantics: after VlppOS's `IChannelClient` promotes a post-connection protocol error, `OnLocalError(..., true)` queues the ordinary disconnected transition directly, without showing a fatal prompt and without waiting for `OnDisconnected`. `OnDisconnected` queues the same idempotent transition when it is delivered. FIFO ordering lets an earlier `ControllerConnectionStopped` or Core `!Error` win before detach. ### Protocol Stack Direction **Core side** (messages flow outward): ``` SetupRemoteNativeController → GuiRemoteProtocolDomDiffConverter → GuiRemoteProtocolFilter → GuiRemoteProtocolCoreChannel → GuiRemoteProtocolAsyncJsonChannel → GuiRemoteProtocolLocalChannelClient / IJsonChannel → GuiRemoteProtocolNetworkChannelServer → Named pipe, Windows HTTP, or MiniHTTP transport ``` **Renderer side** (messages flow inward): ``` Named pipe, Windows HTTP, or MiniHTTP transport → GuiRemoteProtocolChannelClient / IJsonChannel → GuiRemoteProtocolAsyncJsonChannelRenderer → GuiRemoteProtocolRendererChannel → GuiRemoteRendererSingle (IGuiRemoteProtocol) → Native window rendering ``` Events flow in the reverse direction through the same stack.