# Inter-Process Network Protocols and Channels

## Intended Scope and Extension Strategy

The `vl::inter_process` namespace is designed for inter-process communication. Some abstractions are general enough to carry messages inside one process or across a custom transport, but the lifecycle, error handling, connection model and naming are intended for communication between processes.

The built-in `INetworkProtocol*` transports should be treated as reference implementations, validation targets and demo-friendly options. The current concrete `vl::inter_process::named_pipe::NamedPipeServer` / `vl::inter_process::named_pipe::NamedPipeClient` and `vl::inter_process::windows_http::HttpServer` / `vl::inter_process::windows_http::HttpClient` implementations are Windows-only, and they are not meant to be the only production transport choice for every application.

This distinction is especially important for HTTP. `vl::inter_process::windows_http::HttpServer` and `vl::inter_process::windows_http::HttpClient` are raw `INetworkProtocol*` reference/demo transports, while `vl::inter_process::windows_http::HttpServerApi` and `vl::inter_process::windows_http::HttpClientApi` are lower-level Windows HTTP helper utilities. When a Windows feature needs HTTP request/response behavior directly, the helper utilities can still be used without adopting the inter-process raw transport as the feature contract.

When the built-in raw protocol implementation does not fit the platform, security model, deployment shape, performance target or reconnection behavior of a feature, implement a custom `INetworkProtocolServer`, `INetworkProtocolClient` and `INetworkProtocolConnection`. The default channel bridge can still be reused as long as the custom raw transport follows the `INetworkProtocol*` contract.

Feature code should usually depend on `IChannelServer<TPackage>`, `IChannelClient<TPackage>` and `IChannel<TPackage>` instead of a concrete raw transport. Keeping the feature boundary at `IChannel*` decouples package delivery from the underlying `INetworkProtocol*`, so the transport can be replaced later without rewriting the feature logic.

The inter-process communication APIs in `vl::inter_process` are layered:

- The raw protocol layer, represented by `INetworkProtocolServer`, `INetworkProtocolClient`, `INetworkProtocolConnection` and `INetworkProtocolCallback`, exchanges asynchronous `WString` messages between a server and clients.
- The channel layer, represented by `IChannelServer<TPackage>`, `IChannelClient<TPackage>`, `IChannel<TPackage>` and `IChannelReader<TPackage>`, builds typed named channels on top of a connected client id model.
- The default bridge, represented by `NetworkProtocolChannelServer<TPackage, TSerialization, TServerBase>`, `NetworkProtocolChannelClient<TPackage, TSerialization>` and `NetworkProtocolLocalChannelClient<TPackage, TSerialization>`, serializes channel batches into raw `WString` messages over an `INetworkProtocol*` transport.

The transport-agnostic contracts and channel templates stay directly in `vl::inter_process`. Windows concrete implementations are grouped by feature: named-pipe types are in `vl::inter_process::named_pipe`, and HTTP transport, helper, request, response and error types are in `vl::inter_process::windows_http`.

The interfaces are transport-agnostic. Use the abstract interfaces in portable feature code, and bind them to concrete transports at application composition boundaries.

## Choosing the Layer

- Use `INetworkProtocolServer`, `INetworkProtocolClient`, `INetworkProtocolConnection` and `INetworkProtocolCallback` when the application only needs raw asynchronous text messages.
- Use `IChannelServer<TPackage>`, `IChannelClient<TPackage>`, `IChannel<TPackage>` and `IChannelReader<TPackage>` when the application needs named logical channels, typed packages, client ids, direct delivery, broadcast delivery or batched writes.
- Use `NetworkProtocolChannelServer<TPackage, TSerialization, TServerBase>` and `NetworkProtocolChannelClient<TPackage, TSerialization>` when the channel layer should run over an existing `INetworkProtocolServer` / `INetworkProtocolClient` transport.
- Use `NetworkProtocolLocalChannelClient<TPackage, TSerialization>` when server-side logic needs to participate in a channel as a normal speaker with a real positive client id.
- Use `ChannelSerializer<TSerialization>` when an existing `IChannel<DestType>` should be adapted to another package type by serializing and deserializing packages.
- Choose or implement the concrete `INetworkProtocol*` transport at the application composition boundary; keep feature code on `IChannel*` when transport replacement should remain possible.

## Raw Network Protocol Contract

`INetworkProtocolConnection` represents one text-message connection between two peers.

- `InstallCallback` installs one `INetworkProtocolCallback`; passing `nullptr` uninstalls it.
- `BeginReadingLoopUnsafe` starts asynchronous reading. Some implementations may already be receiving data before this call, so user callbacks must tolerate early delivery after callback installation.
- `SendString` sends one `WString` message to the other side.
- `Stop` closes the connection. Implementations should treat `Stop` as the shutdown boundary and drain asynchronous callbacks before returning.

`INetworkProtocolCallback` receives connection events and must be thread-safe, because callbacks may run on any thread.

- `OnInstalled` gives the callback its `INetworkProtocolConnection`.
- `OnConnected` reports that the connection became available.
- `OnReadString` receives a normal text message.
- `OnReadError` receives an error reported by the remote side.
- `OnLocalError` receives a local transport failure; the `fatal` argument indicates whether the connection should be considered disconnected.
- `OnDisconnected` reports that the connection is lost.

`INetworkProtocolClient` owns one connection to a server.

- `GetConnection` always returns an object, but using it before `WaitForServer` finishes is undefined by contract.
- `WaitForServer` blocks until the connection is established or the implementation gives up.
- `GetStatus` reports `ClientStatus::Ready`, `ClientStatus::WaitingForServer`, `ClientStatus::Connected` or `ClientStatus::Disconnected`.

`INetworkProtocolServer` owns the listening side.

- `Start` begins accepting clients.
- `OnClientConnected` is called for each accepted transport connection and returns `WaitForClientResult::Accept` or `WaitForClientResult::Reject`.
- `Stop` ends listening and disconnects owned connections.
- `IsStopped` reports either explicit shutdown or an underlying transport failure.
- No client-connected callback should happen before `Start` or after `Stop`.

The normal raw-protocol usage pattern is:

1. Derive a server from a concrete `INetworkProtocolServer` implementation and override `OnClientConnected`.
2. In `OnClientConnected`, call `InstallCallback` on the new `INetworkProtocolConnection`, then call `BeginReadingLoopUnsafe`, and return `WaitForClientResult::Accept`.
3. Create each client, call `GetConnection()->InstallCallback`, call `WaitForServer`, then call `GetConnection()->BeginReadingLoopUnsafe`.
4. Exchange messages through `INetworkProtocolConnection::SendString`.
5. Call `Stop` and uninstall callbacks during shutdown.

## Channel API Contract

There is no separate `IChannelConnection` type. The channel layer models a connected participant with `IChannelClient<TPackage>`, its assigned positive client id, and its named `IChannel<TPackage>` objects.

`IChannelReader<TPackage>` receives typed channel packages.

- `OnRead` receives a positive `senderClientId` and one `TPackage`.

`IChannel<TPackage>` represents one named logical channel.

- `GetChannelName` returns the channel name.
- `GetReader` returns the installed reader.
- `Initialize` installs one reader and replays unread messages that arrived before the reader was installed.
- `Initialize` can only install a reader once; no reader uninstallation is supported.
- `SendToClient` queues a direct message to one receiver client id.
- `BroadcastFromClient` queues a broadcast to all other participants on the same channel.
- The `BroadcastFromClient` overload with `blockedReceivers` excludes specific receiver ids from a broadcast.
- `BatchWrite` flushes all queued messages and reports disconnection through its `disconnected` output argument.

Channel names:

- User channel names must be non-empty.
- User channel names must not contain `!`.
- Names beginning with `!` are reserved for system channels.
- `ErrorChannel` is used by the default implementation for fatal channel errors.
- `SystemChannel` is declared as reserved, but the default implementation currently does not use it.

`IChannelClient<TPackage>` represents one channel participant.

- `OnGetChannelNames` tells the implementation which channel names this client supports.
- `GetChannels` returns implementation-created channel objects. The channel map is expected to be empty before connection and populated by the implementation.
- `GetClientId` returns `-1` before connection and the server-assigned positive id after connection.
- `WaitForServer` blocks until connected for network clients. Local clients return immediately.
- `GetStatus` reports `Ready`, `WaitingForServer`, `Connected` or `Disconnected`.
- `OnConnected` and `OnDisconnected` report participant lifecycle.
- `OnReadError` receives fatal errors broadcast by the channel server.
- `OnLocalError` receives local transport errors.
- `BroadcastError` raises a fatal channel error.

`IChannelServer<TPackage>` manages connected channel participants.

- `Start` begins accepting channel clients.
- `OnClientConnected` receives the assigned client id, the client's channel names and a `localClient` pointer.
- The `localClient` argument is non-null only when the connection is created by `ConnectLocalClient`.
- `ConnectLocalClient` connects an in-process server-side participant and returns its assigned positive client id, or `-1` if it cannot connect.
- `IsLocalClient` reports whether a client id belongs to a local client.
- `DisconnectClient` disconnects either a network client or a local client.
- `GetClientIds` returns all known client ids.
- `GetClientChannels` returns the client-id to channel-name membership map.
- `BroadcastError` broadcasts a fatal channel error and stops the server.
- `Stop` disconnects clients and stops accepting.
- `IsStopped` reports explicit stop or underlying transport stop.

`IChannelServer<TPackage>` is a delivery and bookkeeping API. It is not itself a channel speaker. When server-side behavior needs to send channel messages, create a `NetworkProtocolLocalChannelClient<TPackage, TSerialization>`, connect it with `ConnectLocalClient`, and send messages through that local client's channels.

## Default Channel Implementation Over Raw Protocols

The default channel bridge serializes batches of typed packages to raw `WString` messages.

`TSerialization` for `NetworkProtocolChannel<TPackage, TSerialization>` must satisfy:

- `TSerialization::SourceType` is `collections::List<TPackage>`.
- `TSerialization::DestType` is `WString`.
- `TSerialization::ContextType` stores optional serializer context.
- `TSerialization::Serialize` converts a package list to `WString`.
- `TSerialization::Deserialize` converts a `WString` back to a package list.

`NetworkPackage` is the raw wire envelope.

- Its string format is `clientId,extraClientId1,...;channelName;messageBody`.
- `clientId` may be null.
- `extraClientIds` may be null.
- A null `clientId` with extra ids is preserved as a leading comma, for example `,1,2;Chat;Message`.
- Missing or empty `extraClientIds` normalize to null after parsing.

Connection handshake:

1. A channel client waits for the raw `INetworkProtocolClient`.
2. The client sends a `NetworkPackage` with null `clientId`, empty `channelName`, and `messageBody` equal to all supported channel names joined by `!`.
3. The server validates the first package as a connection request.
4. The server allocates a positive client id.
5. The server calls `OnClientConnected(clientId, channelNames, nullptr)` for network clients.
6. The server records channel membership.
7. The server sends a response package with the assigned `clientId`, empty `channelName`, and empty `messageBody`.
8. The client records the assigned id, changes status to connected, replays any queued channel packages, and calls `OnConnected`.

Channel message semantics:

- Client-to-server direct messages set `NetworkPackage::clientId` to the receiver client id.
- Client-to-server broadcasts use a null `clientId`.
- For broadcasts, `extraClientIds` means blocked receivers.
- For direct messages, `extraClientIds` are ignored by the delivery semantics.
- Server-to-client channel messages always set `NetworkPackage::clientId` to the positive sender client id.
- A received channel message without a positive sender id is invalid.

`NetworkProtocolChannel<TPackage, TSerialization>` handles per-channel queues.

- `ReadBatch` stores unread packages until `Initialize` installs a reader.
- `Initialize` replays unread packages to the reader.
- `SendToClient` and `BroadcastFromClient` validate ids and queue packages.
- `BatchWrite` moves queued packages out, groups them by receiver id and blocked-receiver list, serializes each group, and calls `WriteBatch`.
- If `WriteBatch` reports disconnection, `BatchWrite` sets `disconnected` and stops flushing.

`NetworkProtocolChannelClientBase<TPackage, TSerialization>` owns the shared client-side channel state.

- It owns serializer context, client id, client status, connection notification state and generated channels.
- `GetChannels` lazily creates channel objects from `OnGetChannelNames`.
- The generated channel validates that a client is connected before sending.
- Direct sends require a positive receiver id.
- Direct sends cannot also specify blocked receivers.
- Blocked receiver ids must be positive and must not be the sender's own id.

`NetworkProtocolChannelClient<TPackage, TSerialization>` adapts a real `INetworkProtocolClient`.

- The constructor installs an `INetworkProtocolCallback` on the raw connection.
- `WaitForServer` waits for the raw transport, sends the channel-name handshake, starts raw reading, waits for the assigned client id, and notifies `OnConnected`.
- Incoming `ErrorChannel` packages call `OnReadError` and disconnect.
- Incoming empty-channel packages are treated as connection responses.
- Incoming channel packages that arrive before the assigned id are queued and replayed after connection.
- `SendBatch` serializes one package list into one `NetworkPackage` and calls `SendString`.
- `BroadcastError` sends an `ErrorChannel` package and disconnects locally.
- The destructor stops the raw connection if still connected and uninstalls the callback.

`NetworkProtocolLocalChannelClient<TPackage, TSerialization>` adapts no transport.

- It connects only through `NetworkProtocolChannelServer::ConnectLocalClient`.
- `WaitForServer` is a no-op.
- `SendBatch` calls the server's local-client send hook.
- `BroadcastError` routes through the server when connected.
- If already disconnected, `BroadcastError` reports a fatal local error and disconnects locally.

`NetworkProtocolChannelServer<TPackage, TSerialization, TServerBase>` combines the channel server with a raw protocol server.

- `TServerBase` is a concrete `INetworkProtocolServer` implementation, such as `vl::inter_process::named_pipe::NamedPipeServer` or `vl::inter_process::windows_http::HttpServer`.
- The class inherits `TServerBase`, `IChannelServer<TPackage>` and the private local-client server interface.
- `Start` records the channel server as started, then calls `TServerBase::Start`.
- Raw `OnClientConnected(INetworkProtocolConnection*)` rejects connections before start or after stop, creates a pending connection context, installs a raw callback, starts raw reading, and waits for the channel handshake.
- Before a connection has a client id, `OnReadString` requires the channel-name handshake.
- After a connection has a client id, `OnReadString` validates sender membership, receiver membership or blocked-receiver membership, deserializes the batch and forwards it through `SendBatch`.
- `SendBatch` handles direct delivery and broadcast delivery for both network clients and local clients.
- Network recipients receive serialized `NetworkPackage` strings.
- Local recipients receive packages through their local channel objects.
- `ConnectLocalClient` requires a started, non-stopped server and a `NetworkProtocolLocalChannelClient<TPackage, TSerialization>`.
- `ConnectLocalClient` validates local channels, allocates a positive id, calls `OnClientConnected(clientId, channels, localClient)`, records membership and notifies the local client.
- `DisconnectClient` removes either a network client or a local client, stops or notifies it, and calls `OnClientDisconnected`.
- `Stop` clears network, pending and local client state, notifies local clients, delegates transport shutdown to `TServerBase::Stop`, and reports network disconnections.
- `BroadcastError` sends `ErrorChannel` packages to network clients, calls `OnReadError` on local clients, gives transport clients a short chance to consume the fatal package, and stops the server.

## Channel Usage Pattern

To build a typed channel application over a raw transport:

1. Define a serializer whose `SourceType` is `collections::List<TPackage>` and whose `DestType` is `WString`.
2. Define a channel server by using `NetworkProtocolChannelServer<TPackage, TSerialization, TServerBase>`, where `TServerBase` is the chosen raw transport server.
3. Override `OnClientConnected(vint, const ChannelNameList&, IChannelClient<TPackage>*)` to accept or reject clients after checking their channel names.
4. Define channel clients by deriving from `NetworkProtocolChannelClient<TPackage, TSerialization>` for network clients.
5. For each client, return supported channel names from `OnGetChannelNames`.
6. Obtain channels through `GetChannels`, call `Initialize` with an `IChannelReader<TPackage>`, then call `WaitForServer`.
7. Queue channel messages with `SendToClient` or `BroadcastFromClient`.
8. Call `BatchWrite` to flush queued channel messages.
9. If the server needs to speak on a channel, derive that participant from `NetworkProtocolLocalChannelClient<TPackage, TSerialization>` and connect it with `ConnectLocalClient`.

## Error and Shutdown Rules

- Keep remote channel errors and local transport errors separate.
- `IChannelClient<TPackage>::OnReadError` is for fatal errors broadcast through the channel server.
- `IChannelClient<TPackage>::OnLocalError` is for local raw-transport failures, such as connection, request or response failures.
- `INetworkProtocolCallback::OnReadError` is for remote errors at the raw transport contract level.
- `INetworkProtocolCallback::OnLocalError` is for local transport failures at the raw transport contract level.
- A fatal local error should lead to disconnection.
- `Stop` should be treated as a hard shutdown boundary: after it returns, transport callbacks should no longer touch the stopped object.
- `BatchWrite` may report disconnection; callers should stop assuming queued messages were delivered once `disconnected` becomes true.
- Server-originated normal channel messages should use a connected local channel client so all normal messages have a real positive sender id.

## Current Windows-Only Raw Transports

The built-in `vl::inter_process::named_pipe::NamedPipeServer` / `vl::inter_process::named_pipe::NamedPipeClient` and `vl::inter_process::windows_http::HttpServer` / `vl::inter_process::windows_http::HttpClient` classes are currently Windows-only implementations of the raw protocol interfaces. They should not be treated as cross-platform transport classes.

### `vl::inter_process::named_pipe::NamedPipeServer` and `vl::inter_process::named_pipe::NamedPipeClient`

`vl::inter_process::named_pipe::NamedPipeConnection` implements `INetworkProtocolConnection`. `vl::inter_process::named_pipe::NamedPipeClient` derives from `vl::inter_process::named_pipe::NamedPipeConnection` and implements `INetworkProtocolClient`. `vl::inter_process::named_pipe::NamedPipeServer` implements `INetworkProtocolServer`.

- `NamedPipeServer::Start` begins overlapped named-pipe accepting.
- `NamedPipeServer` keeps both accepted `NamedPipeConnection` objects and pending `ConnectNamedPipe` operations.
- `NamedPipeServer::OnClientConnected` is the user-overridable accept hook.
- `NamedPipeClient::WaitForServer` completes the client-side setup, switches the pipe to message-read mode, updates status to `ClientStatus::Connected`, and calls callback `OnConnected`.
- `NamedPipeConnection::SendString` frames one `WString` with byte count and string length data.
- `NamedPipeConnection` chunks writes by `MaxMessageSize`, because the named-pipe implementation does not support one message larger than 64K.
- `NamedPipeConnection::BeginReadingLoopUnsafe` accumulates overlapped read chunks until one complete framed message is available, then calls `OnReadString`.
- Broken-pipe cases become `OnDisconnected`; invalid handle or aborted I/O cases become fatal `OnLocalError` followed by disconnect.
- `NamedPipeConnection::Stop` cancels pending overlapped pipe I/O, unregisters pending waits, waits for pending callbacks, and closes handles.
- `NamedPipeServer::Stop` drains both pending accepts and accepted connections.

### `vl::inter_process::windows_http::HttpServer` and `vl::inter_process::windows_http::HttpClient`

`vl::inter_process::windows_http::HttpClient` implements both `INetworkProtocolConnection` and `INetworkProtocolClient`. `vl::inter_process::windows_http::HttpServer` derives from `vl::inter_process::windows_http::HttpServerApi` and implements `INetworkProtocolServer`. This is also a Windows-only reference/demo implementation.

The HTTP protocol uses three routes:

- `GET /VlppInterProcess/Connect` creates a logical connection and returns a pair of per-connection URLs.
- `POST /VlppInterProcess/Request/GUID` is the client-maintained long-poll request for server-to-client messages.
- `POST /VlppInterProcess/Response/GUID` sends client-to-server messages and may also receive one server-to-client message in the response.

`HttpClient` behavior:

- `HttpClient::WaitForServer` sends `/Connect`, waits for the response, validates the `requestUrl;responseUrl` body, stores both URLs, changes status to connected, and calls `OnConnected`.
- `HttpClient::BeginReadingLoopUnsafe` starts the long-poll `/Request` loop.
- After each successful `/Request`, `HttpClient` starts the next `/Request` before delivering a non-empty response body to `OnReadString`.
- `HttpClient::SendString` sends a `/Response` request with the string body.
- `/Connect` and `/Response` failures retry up to `HttpRequestMaxAttempts`; the last failure is fatal.
- `/Request` failures are retried while the client is still running.
- `HttpClient::Stop` stops the underlying `HttpClientApi`, signals any waiting `WaitForServer`, and reports `OnDisconnected`.

`vl::inter_process::windows_http::HttpServer` and `vl::inter_process::windows_http::HttpServerConnection` behavior:

- On `/Connect`, `HttpServer` creates a `HttpServerConnection`, assigns a GUID, calls `OnClientConnected`, and returns the per-connection request and response URLs, or rejects with an HTTP error response.
- `HttpServerConnection::BeginReadingLoopUnsafe` is a no-op because `HttpServerApi` owns the receive loop.
- `HttpServerConnection::InstallCallback` stores the callback and replays queued inbound strings after releasing its queue lock.
- `HttpServerConnection::SendString` responds to a pending `/Request` immediately when possible; otherwise it queues outbound messages.
- If a server message is produced while handling `/Response`, `HttpServerConnection` may return one outbound message in that `/Response` response and queue the rest for future `/Request` calls.
- `HttpServerConnection::OnNewHttpRequestForPendingRequest` cancels an old pending `/Request`, stores the new one, and replies immediately if a message is already queued.
- `HttpServerConnection::SubmitResponse` reads the client body, dispatches or queues it as inbound text, then returns one queued outbound message if available.
- `HttpServer::OnHttpServerStopping` clears connections, cancels pending long-poll requests, disconnects connection objects from the server, and calls `OnDisconnected`.

## Windows HTTP Helper Layer

`vl::inter_process::windows_http::HttpClientApi` and `vl::inter_process::windows_http::HttpServerApi` are reusable Windows helper classes used by `vl::inter_process::windows_http::HttpClient` and `vl::inter_process::windows_http::HttpServer`. The supporting `HttpRequest`, `HttpResponse`, `HttpError` and `HttpServerResponse` value types are in the same `vl::inter_process::windows_http` namespace. Use the raw transport classes only when the reference/demo `INetworkProtocol*` shape is desired; use the helper APIs directly when a Windows-specific feature needs lower-level HTTP request/response behavior.

`HttpClientApi` owns one WinHTTP session and connection for one host and port.

- `HttpClientApi::HttpQuery` sends one asynchronous HTTP request.
- `HttpRequest` describes method, query, body, content type, accept types, credentials, cookies, headers, timeouts and `keepAliveOnStop`.
- `HttpRequest::SetBodyUtf8` converts a `WString` request body to UTF-8 bytes.
- `HttpResponse` carries HTTP status code, response body bytes, cookie and content type.
- `HttpResponse::GetBodyUtf8` converts a UTF-8 response body back to `WString`.
- `HttpError` represents Windows or WinHTTP transport failures.
- HTTP status codes such as 404 are represented as successful `HttpResponse` values, not `HttpError`.
- `HttpClientApi::Stop` marks the helper as stopping, closes active non-keep-alive requests, waits for request handle-closing callbacks, then closes WinHTTP handles.

`HttpServerApi` owns one HTTP.sys URL prefix.

- `HttpServerApi::Start` begins the receive loop.
- `HttpServerApi::Stop` unregisters the pending receive wait, waits for callbacks, calls `OnHttpServerStopping`, and closes HTTP.sys handles.
- `OnHttpRequestReceived` is the virtual request-dispatch hook.
- `OnHttpServerStopping` is the virtual shutdown hook.
- `GetUtf8Body` validates `application/json; charset=utf8`, reads the complete request body and converts it to `WString`.
- `SendResponse` sends a structured `HttpServerResponse`.
- `SendResponseUtf8` sends a UTF-8 body with JSON content type.
- Optional `OPTIONS` handling exists for cross-origin HTTP clients.