NPRPC docs

Calls and cancellation

Every call involves two separate choices: how the client waits for the reply, and how the servant runs. The two sides don't know what the other picked, and the wire format is the same for all of them, so any client style works with any servant style.

The examples below use this interface:

interface Report {
  string Summary(id: u32) raises (NotFound);
  string Build(id: u32) raises (NotFound);
  [coroutine] string Fetch(id: u32) raises (NotFound);
}

Quick guide

The method... Declare it Run it on
Finishes in microseconds Plain The default POA
Does long CPU work, or needs a particular thread Plain A POA that hands off, checking the token
Waits on I/O or another service [coroutine] Any POA

Cancelling always frees the client. Only the servant style decides whether the server stops working as well.

Client: how to wait for the reply

Style Languages Use for Can cancel
Blocking C++ Simple code, lowest overhead No
Suspending C++, Swift, TypeScript Many calls at once without a thread for each Yes

Blocking

Only C++ has this style. The thread waits until the reply arrives or the call times out.

std::string text = report->Summary(7);

Suspending

C++ generates an Async version of each method, except stream and [unreliable] ones. Swift and TypeScript client methods are always async.

nprpc::Task<std::string> both(nprpc::ObjectPtr<Report>& report) {
  auto first = co_await report->BuildAsync(7);
  auto second = co_await report->FetchAsync(8);
  co_return first + second;
}
async let first = report.build(id: 7)
async let second = report.fetch(id: 8)
let text = try await first + second

Cancelling a call

Each language has its own way to cancel. In C++, pass a std::stop_token to the Async method. The call ends with nprpc::OperationCancelled as soon as you request a stop.

std::stop_source stop;
auto task = report->FetchAsync(7, stop.get_token());

stop.request_stop();

try {
  auto text = task.get();
} catch (const nprpc::OperationCancelled&) {
  // The call was cancelled.
}

In Swift, cancel the task. The call throws CancellationError.

let task = Task { try await report.fetch(id: 7) }

task.cancel()

do {
  _ = try await task.value
} catch is CancellationError {
  // The call was cancelled.
}

In TypeScript, pass an AbortSignal as the last argument. The call rejects with an AbortError as soon as you abort.

const controller = new AbortController();
const call = report.Fetch(7, controller.signal);

controller.abort();

try {
  await call;
} catch (e: any) {
  if (e.name !== 'AbortError') throw e;
}

The servant is told over WebSocket and WebTransport. A TypeScript call made through .http ends for the client, but the servant isn't told.

Timeouts

A call fails with ExceptionTimeout if no reply arrives within the object's timeout, which is 1 second by default. The servant isn't told, in any language, so it keeps running. For a TypeScript call through .http, the time includes connecting, because fetch doesn't report when it has connected. A method that takes longer than this needs a longer timeout on the client:

report->set_timeout(30'000);
report.timeout = 30000;

Servant: how the method runs

Style Lets you Told about a cancel Holds up that client's other calls
Plain, on the default POA Get the lowest latency No Yes
Plain, on a POA that hands off Do long work without blocking the transport, or stay on one thread Yes Yes
[coroutine] Wait without holding anything up Yes No

POA.md lists the POA setups that hand off.

Plain, on the default POA

The method runs on the transport thread, which can't read anything else until the method returns. A cancel therefore arrives too late to matter, and there's no token worth checking. Keep these methods short.

std::string ReportImpl::Summary(uint32_t id) {
  return summaries.at(id);
}

Plain, on a POA that hands off

The transport keeps reading while the method runs, so it sees the cancel and stops the token. The method has to check the token itself.

std::string ReportImpl::Build(uint32_t id) {
  const auto stop = nprpc::get_call_context().stop_token;
  std::string out;
  for (auto& part : parts(id)) {
    if (stop.stop_requested()) throw nprpc::OperationCancelled{};
    out += render(part);
  }
  return out;
}
override func build(id: UInt32) throws -> String {
    var out = ""
    for part in parts(id) {
        if currentCallIsCancelled { break }
        out += render(part)
    }
    return out
}

Throwing OperationCancelled is how a C++ method skips sending its result. A Swift method can't do that, so it returns early, and the client ignores whatever it sends.

Later calls from the same client wait until this method returns, even after the client has cancelled it. A client that cancels and retries still waits for the original call to end, so a method that doesn't check the token delays the retry.

[coroutine]

The method suspends while it waits. The session goes on to the next call, and the reply goes out when it's ready, so one client's calls can finish in any order.

nprpc::Task<std::string> ReportImpl::Fetch(std::stop_token stop, uint32_t id) {
  auto rows = co_await backend->BuildAsync(id, stop);
  co_return render(rows);
}
override func fetch(id: UInt32) async throws -> String {
    render(try await backend.build(id: id))
}

In C++, passing stop along cancels the nested call too. In Swift, the nested call is cancelled with the task. Don't use blocking calls inside these methods: they hold up the thread that other calls need.

POA.md covers threads, restrictions and limits.

What a cancel does on the server

The client has already moved on in every case here. It ignores any reply that arrives later.

Where the call is What happens What goes back
Queued, not started The servant never runs Error_Cancelled
Handed off, throws OperationCancelled The token stops, and the method ends early Error_Cancelled
Handed off, ignores the token The token stops, and the method runs to the end The full reply
Running on the default POA Nothing, because the method returns before the transport reads the cancel The full reply
[coroutine] stop fires, or the Swift task is cancelled Whatever the method returns or throws

The full reply still crosses the wire, so a large one costs the same as if the call hadn't been cancelled.

When the client disconnects

A disconnect stops the running call, and any [coroutine] calls, the same way a cancel does. Calls still queued do run, with their tokens already stopped, because some may be one-way. A method on the default POA runs to the end.

Transports

Handoff and [coroutine] work over TCP, WebSocket, WebTransport, QUIC and shared memory. HTTP RPC and the raw use_epoll_tcp and use_uring_tcp servers keep their thread until the method finishes, so a call on them can't be cancelled while it runs.

Large messages over shared memory

Each RPC message has to fit in one ring entry, 512 KiB by default. A request or reply over the limit fails the call straight away, and so does one that still can't get into a full ring after 100 ms. The connection stays up. For anything large, use a stream or pass a file descriptor, or raise the limit with RpcBuilder::shm_channel_sizes.

When a call fails in transport

ExceptionCommFailure says why. reason is for code to act on, and message is for people, with sizes and limits where they apply.

reason Meaning
Unknown Not known, as when an older server sent the error
ConnectionFailed Couldn't connect, or the connection was lost
MessageTooLarge The request or reply is over the transport's limit
RingFull A shared memory ring stayed full, so the peer has probably stopped reading
TooManyCalls The session already has 1024 [coroutine] calls running
try {
  auto image = images->Get(id);
} catch (const nprpc::ExceptionCommFailure& e) {
  if (e.reason == nprpc::CommFailureReason::MessageTooLarge)
    return fetch_as_stream(id);
  log(e.what());
}
do {
    return try await images.get(id: id)
} catch let e as ExceptionCommFailure where e.reason == .MessageTooLarge {
    return try await fetchAsStream(id)
}
try {
  return await images.Get(id);
} catch (e) {
  if (e instanceof NPRPC.ExceptionCommFailure &&
      e.reason === NPRPC.CommFailureReason.MessageTooLarge)
    return fetchAsStream(id);
  throw e;
}

In C++, what() returns an exception's message field if it has one and it isn't empty, so any IDL exception with a message: string works this way. Otherwise what() returns the type's name.

docs/CALLS.md