Summary

• lean & very fast GRPC-like services on JVM with rich streaming model over multiple data-center and internet transports;
• multiple APIs: CompletableFuture & virtual threads; traditional streaming GRPC-stub (StreamObserver), or reactive with smallrye-mutiny, rxjava, reactor
• pluggable networking: tcp, unix sockets, vm sockets; grpc, websockets, websockets-over-http2;
• service APIs / codegen stubs (Message-Streams) are split from library runtime (RSocket-JVM, including network transports, load estimators, load balancers, metrics)
• transparent origin (RPC) & proxy load estimation for low-cost yet efficient load balancers;
• native image support with graalvm.

Project consists of API modules for each vendor library - Message-Streams, and respective runtime implementations - RSocket-JVM.

Message-Streams includes RPC: code-generation based remote procedure call system on top of Protocol Buffers.

Message-Streams sources are available on github, and binaries are published on MavenCentral.

RSocket is low latency/high throughput L5 network protocol intended for high-performance services communication. It is transport agnostic and may run on top of any reliable byte stream transport.

Fast alternative RSocket on JVM

Several problems were present in official RSocket/RSocket-java library:

• low performance and excessive garbage production, as described in 1 million streams. part2. Official RSocket/RSocket-java/RSocket-RPC performed similar to Http2/netty based GPRC, and did not scale on big hosts.
• no reusable code for other vendor implementations: core features, transports, metrics were coded with Pivotal libraries (e.g. project-reactor, reactor-netty and micrometer).
• vulnerabilities outlined in 1 million streams. part1, recently continued with 1, 2, 3, 4 - 3 of 4 interactions (all except fire-and-forget) may be used for trivial denial-of-service of RSocket/RSocket-java servers.
• RSocket/RSocket-RPC abandoned in favor of “Spring way” of data format abstraction, and reflection at runtime instead of code generation.

Message Streams with flow control

Protocol offers rich set of interactions modelled as composable streams of binary messages, with flow control, error handling and cancellation as first-class concepts:

fire-and-forget (1:0), request-response (1:1), request-stream (1:n), request-channel (n:n).

Interactions are exposed to end user with MessageStreams API.

Traditional streaming

GRPC-stubs (StreamObserver) API support allows drop-in replacement of GRPC-java in services for significantly better throughput.

interface MessageStreams {
  void requestResponse(Message message, StreamObserver<Message> responseObserver);
  void requestStream(Message message, StreamObserver<Message> responseObserver);
  StreamObserver<Message> requestChannel(StreamObserver<Message> responseObserver);
  void fireAndForget(Message message, StreamObserver<Message> responseObserver);
}  

Reactive streams

Reactive Streams conformance makes protocol compatible with project-reactor, smallrye-mutiny and rxjava3

interface MessageStreams {
  Publisher<Message> requestResponse(Message message);
  Publisher<Message> requestStream(Message message);
  Publisher<Message> requestChannel(Publisher<Message> messages);
  Publisher<Void> fireAndForget(Message message);
}

This library demonstrates throughput of up to millions of messages per second / per vCPU with every interaction - more details below.

RSocket-JVM

Library relies on shared protocol core with minimal dependencies (only netty-buffer) for use with vendor libraries on JVM.

RSocket-JVM is currently comprised of JDK only RSocket-futures for request-response (CompletableFuture), traditional streaming with RSocket-GRPC (GRPC-stub), several flavors of reactive: RSocket-rxjava (rxjava3), RSocket-reactor (project-reactor) and RSocket-mutiny (smallrye-mutiny).

Notable features:

• substantially higher performance, tiny per message garbage production. It is capable of serving 1 million of simultaneous streams with mid-range commodity PC (1 million streams.part2).

         message size,bytes    8(same payload)        8             128            512

request-response                     2.2m            1.7m           1.6m          1.3m
request-stream                       3.5m            3.2m           3.2m          3.2m
request-channel                      4.8m            3.2m           3.1m          2.4m

Table 1. Single vCPU RSocket throughput (project-reactor, non-tls, TCP EPOLL with linux 5.4.0, jdk11), millions of messages per second.

• requests leasing for circuit breaking, adaptive latency-based load estimation that enables efficient lightweight load balancers.
• request ranks concept, used by load estimator to assist with load balancing
• graceful close of both clients and server, scalable server keep-alive support, lightweight coarse scheduler.
• built-in metrics model reworked so only tiny, practically useful set of metrics is published.

GRPC compatibility

jauntsdn/RSocket and jauntsdn/RSocket-RPC is compatible with GRPC: protocol design allows its streams to accommodate http2 streams. Compatibility was verified with RSocket-GRPC transport, observed performance characteristics were identical to official grpc-java only setup.

RPC for application services

jauntsdn/RSocket-RPC is RPC system based on Message-Streams & Protocol Buffers. It relies on code generation with protobuf C++ plugin extensions and is compatible with GRPC.

Notable features:

• optimal off-heap buffers utilization, compact rpc call encoding, default service concept for small request on-wire overhead
• request ranks, which are used by load estimator and assist with load balancing
• idempotent requests
• simple metadata in form of ASCII encoded key-value pairs (headers)
• instrumentation with no binary dependency on Micrometer or OpenTracing

The result is significantly higher performance: millions of messages per second per core, with each interaction.

         message size,bytes          8             128            512

request-response                    1.45m          1.0m          0.55m
request-stream                      3.3m           2.4m          0.9m
request-channel                     3.5m           2.4m          1.25m

Table 2. Single vCPU RSocket-RPC throughput (project-reactor, non-tls, TCP EPOLL with linux 5.4.0, jdk11), millions of messages per second.

Multiple network transports

RSocket is transport agnostic, and runs on top of any reliable byte stream transport. jauntsdn/RSocket project hosts several transports, shared & directly usable by each RSocket-JVM vendor specific library:

• TCP, unix domain sockets transports: known efficient byte stream protocols for datacenter / inter-process communication.

• Http based transports for interop:

  • GRPC-RSocketRPC server transport for communication with internet clients / external services.

  • WebSocket-over-http1 (rfc6455), WebSocket-over-http2 (rfc8441) for cross-cloud communication with Internet Standards transports.

Load estimation

Network/service latency based load estimation for RSocket & RSocket-RPC with requests leasing, which enables low-cost yet efficient load balancers.

Source code/binaries

Message-Streams (RSocket-JVM API) sources are available on github, and binaries are published on MavenCentral.