logo

Introduction to Phoenix Project

Phoenix is a dataplane service which serves as a framework to develop and deploy various kinds of managed services.

The key features of Phoenix include:

  • Modular plugin systems: engines can be developed as plugins, dynamically load into the Phoenix service at runtime, and cam be live upgraded without distributing user applications.
  • Low-latency networking with kernel bypassing: Phoenix service can utilize userspace solutions like RDMA and DPDK for high throughput and low latency networking. (DPDK support will be added in future releases.)
  • Policy management: Phoenix provides support for application-level policies that infrastructure administers could specify to control the behaviours and resources usages of user applications.

mRPC

mRPC, (stands for managed RPC service), is built on top of Phoenix as an experimental feature. mRPC implements a novel RPC architecture that decouples marshalling/unmarshalling from tranditional RPC libraries into a centralized system service.

Compared to traditional library + sidecar solutions such as gRPC + Envoy, mRPC applies network policies and observability features with both security and low performance overhead, i.e., with minimal data movement and no redundant (un)marshalling. The mechanism supports live upgrade of RPC bindings, policies, transports, and marshalling without disrupting running applications.

mRPC Tutorials

This chapter covers tutorials of how to develop user applications with mRPC library, and how administers can manage applications and apply policies with the mRPC service.

Working with mRPC Library

This tutorial describes how to develop user applications with mRPC library.

Overview

mRPC library provides interfaces similar to tonic, which is a Rust implementation of gRPC. mRPC Rust library is built to have first class support of the async ecosystem. Protocol Buffers, which are used by gRPC to specify services and messages, are also applied by mRPC. mRPC library's codegen generates client and server stubs based on the protobuf specs.

Defining the HelloWorld service

The first step is to define a mRPC service with protobuf. Let us first create a new Cargo project for our hello world demo.

cargo new rpc_hello

Then, let's create the .proto file for the server and client to use.

cd rpc_hello
mkdir proto
touch proto/rpc_hello.proto

Now, we define a simple hello world service:

syntax = "proto3";

package rpc_hello;

// The greeting service definition.
service Greeter {
    // Sends a greeting
    rpc SayHello (HelloRequest) returns (HelloReply) {}
}

// The request message containing the user's name.
message HelloRequest {
    bytes name = 1;
}

// The response message containing the greetings
message HelloReply {
    bytes message = 1;
}

Here we defined a Greater service, with SayHello RPC call. SayHello takes a HelloRequest, which contains a series of bytes, and returns a HelloReply of bytes. Now, our .proto file is ready to go.

Setup applications

We setup Cargo.toml to include mRPC library and other required dependencies, and specify a client binary and a server binary to compile.

[package]
name = "rpc_hello"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[build-dependencies]
mrpc-build = { path = "../../mrpc/mrpc-build" }

[dependencies]
mrpc = { path = "../../mrpc" }
prost = { path = "../../3rdparty/prost", features = ["mrpc-frontend"] }
structopt = "0.3.23"
smol = "1.2.5"

[[bin]]
name = "rpc_hello_client"
path = "src/client.rs"

[[bin]]
name = "rpc_hello_server"
path = "src/server.rs"

Generate client and server stubs

Before implementing the client and server, we need to write a build script to invoke mrpc-build to compile client and server stubs. Create a build.rs in rpc_hello/ with the following code:

const PROTO: &str = "proto/rpc_hello.proto";
fn main() -> Result<(), Box<dyn std::error::Error>> {
    println!("cargo:rerun-if-changed={PROTO}");
    mrpc_build::compile_protos(PROTO)?;
    Ok(())
}

Writing the server

Create a file server.rs. We can start by including our hello world proto's messages types and server stub:

#![allow(unused)]
fn main() {
pub mod rpc_hello {
    // The string specified here must match the proto package name
    mrpc::include_proto!("rpc_hello");
}

use rpc_hello::greeter_server::{Greeter, GreeterServer};
use rpc_hello::{HelloReply, HelloRequest};
}

Now, we need to implement the Greeter service we defined. In particular, we need to implement how say_hello should be handled by the server.

#![allow(unused)]
fn main() {
#[mrpc::async_trait]
impl Greeter for MyGreeter {
    async fn say_hello(
    &self,
    request: RRef<HelloRequest>,
        ) -> Result<WRef<HelloReply>, mrpc::Status> {
            eprintln!("request: {:?}", request);

            let message = format!("Hello {}!", String::from_utf8_lossy(&request.name));
            let reply = WRef::new(HelloReply {
                message: message.as_bytes().into(),
            });

            Ok(reply)
        }
    }
}

Here, RRef<HelloRequest> wraps the received HelloRequest on the receive heap of the shared memory with the mRPC service. RRef provides read-only access to the wrapped message. Copy-on-write can be performed if the application wants to modify a received message. WRef, wraps a RPC message to be sent on the send heap of the shared memory.

With Greeter service implemented, we can implement a simple server that runs our Greeter service with smol runtime (which is a small and fast async runtime).

fn main() -> Result<(), Box<dyn std::error::Error>> {
    smol::block_on(async {
        let mut server = mrpc::stub::LocalServer::bind("0.0.0.0:5000")?;
        server
			.add_service(GreeterServer::new(MyGreeter::default()))
			.serve()
			.await?;
        Ok(())
    })
}

The complete code for the server is:

pub mod rpc_hello {
    // The string specified here must match the proto package name
    mrpc::include_proto!("rpc_hello");
}

use rpc_hello::greeter_server::{Greeter, GreeterServer};
use rpc_hello::{HelloReply, HelloRequest};

use mrpc::{RRef, WRef};

#[derive(Debug, Default)]
struct MyGreeter;

#[mrpc::async_trait]
impl Greeter for MyGreeter {
    async fn say_hello(
		&self,
		request: RRef<HelloRequest>,
    ) -> Result<WRef<HelloReply>, mrpc::Status> {
        println!("request: {:?}", request);

        let message = format!("Hello {}!", String::from_utf8_lossy(&request.name));
        let reply = WRef::new(HelloReply {
            message: message.as_bytes().into(),
        });

        Ok(reply)
    }
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    smol::block_on(async {
        let mut server = mrpc::stub::LocalServer::bind("0.0.0.0:5000")?;
        server
			.add_service(GreeterServer::new(MyGreeter::default()))
			.serve()
			.await?;
        Ok(())
    })
}

Writing the client

We can write a client to send a single request to server and get the reply.

pub mod rpc_hello {
    // The string specified here must match the proto package name
    mrpc::include_proto!("rpc_hello");
}

use rpc_hello::greeter_client::GreeterClient;
use rpc_hello::HelloRequest;


fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = GreeterClient::connect("server-addr:5000")?;
    let req = HelloRequest {
        name: "mRPC".into(),
    };
    let reply = smol::block_on(client.say_hello(req))?;
    println!("reply: {}", String::from_utf8_lossy(&reply.message));
    Ok(())
}

With the client stub, we can just directly call client.say_hello(req) to send a request. It will return a Future that we can await on, which resolves to a Result<RRef<HelloReply>, mrpc::Status>. The HelloRequest and HelloReply types in the generated code internally uses our customized Rust collection types, e.g., mrpc::alloc::Vec<u8>, where it provides similar API as its std counterpart but directly allocates buffers on shared memory.

Running the demo

First, start mRPC services on the machines that we will run the client and the server:

cd experimental/mrpc
cat load-mrpc-plugins.toml >> ../../phoenix.toml
cargo make

Note that mRPC service needs to run on both the client side and the server side. After spinning up mRPC services, we can run the client and server applications:

cd experimental/mrpc/examples/rpc_hello
cargo run --release --bin rpc_hello_server
cargo run --release --bin rpc_hello_client

Policy Management

This tutorial describes how network administrators can apply policies to running applications.

First, start Phoenix service and load mRPC modules.

cargo make run
cargo run --release --bin upgrade -- --config experimental/mrpc/load-mrpc-plugins.toml

Then run the user applications. Here we use rpc_bench as the example. The rpc_bench_client will run for 60 seconds and print the RPC rate every second.

cd experimental/mrpc/
cargo run --release -p rpc_bench --bin rpc_bench_server
cargo run --release -p rpc_bench --bin rpc_bench_client -- -D 60 -i 1 --req-size 64 -c <server_addr>

In phoenixctl/src/bin, we have a series of utilities for network administrators to interact with mRPC service. You can compile all the phoenix-cli tools by

cargo make build-phoenix-cli

To apply a policy to an application, we must first retrieve information regarding it in mRPC service. list is a utility used to list all engines running in mRPC service, along with the corresponding user process the engine serves. The administrator can simply run:

cargo run --release --bin list

It will output a summary of running engines like the following:

+---------+-----+---------+--------+------------------------------------+
| PID     | SID | Service | Addons | Engines                            |
+---------+-----+---------+--------+------------------------------------+
| 2012290 | 0   | Salloc  | None   | +----------+--------------+        |
|         |     |         |        | | EngineId | EngineType   |        |
|         |     |         |        | +----------+--------------+        |
|         |     |         |        | | 0        | SallocEngine |        |
|         |     |         |        | +----------+--------------+        |
+---------+-----+---------+--------+------------------------------------+
| 2012290 | 1   | Mrpc    | None   | +----------+---------------------+ |
|         |     |         |        | | EngineId | EngineType          | |
|         |     |         |        | +----------+---------------------+ |
|         |     |         |        | | 2        | MrpcEngine          | |
|         |     |         |        | +----------+---------------------+ |
|         |     |         |        | | 1        | TcpRpcAdapterEngine | |
|         |     |         |        | +----------+---------------------+ |
+---------+-----+---------+--------+------------------------------------+

The above listing tells us there is a single user application with PID 2012290. The application has two engine subscriptions, one of it is the mRPC engine (MrpcEngine), which handles sending and receiving of RPC messages on the application's behalf. The other (SallocEngine) is for allocating shared memory.

Policies will be applied on the mRPC engine subscription, which has a subscription ID (SID) 1 here. Each policy is implemented as an engine. To apply a policy, we need a descriptor file to specify which policy engine to attach, where the policy engine is inserted, and the configuration of the policy (a configuration string).

For instance, to apply a rate limit policy, we have the following descriptor file:

addon_engine = "RateLimitEngine"
tx_channels_replacements = [
    ["MrpcEngine", "RateLimitEngine", 0, 0],
    ["RateLimitEngine", "TcpRpcAdapterEngine", 0, 0],
]
rx_channels_replacements = []
group = ["MrpcEngine", "TcpRpcAdapterEngine"]
op = "attach"
config_string = '''
requests_per_sec = 1000
bucket_size = 1000
'''

Here, we specify that the rate limit engine should be inserted between MrpcEngine and TcpRpcAdapterEngine. We also specify the rate should be limited at 1000 requests per second.

Then to apply this rate limit policy to the application, the administrator can use addonctl utility, passing in the descriptor file, PID and SID.

cargo run  --release --bin addonctl -- --config eval/policy/ratelimit/attach.toml --pid 2012290 --sid 1

Removing a policy can be achieved in a similar fashion. For the above rate limit policy, we have the following descriptor file to detach the policy, which removes the RateLimitEngine:

addon_engine = "RateLimitEngine"
tx_channels_replacements = [
    ["MrpcEngine", "TcpRpcAdapterEngine", 0, 0],
]
rx_channels_replacements = []
op = "detach"

After the following command is executed, rate limit policy is no longer applied.

cargo run  --release --bin addonctl -- --config eval/policy/ratelimit/detach.toml --pid 2012290 --sid 1

Semantics

The engines can form a graph, and are connected via unidirectional tx/rx channels.

TX/RX represents the direction of message. For example, if a client sends an RPC to a server, the client's tx channel is connected to the server's rx channel. Server's tx channel is connected to the client's rx channel, sending the response of that RPC.

Currently, for each application, all engine's name must be unique.

Attach

Attach means inserting a new engine between two existing engines. The new engine will be inserted between the two existing engines, and the two existing engines will be connected to the new engine via tx/rx channels.

The new engine should be attached to the same group as the two existing engines, as engines in the same group are bound to a single thread.

  • addon_engine: the engine to be attached
  • tx_channels_replacements: You need to specify the channel descriptor after addon_engine has been attached. Suppose you want to insert A between B and C, then tx_channels_replacements should be [[B, A, 0, 0], [A, C, 0, 0]].
  • rx_channels_replacements: Similar to tx_channels_replacements, but for rx channels, and the order of the channels should be reversed.
  • group: the group of the all existing engines, suppose you want to also insert D between A and B, then the group should be [A, B, C].

Detach

Detach means removing an existing engine from the graph. The engine will be removed from the graph, and the two existing engines will be connected to each other via tx/rx channels.

  • addon_engine: the engine to be detached
  • tx_channels_replacements: You need to specify the channel descriptor after addon_engine has been detached. Suppose you want to remove A between B and C, then tx_channels_replacements should be [[B, C, 0, 0]].
  • rx_channels_replacements: Similar to tx_channels_replacements, but for rx channels, and the order of the channels should be reversed.