Penumbra will use ABCI to talk to Tendermint. ABCI is the interface between Tendermint (a consensus engine for BFT replication of a state machine), and an arbitrary application (the state machine to be replicated). The ABCI interface consists of a set of requests and responses the consensus engine makes to drive the application state.
This means that Penumbra needs an ABCI interface. Existing Rust interfaces to ABCI require application developers to write their own ad-hoc synchronization logic to handle concurrent ABCI requests. This makes it difficult to get up-and-running, and difficult to correctly handle concurrent requests.
To address this,
tower-abci is a new design for an asynchronous ABCI
interface using Tower. Tower is a library of modular components for
building networking clients and servers. Tower defines a core abstraction,
Service trait, which represents an asynchronous function with
backpressure, and then provides combinators that allow generic composition of
additional behavior, e.g., timeouts, buffering, load-shedding, rate-limiting,
tower-abci crate has two parts:
An ABCI server, which listens for connections and forwards ABCI requests to one of four user-provided
Services, each responsible for processing one category of requests (consensus, mempool, info, or snapshot).
Middleware that splits a single
Serviceimplementing all of ABCI into four cloneable component services, each implementing one category of requests. The component services use message-passing to share access to the main service, which processes requests with the following category-based prioritization:
ConsensusRequests sent to the
MempoolRequests sent to the
SnapshotRequests sent to the
InfoRequests sent to the
Because the ABCI server takes one service per category, users can apply Tower
layers to the services they pass to the ABCI
Server to add
category-specific behavior, such as load-shedding, buffering, etc.
These parts can be combined in different ways to provide different points on the tradeoff curve between implementation complexity and performance:
At the lowest level of complexity, application developers can implement an ABCI application entirely synchronously. To do this, they implement
Service::callperforms request processing and returns a ready future. Then they use
split::serviceto create four component services that share access to their application, and use those to construct the ABCI
Server. The application developer does not need to manage synchronization of shared state between different clones of their application, because there is only one copy of their application.
At the next level of complexity, application developers can implement an ABCI application partially synchronously. As before, they implement
Service<Request>to create a single ABCI application, but instead of processing all requests in the body of
Service::call, they can defer processing of some requests by immediately returning a future that will be executed on the caller’s task. Although all requests are still received by the application task, not all request processing needs to happen on the application task.
At the highest level of complexity, application developers can implement multiple distinct
Services and manually control synchronization of shared state between them, then use these to construct the ABCI
Because these use the same interfaces in different ways, application developers can move gradually along this curve according to their performance requirements, starting with a synchronous application, then refactoring it to do some processing asynchronously, then doing more processing asynchronously, then splitting out one standalone service, then using entirely distinct services, etc.