Bringing Shielded Transactions to the Web

This week, we deployed Testnet 63, which brings an important and long-awaited feature: a web wallet and frontend site that brings shielded transactions to the browser, without compromising on privacy or decentralization. This project has been long in the making, and it's helped us discover subtle requirements that we've been able to feed back into the design of the core cryptographic protocol. We're so excited to finally be able to release it, and to share our first-principles rethinking of web frontends in the shielded context.

Why rethinking? Because solving on-chain privacy has the side effect of upending fundamental design assumptions about how frontends are built. For instance, on a shielded chain, there's no RPC to query for your account balance, since only you can see it. Shielded chains achieve privacy by moving end-user data off the chain and onto end-user devices, and that means those end-user devices must be responsible for scanning and processing that data. While this poses challenges, it also creates opportunities. As we'll explain in this post, the resulting architecture has significant advantages for decentralization, scalability, and security.

Challenges

Privacy is about control over information disclosure. Privacy-preserving systems are not about hiding everything at all times from everyone -- they are about giving users the ability to choose what information is disclosed to whom at what time, rather than requiring that all information is disclosed to everyone at all times.

This allows building on-chain systems that match the expectations of real-world users. For instance, in traditional finance, being able to log in to someone else's brokerage account and see all of their positions and trading history would be an unacceptable data breach. Yet on on-chain, this is considered normal or even desirable. It is not. On-chain activity is the future of coordination technology, but that future requires control over information disclosure.

Building that future requires rethinking both data modeling as well as data management.

We need to rethink data modeling because introducing privacy requires being able to model what information is visible to whom. The same transaction may look different to different users, who may have different levels of visibility into their contents. Software needs to be able to show users what they see and what others see. And the way that users interact with chain data changes. For instance, conventional authorization flows first build a transaction, then present it to the user for approval. But since shielded transactions are opaque, this flow has to be changed to allow a user to inspect the transaction before it is constructed. To address this challenge, we've introduced first-class modeling of data visibility through the entire transaction lifecycle, which you can read about here and here.

We need to rethink data management because introducing privacy means there's no centralized server with access to all of a user's account information. Privacy is achieved by moving end-user data onto end-user devices, which means those end-user devices must be responsible for scanning and processing that data. Execution happens on the end-user device, not on-chain, so the user's device must maintain and synchronize their state. This requires thinking holistically about how the protocol reaches all the way out to end-user devices, and how to make end-user processing scalable as the system grows. This is a significant challenge, but it is also an opportunity. Privacy forces us to build a truly decentralized system, rather than relying on centralized RPC providers who become chokepoints on users' access to the network.

Design Goals

In thinking about how web interfaces to Penumbra should work, we identified three high-level goals:

We want to be able to support a wide variety of interfaces to Penumbra.

While it's important that there be at least one first-class wallet experience at launch, Penumbra's capabilities are multifaceted, and users will probably be best served by specialized interfaces: e.g., one for basic transfers or swaps, one for governance, one for power users to manage liquidity or examine the liquidity graph, etc.

Supporting a wide variety of interfaces is also important for decentralization: no one entity should "own" the Penumbra userbase via control of a single frontend, or be at risk of becoming a chokepoint for control of those users. This also means that it should be possible to build frontend interfaces to Penumbra that do not require custom backend infrastructure beyond an ordinary pd full node. And the demands on that infrastructure should be as minimal as possible, so that it's easy for anyone to run.

We want those interfaces to be as easy to build as interfaces to transparent chains.

Historically, interfaces to shielded chains have been more difficult to build than interfaces to transparent chains, because they require the application developer to manage synchronization of users' private state, unlike a transparent chain, where user state is accessible via RPC. Penumbra is about privacy without compromise, so we want to make it as easy to build those interfaces for Penumbra as it is for a shielded chain.

We want our users to have security when interacting with those interfaces.

In order for users to benefit from the availability of a wide variety of third-party interfaces, users need to be confident they can use them without risk of losing funds or being hacked. We need to ensure that only the user can authorize a transaction, and be able to understand exactly what actions they're authorizing when they do so.

Penumbra's Web Architecture

Our high-level approach to realizing these goals is to provide a "local RPC" that restores the conventional programming model for frontends for transparent chains -- but without the dependence on centralized RPC providers who see all user data.

We define two GRPC service interfaces, corresponding to "read" and "write" access:

• The View service can be thought of as a personal indexer, providing read access to all private data visible by a given wallet. The view server is responsible for scanning for transactions relevant to the user, downloading them, and decrypting and indexing them locally.
• The Custody service handles authorization requests, receiving a TransactionPlan describing a proposed transaction, allowing the user to review and authorize it, and returning a bundle of AuthorizationData with signatures needed to build the transaction. The service interface is agnostic to the custody backend, allowing software wallets, hardware wallets, or threshold custody.

These services are not just limited to the web context. By defining them as first-class interfaces, we can use them across the entire Penumbra tooling ecosystem. For instance, the command-line wallet pcli runs them internally and does in-memory GRPC, and the client daemon pclientd exposes them as a network service.

In the web context, we provide these services through a browser extension that runs them on the end-user device. We use the Buf.build Typescript packages generated from our Protobuf definitions to model all Penumbra data, and implement a custom transport object that allows frontend code to make GRPC requests to the browser extension exactly the same way it would make them to a remote server.

This restores the conventional programming model, where frontend code is mostly stateless and relies on RPC requests to read and write data. The only difference is that requests relating to private user data are processed locally, while requests relating to public chain state are sent over the network.

To handle the complexity of modeling data visibility, the view service returns transaction views, pre-decrypted and pre-interpreted views of transaction data that highlight exactly what data is visible to the user's wallet. For instance, while a transaction has a Spend action, a transaction view has a SpendView that can be either visible or opaque, with the decrypted data bundled into the visible variant. This allows frontend developers to focus on the semantics, without having to worry about the mechanics of Penumbra's shielded crypto.

All long-term key material is kept inside of the browser extension, so web content only has access to per-transaction data, and all transactions must be approved through the browser extension, which provides a secure display path for the user to review a proposed transaction. This allows a user to selectively grant viewing or spending permissions to web interfaces to relatively untrusted web interfaces, and to revoke those permissions later.

This architecture helps realize our high-level goals for security and decentralization:

• Because the extension exposes common service interfaces to web frontends, we know that a "first-party" frontend is not privileged over a "third-party" frontend. Our test frontend shows that it's possible to build complete functionality using public API surface, so that no entity is in control of what frontends can be written.
• Because the extension performs local indexing, most RPC requests are processed locally, and the demands on the RPC provider are relatively lightweight. Our goal is that any pd node should be able to serve a frontend out of the box, which is why, for instance, pd has zero-configuration support for auto-HTTPS via Let's Encrypt and multiplexed grpc/grpc-web support out of the box.
• Because authorization requires a TransactionPlan that completely describes the effects of a transaction, the extension can show a user exactly what they're authorizing.
• Because the extension performs fine-grained, selective disclosure of transaction contents through transaction views, a compromised frontend cannot compromise any long-term key material. At worst, it can disclose historical information, and users can limit damage by revoking access or prevent it by only approving read access from frontends they trust (or run locally).

While we're not intent on the Penumbra extension being the only custody and view provider for Penumbra, building an end-to-end system has allowed us to flesh out the design. As we mature the code, we're interested in working with existing wallets to integrate Penumbra support -- but we want to know that the components they'd integrate work end-to-end first. Going forward, we're interested in supporting the Penumbra wallet extension as a first-class tool on its own and as a collection of libraries reusable by other wallets.

Getting Started

Penumbra's web extension is now published to the Chrome Web Store. The frontend site is available at https://app.testnet.penumbra.zone. Both are still a work-in-progress, but we appreciate hearing about any and all feedback in the Discord!

Stay tuned for more updates on our pathway to mainnet. Slowly, and then all at once.