How to Build a Decentralized Dark Pool
Now that I’m in robotics, I often get people asking about my past life as a cryptographer — specifically building Renegade. This is a tricky one to explain; Renegade exists at the confluence of two fields which seldom attract deep learning people: market microstructure and cryptography. This post is an attempt to explain what it is we built!
At its core, Renegade is a decentralized dark pool. Understanding these two “D” terms (decentralization and dark pool) is key to understanding Renegade. We’ll deal with these terms in reverse order.
Background — Dark Pools and Decentralization
Dark Pools
A dark pool is a type of trading venue called an alternative trading system (ATS). Technically, these are not exchanges, but that is more of a regulatory distinction. The key property distinguishing a dark pool from, say, the NYSE is that orders in a dark pool are private until they execute. This is particularly useful when executing large block trades, where the trader might worry about the market moving out from under them. Front running is one way this might happen.
When trading on a blockchain, the front running problem is much, much worse. This essentially comes from two properties of blockchains:
- All transactions are public.
- Transactions may be reordered arbitrarily by a special blockchain node called a sequencer.
Together, these properties allow sophisticated actors to view trades before they execute, and bribe sequencers to reorder transactions, landing a profitable front run.
Because the problems which motivate dark pools are much greater on a blockchain, the need for a dark pool is also much greater. However, as we found out at Renegade, building a dark pool on a blockchain ends up being pretty difficult. The main reason for this is decentralization, which brings us to the second “D” term that I mentioned earlier.
Decentralization
Decentralization is the core principle motivating blockchains, cryptocurrencies, and the decentralized finance (DeFi) applications built on top. The promise of decentralization is essentially that it limits intermediary risk. Whether by negligence or malevolence, every intermediary in a financial transaction layers on risk, which is experienced as a cost by the users of the financial system. By decentralizing financial infrastructure, we can verify its operation and limit the risk introduced by bad actors.
Most of the approaches to decentralization achieve these lofty goals by introducing some form of consensus mechanism, which lets any of the nodes composing a blockchain verify that a transaction has been executed correctly.
If you’ve been following carefully, the ideas of decentralization and privacy may seem at odds! More on that in just a moment.
Why Decentralize a Dark Pool?
For two reasons:
- As mentioned above, trading on a blockchain is extremely adversarial. Adding privacy to blockchain trading solves many of the problems introduced by decentralization.
- Traditional dark pools pass significant exchange risk onto their users. Essentially all major dark pools in traditional finance have been sued by the SEC for trading against their own users. Remember, the optimal amount of fraud is not zero; big banks often think like that.
The second point is, in my opinion, the more interesting one because it gives a decentralized dark pool a wedge. By decentralizing a dark pool we can actually build better dark pools than can exist in traditional finance, free from operator risk!
How to Build a Decentralized Dark Pool
As I hinted at previously, there is a fundamental tension between decentralization and dark pools:
- Decentralization seems to require that an exchange’s state be public.
- Dark pools definitionally require exchange state to be private.
So what can we do? Considering the tools at our disposal, we remember that cryptography has a long history of resolving seemingly impossible contradictions. Consider the miracle of public-key encryption.
Before we try getting clever with our cryptography, let’s be explicit about the semantics we’d like our exchange to hold.
The Two Guarantees
Fundamentally, a decentralized dark pool should have two guarantees:
- Pre-Trade Privacy: Before two orders cross, their parameters should not be visible to other market participants.
- Post-Trade Privacy: After a trade settles, its parameters should not be visible to any counterparties except those whose orders crossed.
The first is the classic dark pool guarantee. The second is an extension of traditional dark pool semantics, which solves a whole host of other problems in crypto, like copy trading or preserving privacy across multiple legs of a trade.
To imbue our exchange with these semantics we take three steps which transform a standard central limit order book (CLOB) into a decentralized dark pool.
Step 1: Shard and Distribute the Order Book
We’ll focus on pre-trade privacy first. Traditional dark pools achieve pre-trade privacy by centralizing the order book (CLOB) and trusting the order book operator not to leak private order state or trade on it. For the reasons listed above, this won’t do.
Instead, Renegade decentralizes the order book by sharding order state among nodes in a P2P network. Each node, called a relayer, is responsible for shopping around its orders to find matches. Traders delegate their orders to a relayer or cluster of relayers. A relayer might be run by an infrastructure provider, broker, or the trader themselves. It is up to the individual trader to choose a relayer they trust with their private state, or run their own.
Given an order book distributed across the Renegade P2P network, the question of how to cross two orders while keeping them private remains unanswered. Here we turn to the field of secure multi-party computation (MPC). An MPC scheme is a cryptographic protocol in which parties jointly compute a function of private inputs and all learn the output . Importantly, no party learns another’s private input, or intermediate results in the computation of . Renegade uses a custom MPC scheme inspired by SPDZ which is based on semi-homomorphic encryption 1. To facilitate this, we built a rust framework ark-mpc for describing MPC circuits as normal rust functions. ark-mpc handles secret sharing, network communication, and auto-parallelization under the hood. It is a graph-based approach inspired by PyTorch.2. Note that an entirely different class of MPC approaches is built on boolean circuits; they are faster in many circumstances. In our case, collaboratively generating a zero-knowledge proof within an MPC circuit ends up being a highly linear computation, for which SHE-based MPC approaches require very little overhead. More on this below..
Relayers take part in a continuous MPC protocol in which a match is inferred from two orders. Relayers continuously ping their peers in the network to propose MPC matching engine evaluations, and submit a match to the settlement layer smart contracts when one is found. The structure of this system is visualized in Figure 1 below.
Note that a relayer may manage orders for many users simultaneously. If two users who delegate to the same relayer place crossing orders, a match can be inferred without the need for MPC. This is called an “internal match” and is depicted in the diagram below.
With the order book sharded across relayers and matching done inside an MPC circuit, an order stays private: only the trader and the relayer holding it can see it. This gives the user full pre-trade privacy; a trader only learns about another’s order by successfully matching against it3. A particularly important property of an MPC scheme in this case is its fairness. Essentially, neither party should be able to abort the MPC protocol and learn the protocol output before the other party does. This would break Renegade’s pre-trade privacy, as a malicious party could abandon the match without the honest party learning of it or having the opportunity to settle the match on-chain..
Step 2: Merklize the Exchange State
With pre-trade privacy under our belt, we now seek to make the exchange post-trade private. In order to decentralize and make Renegade composable with other DeFi applications, the settlement layer must be a smart contract on a blockchain. So, we need to introduce a primitive that allows private state to be posted and updated on a public blockchain.
A common solution here is to introduce a Merkle tree and store only the state root, which obfuscates the data in the leaves.
On top of the Merkle tree, we layer a commitment-nullifier scheme. A user groups their exchange state (orders, balances, etc) into an abstraction called a “wallet” and creates a cryptographic commitment to their wallet. The commitment for a wallet is defined as for some cryptographic hash function 4. Renegade uses an algebraic hash function called Poseidon2, which is key to the zero-knowledge proof section below. You can see our rust implementation or the low-level implementation written in Huff — a stack-based assembly language.. These commitments become the leaves of the Merkle tree as visualized below.

The Merkle tree gives us a way to insert a static view of the user’s wallet into contract state, but provides no mechanism for state transitions. To see this, note that a user could simply commit to the same wallet multiple times, or branch state updates off of the same wallet multiple times. We need to enforce a unique, linear wallet history.
To support updates to a user’s wallet (e.g. depositing into the exchange), we introduce a nullifier derived from the wallet’s commitment. Nullifiers are the cryptographic equivalent of tombstone records, for those familiar with database design. The nullifier for wallet is derived from the wallet’s commitment as:
Where is private randomness that serves the same function as a salt in password hashing 5. That is, adding randomness makes inferring from impossible. Without private randomness, a malicious adversary could observe a commitment , compute , and then link the committing and nullifying transactions together as updating the same wallet. Though this doesn’t compromise the pre- or post-trade privacy of the exchange, a motivated observer could infer order information by linking transactions and analyzing their timing..
State transitions then follow the same high-level protocol regardless of the specific updates to wallet state. Suppose represents an initial wallet and represents an updated version of (with a new balance deposited, for example). To transition their exchange state from to a user:
- Computes the nullifier of the original wallet and the commitment to the new wallet , then submits these values to the Renegade smart contract.
- If has already been “spent” in the smart contract, the transaction is rejected as an invalid transition. The user is trying to transition out of the same initial state twice.
- If accepted, is marked as spent, and is inserted into the next leaf in the Merkle tree.
With the Merkle tree and commitment-nullifier scheme we have a state inclusion and transition primitive which is both hiding and binding.
Step 3: Wrap State Transitions in Zero-Knowledge Proofs
Merklizing the exchange state is a strong first step towards post-trade privacy, but on its own fails to address the question of how state updates are verified. In the update process described above, the user supplies commitments and nullifiers directly. A malicious user could build a commitment to an arbitrary state transition, fabricating a deposit or settling a fictitious trade.
To prevent this, the settlement layer (practically, a smart contract) needs a means to verify updates in order to protect the exchange state’s integrity. As a straw man, the user could simply submit their pre- and post-update wallets along with the commitments and nullifiers previously described. The Renegade contract would then examine the wallet contents directly and verify that the transaction represents a valid state transition — e.g. that a deposit into the exchange is reflected as a commensurate increase in the balances of . Of course, submitting wallets in the clear completely forgoes privacy, so the naive solution is insufficient.
Instead, we turn to zero-knowledge proofs (ZKPs) 6. Renegade uses a custom proof system based on PlonK. We extend PlonK with stronger arithmetization primitives for efficient ZKPs and a proof linking construction discussed below. Our rust implementation is here and has a bunch of fun mathematical and systems optimizations. The verifier lives in a smart contract, found here.. A ZKP is a two-party cryptographic protocol which allows one party, the prover, to convince the other, a verifier, that an arbitrary NP statement is true. Moreover, the statement which the prover asserts may reference both private and public inputs. For example, when depositing a balance of 1 BTC into the dark pool, a prover constructs a ZKP for the statement:
Wallet , with commitment and nullifier , is included in the Merkle tree with root . Wallet , with commitment , is equivalent to except for a 1 BTC balance increase.
Here, the prover will keep , and the Merkle proof private. However, will be public inputs, so that they may be verified by the settlement layer and used in the commitment-nullifier scheme described in the last section. The settlement layer’s operation is then:
- Verify the ZKP by running the proof system’s verification protocol.
- Check that the Merkle root matches the root stored in the contract state.
- Assert that the nullifier has not been spent, then spend it.
- Insert the new commitment into the next Merkle leaf.
The exchange state is thereafter successfully updated with as the active wallet state for the user 7. We’ve used the example of depositing a balance here, but settling a trade between two counterparties is much the same. The only fundamental differences are that the NP statement asserts certain trade validity constraints and that we submit two nullifiers for old wallets and two commitments to new wallets.. Only nullifiers and commitments, which hide the data they represent, are revealed in the clear. Thus, we’ve achieved post-trade privacy; third-party observers of the exchange see only hiding commitments of underlying data.
Tying it Together
We’ve now got an exchange that guarantees both pre- and post-trade privacy. To crystallize all that’s been described, it’s helpful to explicitly examine the data flow of a user’s order from an intent’s creation to a match’s settlement on-chain.
The flow is visualized in Figure 2 below.
- The user creates an order to buy one BTC, and submits this order to the relayer managing their wallet.
- The relayer generates a ZKP proving that the update was authorized by the user and that the wallet was updated accordingly. The relayer then submits this proof to the exchange on-chain in order to update the Merkle state.
- The relayer begins matching the order once it has verified Merkle inclusion. It finds a crossing counterparty order managed by another relayer, say, selling 0.5 BTC.
- Both relayers receive the match result and attempt to submit it on-chain. Only one will get there first, but both transactions update all state, so only one copy of the transaction will land. The relayer submits the match on-chain and the exchange state updates both wallets involved in the trade.
Appendix & Extras
The above describes the big ideas and general structure of Renegade’s design. However, as is true with many systems, Renegade’s design is equally the story of a collection of small, interesting design decisions which make all the difference. For us, these are mostly aimed at efficiency. In the exchange business, latency has a very real cost.
This section is aimed at describing some of these more interesting details. The sub-sections certainly don’t need to be read in order. I’ll add to this list over time.
A1. Collaborative Proofs
You may have noticed that the connection between pre- and post-trade privacy is underdeveloped in the description above. Trade settlement requires a proof referencing orders from two wallets; the question is who generates this proof? The zero-knowledge prover must have access to the witness (private inputs) in order to generate a proof. This is definitionally true; security arguments for ZKP frameworks often require that a prover be “witness extractable” — essentially, the prover must actually know the witness.
If party 1 generates the proof, then party 2 must share their private state in the clear, so that it may be included in the witness. If party 2 generates the proof, the opposite. Clearly, this breaks forward pre-trade privacy, so neither option is tenable.
What we’d really like is a multi-prover system; in which a distributed prover generates a ZKP over a sharded witness. Party 1 and party 2 would each contribute half of the witness, neither would learn of the other’s input, but both would receive the ZKP produced at the end. This is starting to sound a lot like the MPC formalism described above.
The approach Renegade takes is essentially to wrap a zero-knowledge prover inside an MPC circuit. Recall that a 2PC circuit computes where are supplied by parties 1 and 2 respectively. In the description above, was the evaluation of the Renegade matching engine. Now, we extend to be a function which:
- Runs the Renegade matching engine on two orders.
- Computes updated wallet state —changing balances, decreasing order size, etc— for both wallets.
- Generates a ZKP attesting to that state transition, the same statement the settlement contract verifies in “Step 3” described above.
Both parties open the proof, and one of them submits it on-chain. Each relayer’s wallet stays shared inside the MPC. The contract sees only the new commitments, the spent nullifiers, and a hiding proof. Pre-trade privacy holds because the orders never leave the MPC; post-trade privacy holds because that proof is all the chain learns about them.
Collaborative proofs were just beginning to be studied at the time we started Renegade. We were one of the first teams to conceptualize this idea, and spent a lot of time figuring out how to make collaborative provers fast. As a heuristic, a computation incurs roughly a 1000x slowdown when encoded in either a zero-knowledge circuit or an MPC circuit. Put together, a slowdown of 10^6 would be impractical for trading use cases; trades should settle in milliseconds not minutes.
Most of the following sections deal with how to make this collaborative proof system low latency. When we first benchmarked collaborative proofs, settlement took ~2 minutes. Last time I checked it was around 10ms.
Removing Hashing From the MPC
The dominant cost in our MPC scheme is network latency incurred when the two parties communicate. As mentioned above, our MPC scheme is based on semi-homomorphic secret sharing. This means that linear functions of the MPC inputs can be computed without communication between the parties. Only non-linear functions require communication. So, reducing MPC latency is often about reducing non-linearity of the computation.
The first step to reducing non-linearity is choosing the right proof system. PlonK’s operations are largely linear. Most of the cycles in proving PlonK arugments come from number-theoretic transforms and multi-scalar multiplications, each of which are linear operators. As compared with other proof systems, PlonK is a good fit for an MPC circuit.
Beyond the choice of proof system, one common facet of all modern proof systems is the use of the Fiat-Shamir transformation. This heuristic allows for interactive proof systems (requiring the prover and verifier to communicate online) to be transformed into non-interactive proof systems. It does so by:
- Assuming that the verifier is only online to provide randomness. For example, the verifier might issue the prover a random challenge as part of the verification algorithm.
- If the verifier is only online to provide randomness, a prover could simulate the verifier’s randomness. To do this, the prover hashes elements of the proof generated thusfar (known as the prover’s transcript). The output of the hash can be considered a pseudorandom value. The verifier can repeat this computation to ensure that the prover simulated randomness correctly.
It is not important to understand the Fiat-Shamir transform beyond the fact that it requires hashing a partially generated zero-knowledge proof.
Hashing is, almost necessarily, a deeply non-linear operation. This is troublesome for MPC.
Overcoming this burden requires noticing that the values which are hashed are already privacy-preserving. They are pieces of a zero-knowledge proof which will be submitted on-chain once the proof is complete. So, rather than encoding a hash function in the MPC circuit, the two parties can:
- Combine their secret shares of the hash input to recover the input value in the clear.
- Evaluate the hash function locally, without communication, on the cleartext input. That is, compute .
- Generate secret shares for and re-share the value as , so that it can be used in further MPC computation.
Now, rather than hundreds of communication rounds to evaluate a hash function, our MPC circuit requires two. One to combine secret shares, and another to re-share the randomness.
These types of co-design optimizations are extremely common in Renegade, and largely make up the difference between an impractical system and the fast one which is deployed today.
A2. Proof Linking
Removing Fiat-Shamir hashing from the MPC protocol is a great first step, but the computation inside the MPC is still too expensive. As a reminder, the function evaluated by the MPC has three steps:
- Run the matching engine on two orders.
- Update the wallet state of the two counterparties to reflect trade settlement.
- Run the zero-knowledge prover protocol to generate a ZKP of these state updates.
The first two steps are relatively cheap, requiring a handful of additions and fixed-point multiplications. The prover’s protocol constitutes the bulk of the MPC latency. One option is to optimize the prover’s algorithm for MPC evaluation, we do this in several ways:
- Implementing polynomial division in the ring with coefficient rotation tricks.
- Using linear operators like number theoretic transforms to compute polynomial arithmetic.
- Implementing prefix product as a low-communication, local operator with a one-time pad.
Beyond optimizing the proof system itself, we can shrink the statement that we’re proving. The PlonK prover’s runtime grows as , where is the number of constraints needed to represent the computation 8. The process of turning a computation into a sequence of polynomial identities is called arithmetization. The value is then the degree of the polynomial representing the computation. The growth factor comes from the Fourier transforms used to evaluate these polynomials and perform arithmetic on them under the hood..
As it stands, the NP statement groups matching and wallet updates as one claim. That is more collaborative proving than we need. We introduced a collaborative prover only because the witness is split across two parties. The matching engine is the only step that actually consults both shares at once. Once a match is known, each party can update their own wallet and generate a proof of correctness.
So we factor the statement into three proofs:
- A small matching-engine circuit that reads both parties’ orders.
- A wallet-update circuit for party 1.
- The same wallet-update circuit for party 2.
Only (1) needs a collaborative prover. Both (2) and (3) can be proven locally, because they only reference a single share of the collective witness.
The only catch is that we need to make sure witness values used across proofs are consistent. Without a mechanism to enforce this, party 1 could use a different match result in their local proof’s witness, giving themselves a better price, for example.
We can’t fix this by simply exposing the match result as a public input to all three proofs; the match is exactly the thing we’re trying to hide. Instead, we lean on how PlonK represents the witness. The prover encodes each wire of the circuit as a polynomial whose evaluations at the roots of unity are the witness values on that wire.
If the match result is allocated to the same domain locations in two circuits, then “both proofs used the same match” is equivalent to the statement that the two wire polynomials agree at those locations. That is, for witness polynomials we want to show:
To this end, we designed a polynomial protocol which asserts this sub-domain identity, for some . In practice, the sub-domain must be a run of successive powers of 9. We must be careful about how we allocate witness values onto roots of unity. PlonK provers round the statement size to the next power of 2. This allows PlonK arguments to be made over a cyclic subgroup of their embedding field — like the 2-adic roots of unity for some order . If we allocate proof linking witness slots on a subgroup with high value , smaller circuits that we link into might have to increase their own domain’s size to include the roots of unity of order . This would increase prover latency for small circuits; remember prover latency is where in this case. To avoid this, we designed a proof linking allocator that takes inventory of the circuits composing a proof linking relation, and tries to allocate linking sub-domains on the roots of unity which cause minimal domain expansion.. This protocol operates on pairing-based polynomial commitments rather than cleartext witness polynomials, so we inherit zero-knowledge from the hiding property of the commitment scheme.
The linking relation is visualized in Figure 3 below.
At some point I’ll write a separate post explaining the details of this protocol. For now, our first audit report is a great reference on how this protocol works algebraically.
In the end this means that only a very small NP statement must be proved collaboratively, and the larger circuits are offloaded to a fast, single-prover context. The collaborative circuit’s proving latency shrinks by ~20x.