A Coprocesor backend is needed to run alongside the geth node. The Coprocessor backend executes the actual FHE computation. Please look at FHE Computation for more info.
The coprocessor backend is implemented in the Coprocessor directory of fhevm-engine
.
It consists of the following components:
server that handles:
gRPC requests for computation from geth
input insertion requests to the DB from the Gateway
FHE ciphertext read requests from the Gateway
PostgreSQL DB for storing computation requests and FHE ciphertexts
worker that reads comoutation requests from the DB, does the FHE computation and inserts result FHE ciphertexts into the DB
The server and the worker can be run as separate processes or as a single process. In both cases they communicate with one another through the DB.
The Coprocessor backend supports multi-tenancy in the sense that it can perform FHE computation for separate host blockchains, under different FHE keys.
You can use pre-generated Docker images for the Coprocessor backend node or build them yourself as described in the README.
Please note that a Coprocessor geth full node is needed in order to execute blocks on the host blockchain and trigger FHE computation.
This document is based on go-ethereum v1.14.3
To start a local geth coprocessor full node, please have a look at: https://github.com/zama-ai/go-ethereum-coprocessor/blob/master/Dockerfile.devnode
This document is still work in progress. Above repositories serve as reference for now.
This document is a guide listing detailed steps to integrate fhevm-backend
into or any other implementations that follow the same architecture. We use geth
and go-ethereum
interchangeably from now on.
The Go library that we integrate into geth is .
We also have changes to geth itself here .
To execute the actual FHE computation, a is needed.
Some settings of the Coprocessor backend are configured by inserting entries in the PostgreSQL DB.
At the time of writing, we don't have a tool or automation for doing the configuration. The DB schema can be used as a reference, though: schema.
The tenants
table contains a list of tenants that are using the Coprocessor backend. A tenant could be thought of as a separate blockchain (or a separate FHE key, i.e. using multiple FHE keys on a blockchain). The fields in tenants
are:
tenant_id
unique tenant identifier
tenant_api_key
an API key that authenticates access to the server
chain_id
the chain ID of the chain this tenant operates on
verifying_contract_address
address of the InputVerifier contract
acl_contract_address
address of the ACL contract
pks_key
a serialization of the FHE public key
sks_key
a serialization of the FHE server key
public_params
a serialization of the CRS public params
cks_key
optional secret FHE key, for debugging only
is_admin
if tenant is an administrator
You can use the --help
command line switch on the coprocessor to get a help screen as follows:
Note that there are two thread pools in the Coprocessor backend:
tokio
FHE compute
The tokio one (set via --tokio-threads
) determines how many tokio threads are spawned. These threads are used for async tasks and should not be blocked.
The FHE compute threads are the ones that actually run the FHE computation (set via --coprocessor-fhe-threads
).
A secret signing key is needed to allow the Coprocessor backend sign input insertion requests. A coprocessor.key
file can be given on the command line for the server as:
The secret signing key must be kept safe when operating the Coprocessor.
Following is an example of how to deploy initial contracts on the Ethereum Sepolia testnet. Deploying on Ethereum mainnet should be almost identical and should be possible by just changing the SEPOLIA_RPC_URL to ETHEREUM_MAINNET_RPC_URL and poiting to a correct RPC node.
0/ Prerequisites: First, git clone fhevm
repo and checkout the 0.6.0-0
prerelease branch, install dependencies with npm i
(node.js version should be at least 20
), then create a.env
file in root of the repo.
1/ Fill correct values in the .env
file by first copying the .env.example.deployment
file. Your .env
file should have the following format, while replacing the 2 private keys values PRIVATE_KEY_FHEVM_DEPLOYER
and PRIVATE_KEY_GATEWAY_DEPLOYER
with your own keys, and taking the KMS addresses to be aligned with what is used by KMS, as well as the ADDRESS_GATEWAY_RELAYER
relayer address to be aligned with the Gateway service, and the ADDRESS_COPROCESSOR
coprocessor account address to be aligned with the coprocessor service:
For the SEPOLIA_RPC_URL
env variable, you can either get one from a service provider for free like Infura, or use your own RPC URL if you are running a node yourself. For ETHERSCAN_API_KEY
it is needed to verify source code of smart contracts on Sepolia/mainnet Etherscan (if you deploy coprocessor on Sepolia or mainnet for instance) and you can get a free Etherscan API key from: https://docs.etherscan.io/getting-started/viewing-api-usage-statistics.
Important : the PRIVATE_KEY_FHEVM_DEPLOYER
and PRIVATE_KEY_GATEWAY_DEPLOYER
are expected to have a nonce of 0
initially (i.e never sent any tx before with those) for the deployment scripts to succeed later. If you have foundry installed, you can generate fresh Ethereum private key / address pairs locally with this command: cast wallet new
.
2/ Then run the precomputing contract addresses script ./precompute-addresses.sh
-> this will write on disk the correct contract addresses needed to launch the modified Geth node (TFHEExecutor
address) and the Gateway service (GatewayContract
one) and the ACL
and KMSVerifier
addresses which would be needed to setup some values inside the ASC contract on KMS chain. The precomputed addresses for the core contracts are located inside:
node_modules/fhevm-core-contracts/addresses/.env.acl
for ACL address
node_modules/fhevm-core-contracts/addresses/.env.exec
for TFHEExecutor address
node_modules/fhevm-core-contracts/addresses/.env.kmsverifier
for KMSVerifier address
node_modules/fhevm-core-contracts/addresses/.env.inputverifier
for InputVerifier address
node_modules/fhevm-core-contracts/addresses/.env.fhegaslimit
for FHEGasLimit address
gateway/.env.gateway
for GatewayContract address.
This script is found exactly inside the ./precompute-addresses.sh
file:
3/ Launch and setup the Geth node, the Gateway service, setup the KMS, all these using previously precomputed addresses.
4/ Use the faucet to give some funds to the FHEVM_DEPLOYER
address, and the and the GATEWAY_DEPLOYER
addresses. Note that you can compute the address corresponding to a private key, if you have foundry installed, via this command: cast wallet address [PRIVATE_KEY]
.
Important: Later after deployment, the GATEWAY_RELAYER
should also receive a LOT more funds than the other addresses since this is the account which will send tx one each decryption fulfilment. I would advise for eg 3
ETH for the FHEVM_DEPLOYER
account, 0.5
ETH for the GATEWAY_DEPLOYER
to be safe, and at least 10
ETH for the GATEWAY_RELAYER
at first (to be refilled later if its balance becomes low) to be on the safe side. Another advise is to check current gas price on Sepolia before deployment and avoiding periods where gas prices are spiking to huge values (>100 GWei), for eg by consulting
https://sepolia.beaconcha.in/gasnow.
The funding of GATEWAY_RELAYER
account is the only part which is not strictly needed during deployment, and this account could be funded later, after all deployment steps will be completed.
5/ Finally, run the deployment script ./launch-fhevm-sepolia.sh
. This script is found exactly inside the ./launch-fhevm-sepolia.sh
file :
Note that in previous example, we supposed deployment would happen on Sepolia, this is why we used the --network sepolia
flag. Note also the last 6 lines which are responsible to verify all contracts which have just been deployed on Sepolia, using the Etherscan API key, after waiting 2 minutes to make sure that the contracts bytecode were correctly propagated. This last command only makes sense if you plan to deploy on a network with Etherscan support, such as Sepolia or Ethereum mainnet.
Important: at this stage the openzeppelin-upgrades
plugin will write on disk an .openzeppelin/
folder with some files in it (for eg typically a sepolia.json
file if you deploy on Sepolia). It will be critical to keep this folder saved, to be able to do safe upgrades of smart contracts if needed some day. Similarly, you must also keep all the .env.XXX
files written inside both of node_modules/fhevm-core-contracts/addresses
and gateway/
directories saved, because those contains the proxy addresses which might also be needed during the upgrade process.
1/ After deployment, if you wish to launch some hardhat test on Sepolia, you first need to add a MNEMONIC
env variable inside your .env
file. You can generate this variable using foundry with this command: cast wallet new-mnemonic -w 15 -a 5
. Your new .env
file should then looks like this - please replace MNEMONIC
by your own value:
2/ The previous command will also show you the 5
addresses of accounts (alice, bob, carol, dave, eve) used inside the hardhat tests, that you will need to fund via the faucet. An alternative way to get the addresses of those accounts could be to simply run: npx hardhat accounts --network sepolia
, then the first 5 addresses logged will be those that need funding.
Important: Currently, for all tests, you just need to fund the first 3 accounts (alice, bob, carol), this could help you to avoid wasting gas. For the EncryptedERC20 ones in particular, you only need to fund the first 2 accounts (alice, bob). No need to actually fund all of the first 5 accounts, unless you want later to run more complex tests like the Confidential AMM, etc, which have not been updated for new Solidity API anyways, yet.
3/ Now you can run any hardhat test with the usual command. For instance, if you want to run the EncryptedERC20 related tests you can use: npx hardhat test test/encryptedERC20/* --network sepolia
. As another example, you could run the asynchronous decryption tests via: npx hardhat test test/gatewayDecrypt/* --network sepolia
.
1/ For upgrading the ACL contract, save the new implementation in a solidity file somewhere in your local fhevm
repo, then simply run this command:
In previous command, you should modify PRIVATE_KEY_FHEVM_DEPLOYER
with your own value, and replace examples/ACLUpgradedExample.sol:ACLUpgradedExample
with the path and the name of the new ACL implementation contract that you wish to use. The --verify-contract
is an optional flag, whish will add an Etherscan verification step after deployment of the new implementation, pausing 2 minutes between deployment and Etherscan verification.
You can use similar commands for any contracts that you wish to upgrade, just using any of the following tasks, to replace the task:upgradeACL
from previous upgrade command :
Important: Note that for the task:upgradeGatewayContract
you should use the PRIVATE_KEY_GATEWAY_DEPLOYER
as the private-key
value. Also, note that those upgrade scripts would only work if the updated implementations do not have a reintializer function, i.e if part of storage does not need to be reinitialized, which should be almost always the case. In the rare case where you need a reinitializer in the new implementation, the upgrade script should be customized according to the arguments needed for the reinitialization.