Performance comparisons

We’ve benchmarked our FHE compiler against existing state-of-the-art (that support comparisons over encrypted data).

Of the 5 compilers we consider, ours is the only one to rely on circuit bootstrapping as the primary bootstrapping method. Concrete relies on programmable/functional bootstrapping whereas Google, E³, and Cingulata rely on the older gate bootstrapping approach.

We consider a variety of programs that showcase comparisons over encryped data, arithmetic operations over encrypted data, etc.

Experiments were performed on an AWS c7a.16xlarge instance (64 vCPUs, 128GB RAM). Using Lattice Estimator, we estimate that our compiler and Concrete should provide 128 bits of security. Google’s transpiler and Cingulata provide 118 bits of security, whereas E3 provides approximately 94 bits of security.

Transparently, our approach shines when sufficient cores are provided with which we can parallelize tasks across. You will see less of a performance benefit with our compiler/approach if the FHE computation is being carried out on a machine with relatively few cores (e.g. 16, 32 cores). We see a significant speed up in performance time as we go from 1 thread to 64 for the below programs. Please note that this only applies to the computing party, there’s no issue at all of the end-user (providing data) uses a laptop!

Chi-squared benchmarks

We look at how our compiler performs for a FHE-friendly variant of chi-squared test with respect to runtime of the generated FHE program, size of the compiler output, and memory usage.

For Concrete (which determines precision based on developer-provided example inputs), we pass in 8-bit inputs to maintain the same output precision as all the other compilers.

Cardio benchmarks

We also consider how our compiler performs for the cardio program with respect to runtime of the generated FHE programs, size of the compiler output, and memory usage.

For Concrete, we don’t restriction precision as the output is a small integer anyway.

Scaling results

Next, we consider how the FHE compilers scale as we vary the number of inputs.

Auction benchmarks

We consider how FHE compilers perform on a first-price sealed-bid auction as we vary the number of bids.

For our work, please note that we can obtain significantly better times when we hand-code the auction using our variant of the TFHE scheme. Nevertheless, we still observe that Parasol performs best here against all other FHE compilers.

We use 16-bit precision which signficantly slows down the auction times for everyone. For Concrete, we pass in 16-bit example inputs to maintain the same input precision as the other works.