For BFV, OpenFhe seems to be faster than SEAL at most operations. For example, BFVrns_MultNoRelin/32768/4 in OpenFhe takes 19.6ms, whereas a bench for same operation (if I am not wrong) n=32768 / log(q)=881 / BFV / EvaluateMulCt/iterations:10 in SEAL takes 201ms. OpenFhe performs 10x better.
After observing the difference in benchmarks, I was curious about how and why OpenFhe has better performance than SEAL in BFV operations. Any pointers will be of great help!
In OpenFhe, I notice the performance difference between operations with suffix /3 and /4. Are they related to log(q) ? What’s the value of log(q) in benchmarks ?
I have also attached links to json file containing the benchmark results for reference.
The suffixes /3 and /4 are the number of RNS channels in the underlying ciphertext coefficient modulus q. And yes, they are related to \log_2{}q. If you built OpenFHE with NATIVE_SIZE=64-bit, 4 channels mean your \log_2{}q = 4*60=240 bits. You can increase the size of q by setting the multiplicative depth to a higher value.
Apples-to-apples comparison among different FHE libraries is not an easy task, you need to ensure at least the cryptographic parameters are the same, the number of CPU threads used is the same, and whether any optimization or acceleration is enabled/disabled.
I benchmarked OpenFHE (compiled with NTL, GMP, tmalloc and with hexl backend) with log(q) = 900 and the results for *32768/15 are here.
OpenFHE still performs better than SEAL (with hexl). BFVrns_MultNoRelin/32768/15 in OpenFHE takes 66ms whereas n=32768 / log(q)=881 / BFV / EvaluateMulCt/iterations:10 in SEAL takes 201ms.
Please let me know, if I am making incorrect comparisons.
First, check if EvaluateMulCt in SEAL does relinearization or not as part of ciphertext-ciphertext multiplication. In OpenFHE, EvalMultNoRelin does only ciphertext-ciphertext multiplication without Relinearization. Note that in OpenFHE, you can use EvalMult which invokes both multiplication and relinearization.
The second thing I would do is to run these experiments under single-thread (set OMP_NUM_THREADS=1) with no HEXL optimization and no NTL (that is, using native math backend in OpenFHE) benchmarks and observe the behavior.
In my opinion, in order to have the fairest comparison possible, you need to be equally well-versed in both libraries.
I confirm that EvaluateMulCt does not perform relinearization as part of ciphertext-ciphertext multiplication. Also both benchmarks were performed on same machine (i.e m6i.xlarge instance) using single thread.
I will try running benchmarks again with native backends.