# Comparing BFV benchmarks in OpenFhe (with hexl) and SEAL (with hexl)

**URL:** <https://openfhe.discourse.group/t/comparing-bfv-benchmarks-in-openfhe-with-hexl-and-seal-with-hexl/322>\
**Category:** Library Questions\
**Created:** [January 28, 2023, 11:14am UTC](https://openfhe.discourse.group/t/comparing-bfv-benchmarks-in-openfhe-with-hexl-and-seal-with-hexl/322 "2023-01-28T11:14:32Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jmall](https://avatars.discourse-cdn.com/v4/letter/j/c67d28/32.png) [@jmall](https://openfhe.discourse.group/u/jmall)\
**Post date:** [January 28, 2023, 11:14am UTC](https://openfhe.discourse.group/t/comparing-bfv-benchmarks-in-openfhe-with-hexl-and-seal-with-hexl/322/1 "2023-01-28T11:14:32Z")

</div>

For BFV, OpenFhe seems to be faster than SEAL at most operations. For example, `BFVrns_MultNoRelin/32768/4` in OpenFhe takes 19.6ms, whereas a bench for same operation (if I am not wrong) `n=32768 / log(q)=881 / BFV / EvaluateMulCt/iterations:10` in SEAL takes 201ms. OpenFhe performs 10x better.

After observing the difference in benchmarks, I was curious about how and why OpenFhe has better performance than SEAL in BFV operations. Any pointers will be of great help!

In OpenFhe, I notice the performance difference between operations with suffix `/3` and `/4`. Are they related to `log(q)` ? What’s the value of `log(q)` in benchmarks ?

I have also attached links to json file containing the benchmark results for reference.

[OpenFhe results](https://gist.github.com/Janmajayamall/fb5401e976a9954598ae523e3004b699)  
[SEAL results](https://gist.github.com/Janmajayamall/217ae4a59d528a99c45cbab06b2d9336)

---

<div class="post-metadata">

**Author:** ![Caesar](https://yyz1.discourse-cdn.com/flex031/user_avatar/openfhe.discourse.group/caesar/32/63_2.png) [@Caesar](https://openfhe.discourse.group/u/Caesar)\
**Post date:** [January 28, 2023, 9:52pm UTC](https://openfhe.discourse.group/t/comparing-bfv-benchmarks-in-openfhe-with-hexl-and-seal-with-hexl/322/2 "2023-01-28T21:52:37Z")

</div>

The suffixes `/3` and `/4` are the number of RNS channels in the underlying ciphertext coefficient modulus q. And yes, they are related to \log\_2{}q. If you built OpenFHE with NATIVE\_SIZE=64-bit, 4 channels mean your \log\_2{}q = 4\*60=240 bits. You can increase the size of q by setting the multiplicative depth to a higher value.

Apples-to-apples comparison among different FHE libraries is not an easy task, you need to ensure at least the cryptographic parameters are the same, the number of CPU threads used is the same, and whether any optimization or acceleration is enabled/disabled.

---

<div class="post-metadata">

**Author:** ![jmall](https://avatars.discourse-cdn.com/v4/letter/j/c67d28/32.png) [@jmall](https://openfhe.discourse.group/u/jmall)\
**Post date:** [January 29, 2023, 8:37am UTC](https://openfhe.discourse.group/t/comparing-bfv-benchmarks-in-openfhe-with-hexl-and-seal-with-hexl/322/3 "2023-01-29T08:37:15Z")

</div>

Thanks @Caesar for your reply.

I benchmarked OpenFHE (compiled with NTL, GMP, tmalloc and with hexl backend) with `log(q) = 900` and the results for `*32768/15` are [here](https://gist.github.com/Janmajayamall/fb5401e976a9954598ae523e3004b699).

OpenFHE still performs better than SEAL (with hexl). `BFVrns_MultNoRelin/32768/15` in OpenFHE takes 66ms whereas `n=32768 / log(q)=881 / BFV / EvaluateMulCt/iterations:10` in SEAL takes 201ms.

Please let me know, if I am making incorrect comparisons.

---

<div class="post-metadata">

**Author:** ![Caesar](https://yyz1.discourse-cdn.com/flex031/user_avatar/openfhe.discourse.group/caesar/32/63_2.png) [@Caesar](https://openfhe.discourse.group/u/Caesar)\
**Post date:** [January 29, 2023, 5:24pm UTC](https://openfhe.discourse.group/t/comparing-bfv-benchmarks-in-openfhe-with-hexl-and-seal-with-hexl/322/4 "2023-01-29T17:24:07Z")

</div>

First, check if `EvaluateMulCt` in SEAL does relinearization or not as part of ciphertext-ciphertext multiplication. In OpenFHE, `EvalMultNoRelin` does only ciphertext-ciphertext multiplication without Relinearization. Note that in OpenFHE, you can use `EvalMult` which invokes both multiplication and relinearization.

The second thing I would do is to run these experiments under single-thread (set OMP\_NUM\_THREADS=1) with no HEXL optimization and no NTL (that is, using native math backend in OpenFHE) benchmarks and observe the behavior.

In my opinion, in order to have the fairest comparison possible, you need to be equally well-versed in both libraries.

---

<div class="post-metadata">

**Author:** ![jmall](https://avatars.discourse-cdn.com/v4/letter/j/c67d28/32.png) [@jmall](https://openfhe.discourse.group/u/jmall)\
**Post date:** [January 30, 2023, 6:51am UTC](https://openfhe.discourse.group/t/comparing-bfv-benchmarks-in-openfhe-with-hexl-and-seal-with-hexl/322/5 "2023-01-30T06:51:59Z")

</div>

I confirm that `EvaluateMulCt ` does not perform relinearization as part of ciphertext-ciphertext multiplication. Also both benchmarks were performed on same machine (i.e m6i.xlarge instance) using single thread.

I will try running benchmarks again with native backends.

Thanks for the help!