Hi OpenFHE team,
I would like to report a ciphertext deserialization robustness issue and ask whether the following behavior is expected.
Summary
In OpenFHE v1.5.1, a bounded corruption of an OpenFHE-generated BFVrns ciphertext binary stream can make lbcrypto::Serial::Deserialize trust a malformed serialized length and attempt an impractical allocation.
On an ASan build this terminates as allocation-size-too-big. On a plain build under a 2 GiB virtual-memory limit, the same reproducer throws std::bad_alloc. In both cases, the malformed ciphertext is not rejected earlier as invalid serialized input.
The corrupted input is not arbitrary random bytes. It is a small overwrite of a valid ciphertext stream produced by OpenFHE itself.
Because this reproduces both on an ASan build and on a plain build, I believe this is a meaningful parser robustness issue.
The core problem is that malformed serialized metadata is not rejected early at the deserialization boundary.
Environment
- OpenFHE version: 1.5.1
- Scheme: BFV (
BFVrns) - OS: Linux x86_64
- Compiler used for confirmation: Clang 14
- Instrumentation used during confirmation: AddressSanitizer and UndefinedBehaviorSanitizer
Reproduction
This standalone reproducer uses only public OpenFHE APIs:
#include <cstddef>
#include <cstdint>
#include <exception>
#include <iomanip>
#include <iostream>
#include <sstream>
#include <string>
#include <string_view>
#include <vector>
#include "openfhe.h"
#include "ciphertext-ser.h"
#include "cryptocontext-ser.h"
#include "key/key-ser.h"
#include "scheme/bfvrns/bfvrns-ser.h"
namespace {
constexpr size_t kCorruptionOffset = 125U;
constexpr size_t kCorruptionSpan = 7U;
constexpr unsigned char kFirstCorruptionValue = 0x04U;
lbcrypto::CryptoContext<lbcrypto::DCRTPoly> make_context() {
lbcrypto::CCParams<lbcrypto::CryptoContextBFVRNS> parameters;
parameters.SetPlaintextModulus(65537U);
parameters.SetMultiplicativeDepth(2U);
auto context = lbcrypto::GenCryptoContext(parameters);
context->Enable(lbcrypto::PKE);
context->Enable(lbcrypto::KEYSWITCH);
context->Enable(lbcrypto::LEVELEDSHE);
return context;
}
std::string make_serialized_ciphertext() {
const auto context = make_context();
const auto keys = context->KeyGen();
if (!keys.good()) {
throw std::runtime_error("OpenFHE key generation failed");
}
const auto plaintext =
context->MakePackedPlaintext(std::vector<int64_t>{1, 17, -1, 0});
const auto ciphertext = context->Encrypt(keys.publicKey, plaintext);
if (!ciphertext) {
throw std::runtime_error("OpenFHE encryption failed");
}
std::stringstream stream(std::ios::in | std::ios::out | std::ios::binary);
lbcrypto::Serial::Serialize(ciphertext, stream, lbcrypto::SerType::BINARY);
const std::string serialized = stream.str();
if (serialized.empty() || stream.fail()) {
throw std::runtime_error("serialized ciphertext is empty");
}
if (serialized.size() < kCorruptionOffset + kCorruptionSpan) {
throw std::runtime_error("serialized ciphertext shorter than corruption window");
}
return serialized;
}
void apply_candidate_corruption(std::string& serialized) {
for (size_t index = 0; index < kCorruptionSpan; ++index) {
serialized[kCorruptionOffset + index] =
static_cast<char>(kFirstCorruptionValue + index);
}
}
void dump_corruption_program(std::ostream& out) {
out << "overwrite " << kCorruptionSpan << " bytes at offset "
<< kCorruptionOffset << " with:";
for (size_t index = 0; index < kCorruptionSpan; ++index) {
out << ' ' << "0x" << std::hex << std::setw(2) << std::setfill('0')
<< static_cast<unsigned int>(kFirstCorruptionValue + index);
}
out << std::dec << '\n';
}
int usage(const char* argv0) {
std::cerr << "usage: " << argv0 << " [--dump-mutation]\n";
return 2;
}
} // namespace
int main(int argc, char** argv) {
if (argc == 2 && std::string_view(argv[1]) == "--dump-mutation") {
dump_corruption_program(std::cout);
return 0;
}
if (argc != 1) {
return usage(argv[0]);
}
const std::string baseline = make_serialized_ciphertext();
std::string mutated = baseline;
apply_candidate_corruption(mutated);
std::cerr << "serialized ciphertext size: " << baseline.size() << " bytes\n";
dump_corruption_program(std::cerr);
try {
std::stringstream input(mutated, std::ios::in | std::ios::binary);
lbcrypto::Ciphertext<lbcrypto::DCRTPoly> restored;
lbcrypto::Serial::Deserialize(restored, input, lbcrypto::SerType::BINARY);
std::cout << "deserialize returned "
<< (restored ? "non-null ciphertext" : "null ciphertext")
<< '\n';
return 0;
} catch (const std::bad_alloc& error) {
std::cerr << "caught std::bad_alloc: " << error.what() << '\n';
return 3;
} catch (const std::exception& error) {
std::cerr << "caught std::exception: " << error.what() << '\n';
return 4;
}
}
The reproducer creates a valid BFVrns ciphertext, serializes it, overwrites 7 bytes starting at offset 125 with:
0x04 0x05 0x06 0x07 0x08 0x09 0x0a
and then calls lbcrypto::Serial::Deserialize on the corrupted stream.
On the checked build, the baseline serialized ciphertext is 394205 bytes.
Observed Results
ASan build
Command:
ASAN_OPTIONS=detect_leaks=0 ./repro_openfhe_bfvrns_ciphertext_allocation_candidate
Observed output:
serialized ciphertext size: 394205 bytes overwrite 7 bytes at offset 125 with: 0x04 0x05 0x06 0x07 0x08 0x09 0x0a
ERROR: AddressSanitizer: requested allocation size 0x3830282000000018
SUMMARY: AddressSanitizer: allocation-size-too-big
Plain build
I also confirmed the same source against a separately built plain OpenFHE v1.5.1 static install. Running under a 2 GiB virtual-memory cap:
ulimit -v 2097152
./repro_openfhe_bfvrns_ciphertext_allocation_candidate_plain
produces:
serialized ciphertext size: 394205 bytes
overwrite 7 bytes at offset 125 with: 0x04 0x05 0x06 0x07 0x08 0x09 0x0a
caught std::bad_alloc: std::bad_alloc
So the malformed ciphertext still reaches the large allocation path in an ordinary build; it is not rejected earlier as invalid serialized input.
Easy To Confirm
This should be straightforward for maintainers to verify
- The reproducer is a complete standalone program that uses only public OpenFHE APIs.
- The corruption is explicit and fixed: overwrite 7 bytes at offset
125with0x04through0x0a. - The outcome is deterministic:
ASan reportsallocation-size-too-big, and a plain build under bounded memory throwsstd::bad_alloc.
Root Cause Analysis
The failing stack reaches the bundled cereal vector loader:
size_type vectorSize;
ar(make_size_tag(vectorSize));
vector.resize(static_cast<std::size_t>(vectorSize));
In the checked installation, this is:
install/include/openfhe/cereal/types/vector.hpp:57
The stack then continues through OpenFHE bigint / element-parameter deserialization below ciphertext reconstruction. In other words, malformed ciphertext metadata can propagate into a trusted container length before the library rejects the input.
My understanding is that the concrete problem is that the deserialization path accepts an unreasonable serialized length long enough to request a practically impossible allocation.
Expected Result
Malformed ciphertext metadata should be rejected as invalid serialized input before it reaches allocation-sensitive code paths such as vector.resize(...) with a huge malformed serialized size.
Why This Matters
This affects the public BFVrns ciphertext deserialization boundary. A small structured corruption of a valid OpenFHE-generated ciphertext stream can cause:
- ASan termination as
allocation-size-too-big - allocator-level failure such as
std::bad_allocin a plain build
So this is still a concrete and actionable bug: malformed input is accepted far enough to trigger allocator failure instead of a normal parse rejection. For callers that deserialize ciphertexts, that difference matters because it turns an input-validation problem into process-level failure behavior.
Suggested Direction
A possible direction would be to validate serialized container lengths and related parameter counts before resizing vectors during ciphertext deserialization, so malformed ciphertext metadata is rejected as invalid input instead of reaching the allocator.
If early rejection of malformed serialized lengths is the intended behavior, then this seems to warrant a validation fix in the deserialization path.
Reported by Jiang Chao, Beijing University of Posts and Telecommunications