CKKS Bootstrapping during multiplication gives weird output

Hi,

I have been playing with bootstrapping to multiply a given vector with 3. When I multiply by 3^20 the results get weird. May I know what is the problem. The code is a minor modification from the original code on OpenFHE.

// BSD 2-Clause License
//
// Copyright (c) 2014-2022, NJIT, Duality Technologies Inc. and other contributors
//
// All rights reserved.
//
// Author TPOC: contact@openfhe.org
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are met:
//
// 1. Redistributions of source code must retain the above copyright notice, this
//    list of conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright notice,
//    this list of conditions and the following disclaimer in the documentation
//    and/or other materials provided with the distribution.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
// AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
// DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
// SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
// CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
// OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//==================================================================================

/*

Example for CKKS bootstrapping with sparse packing

*/

#define PROFILE

#include "openfhe.h"

using namespace lbcrypto;

void BootstrapExample(uint32_t numSlots);

int main(int argc, char *argv[])
{
    // We run the example with 8 slots and ring dimension 4096 to illustrate how to run bootstrapping with a sparse plaintext.
    // Using a sparse plaintext and specifying the smaller number of slots gives a performance improvement (typically up to 3x).
    BootstrapExample(8);
}

void BootstrapExample(uint32_t numSlots)
{
    // Step 1: Set CryptoContext
    CCParams<CryptoContextCKKSRNS> parameters;

    // A. Specify main parameters
    /*  A1) Secret key distribution
     * The secret key distribution for CKKS should either be SPARSE_TERNARY or UNIFORM_TERNARY.
     * The SPARSE_TERNARY distribution was used in the original CKKS paper,
     * but in this example, we use UNIFORM_TERNARY because this is included in the homomorphic
     * encryption standard.
     */
    SecretKeyDist secretKeyDist = UNIFORM_TERNARY;
    parameters.SetSecretKeyDist(secretKeyDist);

    /*  A2) Desired security level based on FHE standards.
     * In this example, we use the "NotSet" option, so the example can run more quickly with
     * a smaller ring dimension. Note that this should be used only in
     * non-production environments, or by experts who understand the security
     * implications of their choices. In production-like environments, we recommend using
     * HEStd_128_classic, HEStd_192_classic, or HEStd_256_classic for 128-bit, 192-bit,
     * or 256-bit security, respectively. If you choose one of these as your security level,
     * you do not need to set the ring dimension.
     */
    parameters.SetSecurityLevel(HEStd_NotSet);
    parameters.SetRingDim(1 << 12);

    /*  A3) Key switching parameters.
     * By default, we use HYBRID key switching with a digit size of 3.
     * Choosing a larger digit size can reduce complexity, but the size of keys will increase.
     * Note that you can leave these lines of code out completely, since these are the default values.
     */
    parameters.SetNumLargeDigits(3);
    parameters.SetKeySwitchTechnique(HYBRID);

    /*  A4) Scaling parameters.
     * By default, we set the modulus sizes and rescaling technique to the following values
     * to obtain a good precision and performance tradeoff. We recommend keeping the parameters
     * below unless you are an FHE expert.
     */
#if NATIVEINT == 128 && !defined(__EMSCRIPTEN__)
    // Currently, only FIXEDMANUAL and FIXEDAUTO modes are supported for 128-bit CKKS bootstrapping.
    ScalingTechnique rescaleTech = FIXEDAUTO;
    usint dcrtBits = 78;
    usint firstMod = 89;
#else
    // All modes are supported for 64-bit CKKS bootstrapping.
    ScalingTechnique rescaleTech = FLEXIBLEAUTO;
    usint dcrtBits = 59;
    usint firstMod = 60;
#endif

    parameters.SetScalingModSize(dcrtBits);
    parameters.SetScalingTechnique(rescaleTech);
    parameters.SetFirstModSize(firstMod);

    /*  A4) Bootstrapping parameters.
     * We set a budget for the number of levels we can consume in bootstrapping for encoding and decoding, respectively.
     * Using larger numbers of levels reduces the complexity and number of rotation keys,
     * but increases the depth required for bootstrapping.
     * We must choose values smaller than ceil(log2(slots)). A level budget of {4, 4} is good for higher ring
     * dimensions (65536 and higher).
     */
    std::vector<uint32_t> levelBudget = {3, 3};

    /* We give the user the option of configuring values for an optimization algorithm in bootstrapping.
     * Here, we specify the giant step for the baby-step-giant-step algorithm in linear transforms
     * for encoding and decoding, respectively. Either choose this to be a power of 2
     * or an exact divisor of the number of slots. Setting it to have the default value of {0, 0} allows OpenFHE to choose
     * the values automatically.
     */
    std::vector<uint32_t> bsgsDim = {0, 0};

    /*  A5) Multiplicative depth.
     * The goal of bootstrapping is to increase the number of available levels we have, or in other words,
     * to dynamically increase the multiplicative depth. However, the bootstrapping procedure itself
     * needs to consume a few levels to run. We compute the number of bootstrapping levels required
     * using GetBootstrapDepth, and add it to levelsAvailableAfterBootstrap to set our initial multiplicative
     * depth.
     */
    uint32_t levelsAvailableAfterBootstrap = 10;
    usint depth = levelsAvailableAfterBootstrap + FHECKKSRNS::GetBootstrapDepth(levelBudget, secretKeyDist);
    // std::cout<<"FHECKKSRNS::GetBootstrapDepth(levelBudget, secretKeyDist);"<<FHECKKSRNS::GetBootstrapDepth(levelBudget, secretKeyDist);
    parameters.SetMultiplicativeDepth(depth);

    // Generate crypto context.
    CryptoContext<DCRTPoly> cryptoContext = GenCryptoContext(parameters);

    // Enable features that you wish to use. Note, we must enable FHE to use bootstrapping.
    cryptoContext->Enable(PKE);
    cryptoContext->Enable(KEYSWITCH);
    cryptoContext->Enable(LEVELEDSHE);
    cryptoContext->Enable(ADVANCEDSHE);
    cryptoContext->Enable(FHE);

    usint ringDim = cryptoContext->GetRingDimension();
    std::cout << "CKKS scheme is using ring dimension " << ringDim << std::endl
              << std::endl;

    // Step 2: Precomputations for bootstrapping
    cryptoContext->EvalBootstrapSetup(levelBudget, bsgsDim, numSlots);

    // Step 3: Key Generation
    auto keyPair = cryptoContext->KeyGen();
    cryptoContext->EvalMultKeyGen(keyPair.secretKey);
    // Generate bootstrapping keys.
    cryptoContext->EvalBootstrapKeyGen(keyPair.secretKey, numSlots);

    // Step 4: Encoding and encryption of inputs
    // Generate random input
    std::vector<double> x;
    std::random_device rd;
    std::mt19937 gen(rd());
    std::uniform_real_distribution<> dis(0.0, 1.0);
    for (size_t i = 0; i < numSlots; i++)
    {
        x.push_back(dis(gen));
    }
    x = {0.0222817, 0.18545, 0.209035, 0.0086332, 0.0220301, 0.875544, 0.219529, 0.376861};
    // Encoding as plaintexts
    // We specify the number of slots as numSlots to achieve a performance improvement.
    // We use the other default values of depth 1, levels 0, and no params.
    // Alternatively, you can also set batch size as a parameter in the CryptoContext as follows:
    // parameters.SetBatchSize(numSlots);
    // Here, we assume all ciphertexts in the cryptoContext will have numSlots slots.
    // We start with a depleted ciphertext that has used up all of its levels.
    // Plaintext ptxt = cryptoContext->MakeCKKSPackedPlaintext(x, 1, depth - 1, nullptr, numSlots);
    Plaintext ptxt = cryptoContext->MakeCKKSPackedPlaintext(x, 1, 0, nullptr, numSlots);
    ptxt->SetLength(numSlots);
    std::cout << "Input: " << ptxt << std::endl;

    // Encrypt the encoded vectors
    Ciphertext<DCRTPoly> ciph = cryptoContext->Encrypt(keyPair.publicKey, ptxt);

    std::cout << "Initial number of levels remaining: " << depth  - ciph->GetLevel()<< std::endl;

    // Step 5: Perform the bootstrapping operation. The goal is to increase the number of levels remaining
    // for HE computation.
    auto ciphertextAfter = cryptoContext->EvalBootstrap(ciph);

    std::cout << "Number of levels remaining after bootstrapping: " << depth - ciphertextAfter->GetLevel() << std::endl
              << std::endl;

    // Step 7: Decryption and output
    Plaintext result;
    cryptoContext->Decrypt(keyPair.secretKey, ciphertextAfter, &result);
    result->SetLength(numSlots);
    std::cout << "Output after bootstrapping \n\t" << result << std::endl;

    // Bootstrap_ntime(x)=Bootstrap(x)
    for (size_t i = 0; i < 29; i++)
    {
        // ciphertextAfter = cryptoContext->EvalBootstrap(ciphertextAfter);
        // ciphertextAfter = cryptoContext->EvalAdd(ciphertextAfter,2);
        // ciphertextAfter = cryptoContext->EvalAdd(ciphertextAfter,ciphertextAfter);
        ciphertextAfter = cryptoContext->EvalMult(ciphertextAfter,3);

        // ciphertextAfter = cryptoContext->EvalMult(ciphertextAfter,ciphertextAfter);
        // auto ciphertextTwoIterations = cryptoContext->EvalBootstrap(ciph, numIterations, precision);
        // https://github.com/openfheorg/openfhe-development/blob/7b8346f4eac27121543e36c17237b919e03ec058/src/pke/examples/iterative-ckks-bootstrapping.cpp#L175C5-L175C97
        // ciphertextAfter = cryptoContext->EvalBootstrap(ciphertextAfter,2,30);
        // ciphertextAfter = cryptoContext->EvalBootstrap(ciphertextAfter,2,17);

        ciphertextAfter = cryptoContext->EvalBootstrap(ciphertextAfter);

        std::cout << "Number of levels remaining after bootstrapping: " << depth - ciphertextAfter->GetLevel() << std::endl
                  << std::endl;

        // Step 7: Decryption and output
        // Plaintext result;
        cryptoContext->Decrypt(keyPair.secretKey, ciphertextAfter, &result);
        result->SetLength(numSlots);
        std::cout << "Output after " << i + 2 << "-th bootstrapping \n\t" << result << std::endl;
    }
}

The output wrong output I get is,

	(7.76915e+07, 6.46624e+08, 7.2886e+08, 3.01021e+07, 7.68142e+07, 3.05283e+09, 7.6545e+08, 1.31403e+09,  ... ); Estimated precision: 17 bits

Number of levels remaining after bootstrapping: 11

Output after 22-th bootstrapping 
	(-367.7, -2043.2, -32.9019, 456.583, 1033.32, 910.398, 2948.52, 2115.58,  ... ); Estimated precision: 16 bits

Number of levels remaining after bootstrapping: 11

Output after 23-th bootstrapping 
	(133.518, -1984.94, -854.728, 301.341, 84.9585, -238.205, 4111.82, -245.682,  ... ); Estimated precision: 16 bits

Number of levels remaining after bootstrapping: 11

Output after 24-th bootstrapping 
	(-692.562, -1220.03, 548.961, -843.712, -147.574, -669.181, 4834.61, 1753.75,  ... ); Estimated precision: 14 bits

Number of levels remaining after bootstrapping: 11

Output after 25-th bootstrapping 
	(203.905, -2061.53, -29.8208, -366.293, 1063.94, 1538.95, 3953.17, 324.673,  ... ); Estimated precision: 13 bits

Number of levels remaining after bootstrapping: 11

Output after 26-th bootstrapping 
	(-1672.04, -1027.19, 550.593, -109.611, 529.485, -525.458, 4442.74, 219.507,  ... ); Estimated precision: 12 bits

Number of levels remaining after bootstrapping: 11

Output after 27-th bootstrapping 
	(-133.95, -1354.19, -432.28, -287.268, 1580.93, -558.664, 4627.9, 1683.16,  ... ); Estimated precision: 11 bits

Number of levels remaining after bootstrapping: 11

Output after 28-th bootstrapping 
	(-529.766, -2484.34, -293.372, 429.385, -435.883, 615.414, 3950.3, -250.882,  ... ); Estimated precision: 9 bits

Number of levels remaining after bootstrapping: 11

Output after 29-th bootstrapping 
	(-1372, -1407.39, 1247.81, -1029.61, 841.563, -853.237, 4307.84, 1104.41,  ... ); Estimated precision: 8 bits

Number of levels remaining after bootstrapping: 11

Output after 30-th bootstrapping 
	(-244.028, 2.93125, -958.362, 816.723, 411.217, 43.7412, 4156.76, 975.984,  ... ); Estimated precision: 6 bits

Thanks very much.
Bhavin.

Please see Questions on CKKS bootstrapping with some computations after bootstrapping - #5 by ypolyakov The high-level idea is that the message should be close [-1,1] in magnitude for CKKS bootstrapping to achieve good precision. In your case, the messages become too large, and this is why you get bad results.

1 Like

Thank you very much for the detailed clarification. I have few more questions,

  1. If |x|,|y| <1, then I could multiply y forever with x, or things get jumpy near zero.
  2. If I have to scale down the number to [-1,1], I have to decrypt the ciphertext and set up an if else statement to do the scaling.

Thanks very much.

If the encrypted value gets close to 0, then the same error will occur.

You can scale it down homomorphically by multiplying by a double less than 1 if you can estimate the range of encrypted values.

1 Like