Issue with evaluating the Chebyshev formula. Inaccurate results are obtained

I have a problem using the Chebyshev evaluation.
I’ve attached a minimal example.
Input2 contains my data and I need to perform a square root operation on it.
With this data I always gettig a decryption error (approximation error too high).
I tried several things like increasing the polyDegree and multiplicative depth according to FUNCTION_EVALUATION.MD.
The range of values is pretty high, therefore I divided the input by 1E8.
With this scaling the decryption works somehow but the results are not correct.

/*
Example of evaluating arbitrary smooth functions with the Chebyshev approximation using CKKS.
*/

#include “openfhe.h”

using namespace lbcrypto;

void EvalFunctionExample();

int main(int argc, char* argv ) {
EvalFunctionExample();
return 0;
}

void EvalFunctionExample() {
std::cout << “--------------------------------- EVAL SQUARE ROOT FUNCTION ---------------------------------”
<< std::endl;
CCParams parameters;

// We set a smaller ring dimension to improve performance for this example.
// In production environments, the security level should be set to
// HEStd_128_classic, HEStd_192_classic, or HEStd_256_classic for 128-bit, 192-bit,
// or 256-bit security, respectively.
parameters.SetSecurityLevel(HEStd_NotSet);
parameters.SetRingDim(1 << 10);

#if NATIVEINT == 128
usint scalingModSize = 78;
usint firstModSize = 89;
#else
usint scalingModSize = 50;
usint firstModSize = 60;
#endif
parameters.SetScalingModSize(scalingModSize);
parameters.SetFirstModSize(firstModSize);

// Choosing a higher degree yields better precision, but a longer runtime.
uint32_t polyDegree = 100;

// The multiplicative depth depends on the polynomial degree.
// See the FUNCTION_EVALUATION.md file for a table mapping polynomial degrees to multiplicative depths.
uint32_t multDepth = 10;

parameters.SetMultiplicativeDepth(multDepth);
CryptoContext<DCRTPoly> cc = GenCryptoContext(parameters);
cc->Enable(PKE);
cc->Enable(KEYSWITCH);
cc->Enable(LEVELEDSHE);
// We need to enable Advanced SHE to use the Chebyshev approximation.
cc->Enable(ADVANCEDSHE);

auto keyPair = cc->KeyGen();
// We need to generate mult keys to run Chebyshev approximations.
cc->EvalMultKeyGen(keyPair.secretKey);

std::vector<std::complex<double>> input{1, 2, 3, 4, 5, 6, 7, 8, 9};
std::vector<std::complex<double>> input2{4438.1700, 166.5460, 78721900.0000, 43896800000.0000, 171859000.0000, 31678300.0000, 3647310.0000, 945615.0000};
size_t encodedLength = input.size();
Plaintext plaintext  = cc->MakeCKKSPackedPlaintext(input);
auto ciphertext      = cc->Encrypt(keyPair.publicKey, plaintext);

Plaintext plaintext2  = cc->MakeCKKSPackedPlaintext(input2);
auto ciphertext2      = cc->Encrypt(keyPair.publicKey, plaintext2);

double lowerBound = 0;
double upperBound = 10;

double lowerBound2 = 0;
double upperBound2 = 6;

// We can input any lambda function, which inputs a double and returns a double.
auto result = cc->EvalChebyshevFunction([](double x) -> double { return std::sqrt(x); }, ciphertext, lowerBound,
                                        upperBound, polyDegree);

auto result2 = cc->EvalChebyshevFunction([](double x) -> double { return std::sqrt(x); }, ciphertext2, lowerBound2,
                                        upperBound2, polyDegree);

Plaintext plaintextDec, plaintextDec2;
cc->Decrypt(keyPair.secretKey, result, &plaintextDec);
plaintextDec->SetLength(encodedLength);
cc->Decrypt(keyPair.secretKey, result2, &plaintextDec2);
plaintextDec2->SetLength(encodedLength);

std::vector<std::complex<double>> expectedOutput(
    {1, 1.414213, 1.732050, 2, 2.236067, 2.449489, 2.645751, 2.828427, 3});
std::cout << "Expected output\n\t" << expectedOutput << std::endl;

std::vector<std::complex<double>> finalResult = plaintextDec->GetCKKSPackedValue();
std::cout << "Actual output\n\t" << finalResult << std::endl << std::endl;

std::vector<std::complex<double>> expectedOutput2(
    {66.6196, 12.9053, 8872.5400, 209516.0000, 13109.5000, 5628.3500, 1909.7900, 972.4270});
std::cout << "Expected output\n\t" << expectedOutput2 << std::endl;

std::vector<std::complex<double>> finalResult2 = plaintextDec2->GetCKKSPackedValue();
std::cout << "Actual output\n\t" << finalResult2 << std::endl << std::endl;

}

Your upperBound2 = 6 is smaller than the largest input2 even after you divide by 1e8. Setting the interval correctly in EvalChebyshevFunction will give you the correct result.

@andreea.alexandru I’ve set the boundaries according to the input vector, so that shouldn’t be the issue anymore. I’ve also tried a few variations in the size of the polynomial, but still not getting good values.

The code below works correctly for me.
You need to keep in mind that the precision around the boundaries will not be perfect. Once you divide your input by 1e8, you will get many values that are very close to zero. To improve the precision, you need to grow the degree of the approximation polynomial (and increase the multDepth if necessary).

void EvalFunctionExample() {
std::cout << "--------------------------------- EVAL SQUARE ROOT FUNCTION ---------------------------------"
<< std::endl;
CCParams<CryptoContextCKKSRNS> parameters;

// We set a smaller ring dimension to improve performance for this example.
// In production environments, the security level should be set to
// HEStd_128_classic, HEStd_192_classic, or HEStd_256_classic for 128-bit, 192-bit,
// or 256-bit security, respectively.
parameters.SetSecurityLevel(HEStd_NotSet);
parameters.SetRingDim(1 << 10);

#if NATIVEINT == 128
usint scalingModSize = 78;
usint firstModSize = 89;
#else
usint scalingModSize = 50;
usint firstModSize = 60;
#endif
parameters.SetScalingModSize(scalingModSize);
parameters.SetFirstModSize(firstModSize);

// Choosing a higher degree yields better precision, but a longer runtime.
uint32_t polyDegree = 512;

// The multiplicative depth depends on the polynomial degree.
// See the FUNCTION_EVALUATION.md file for a table mapping polynomial degrees to multiplicative depths.
uint32_t multDepth = 11;

parameters.SetMultiplicativeDepth(multDepth);
CryptoContext<DCRTPoly> cc = GenCryptoContext(parameters);
cc->Enable(PKE);
cc->Enable(KEYSWITCH);
cc->Enable(LEVELEDSHE);
// We need to enable Advanced SHE to use the Chebyshev approximation.
cc->Enable(ADVANCEDSHE);

auto keyPair = cc->KeyGen();
// We need to generate mult keys to run Chebyshev approximations.
cc->EvalMultKeyGen(keyPair.secretKey);

std::vector<std::complex<double>> input{1, 2, 3, 4, 5, 6, 7, 8, 9};
std::vector<std::complex<double>> input2{4438.1700, 166.5460, 78721900.0000, 43896800000.0000, 171859000.0000, 31678300.0000, 3647310.0000, 945615.0000};

size_t encodedLength = input.size();
size_t encodedLength2 = input2.size();

std::transform(input2.begin(), input2.end(), input2.begin(),
                 [](std::complex<double> d) { return d/1e8; });

std::cout << "input2 values: " << input2 << std::endl;

Plaintext plaintext  = cc->MakeCKKSPackedPlaintext(input);
auto ciphertext      = cc->Encrypt(keyPair.publicKey, plaintext);

Plaintext plaintext2  = cc->MakeCKKSPackedPlaintext(input2);
auto ciphertext2      = cc->Encrypt(keyPair.publicKey, plaintext2);

double lowerBound = 0;
double upperBound = 10;

double lowerBound2 = 0;
double upperBound2 = 450;

// We can input any lambda function, which inputs a double and returns a double.
auto result = cc->EvalChebyshevFunction([](double x) -> double { return std::sqrt(x); }, ciphertext, lowerBound,
                                        upperBound, polyDegree);

auto result2 = cc->EvalChebyshevFunction([](double x) -> double { return std::sqrt(x); }, ciphertext2, lowerBound2,
                                        upperBound2, polyDegree);

Plaintext plaintextDec, plaintextDec2;
cc->Decrypt(keyPair.secretKey, result, &plaintextDec);
plaintextDec->SetLength(encodedLength);
cc->Decrypt(keyPair.secretKey, result2, &plaintextDec2);
plaintextDec2->SetLength(encodedLength2);

std::vector<std::complex<double>> expectedOutput(
    {1, 1.414213, 1.732050, 2, 2.236067, 2.449489, 2.645751, 2.828427, 3});
std::cout << "Expected output\n\t" << expectedOutput << std::endl;

std::vector<std::complex<double>> finalResult = plaintextDec->GetCKKSPackedValue();
std::cout << "Actual output\n\t" << finalResult << std::endl << std::endl;

std::vector<std::complex<double>> expectedOutput2(encodedLength2);
std::transform(input2.begin(), input2.end(), expectedOutput2.begin(),
                 [](std::complex<double> d) { return std::sqrt(d); });
std::cout << "Expected output\n\t" << expectedOutput2 << std::endl;

std::vector<std::complex<double>> finalResult2 = plaintextDec2->GetCKKSPackedValue();
std::cout << "Actual output\n\t" << finalResult2 << std::endl << std::endl;

}

Hi @andreea.alexandru , I appreciate your input. When I run your code I’m getting these results:
Expected output
[ (6.66196e-07,0) (1.29053e-07,0) (8.87254e-05,0) (0.00209516,0) (0.000131095,0) (5.62835e-05,0) (1.90979e-05,0) (9.72427e-06,0) ]
Actual output
[ (0.0206757,0) (0.0206757,0) (0.0206758,0) (0.0207271,0) (0.0206759,0) (0.0206757,0) (0.0206757,0) (0.0206757,0) ]
Is this the same result as you got? Since the values are more or less equa and doesn’t match with the expected results, I’m a bit confused.

This is the printout for both NATIVE_SIZE 64 and 128.

--------------------------------- EVAL SQUARE ROOT FUNCTION ---------------------------------
input2 values: [ (4.43817e-05,0) (1.66546e-06,0) (0.787219,0) (438.968,0) (1.71859,0) (0.316783,0) (0.0364731,0) (0.00945615,0) ]
Expected output
        [ (1,0) (1.41421,0) (1.73205,0) (2,0) (2.23607,0) (2.44949,0) (2.64575,0) (2.82843,0) (3,0) ]
Actual output
        [ (1,0) (1.41421,0) (1.73205,0) (2,0) (2.23607,0) (2.44949,0) (2.64575,0) (2.82843,0) (3,0) ]

Expected output
        [ (0.00666196,0) (0.00129053,0) (0.887254,0) (20.9516,0) (1.31095,0) (0.562835,0) (0.190979,0) (0.0972427,0) ]
Actual output
        [ (0.0211951,0) (0.0206952,0) (0.887267,0) (20.9516,0) (1.31096,0) (0.562801,0) (0.190457,0) (0.0972277,0) ]

Hi @andreea.alexandru, i become very frsutrated becahuse of the following problem.

Here is a minimal working example of my code. I’m trying to calculate the pearson correlation coefficients for my vectors but I’m getting very bad results. Since the range of values is pretty high, I tried to scale the values down before Chebyshev Evaluation. Do you have any suggestions how to improve precision in this concept?

#include "openfhe.h"
#include <cmath>

using namespace lbcrypto;

/**
 * Calculates the mean value for each element in the input vector of vectors.
 *
 * @param input The input vector of vectors of doubles.
 * @return The vector containing the mean values.
 */
std::vector<double> CalcPlaintextPearsonCorr(const std::vector<std::vector<double>>& input) {
    int n = input.size(); // Anzahl der Datenpunkte
    int m = input[0].size(); // Anzahl der Variablen

    std::vector<double> correlations;

    // Berechne die Mittelwerte
    std::vector<double> meanX(m, 0.0);
    std::vector<double> meanY(m, 0.0);

    for (int i = 0; i < n; ++i) {
        for (int j = 0; j < m; ++j) {
            meanX[j] += input[i][j];
            meanY[j] += input[i][(j + 1) % m]; // Rotation um 1 nach links
        }
    }

    for (int j = 0; j < m; ++j) {
        meanX[j] /= n;
        meanY[j] /= n;
    }            
    
    // Berechne die Kovarianz und Varianz
    std::vector<double> covariance(m, 0.0);
    std::vector<double> varX(m, 0.0);
    std::vector<double> varY(m, 0.0);

    for (int i = 0; i < n; ++i) {
        for (int j = 0; j < m; ++j) {
            double diffX = input[i][j] - meanX[j];
            double diffY = input[i][(j + 1) % m] - meanY[j]; // Rotation um 1 nach links
            covariance[j] += diffX * diffY;
            varX[j] += diffX * diffX;
            varY[j] += diffY * diffY;
        }
    }            

    //only for statistics to determine bounderies for Chebyshev evaluation
    double min=0,
            max=0;
    
    std::cout << "VarX\t\tVarY\t\tCovariance\\ttDenom\t\tSqrt(Denom)" << std::endl;
    // Berechne die Korrelationskoeffizienten
    for (int j = 0; j < m; ++j) {
        if (varX[j] == 0 || varY[j] == 0) {
            correlations.push_back(0); // Setze Korrelation auf 0, wenn eine der Varianzen gleich Null ist
        } else {
            double denom = varX[j] * varY[j];
            if (denom > max) { max = denom;}
            if (denom < min) { min = denom;}


            double correlation = covariance[j] / (sqrt(denom));
            correlations.push_back(correlation);

            std::cout << varX[j] << "\t\t" << varY[j] << "\t\t" << covariance[j] << "\t\t" << denom << "\t\t" << sqrt(denom) << std::endl;
        }
    }

    std::cout << "\nArea range Pearson Correlation: " << min << " - " << max << std::endl;

    return correlations;
}

/**
 * Prints the elements of a vector up to a specified number of elements.
 *
 * @param FirstText The text to print before the vector elements.
 * @param vec The vector to print.
 * @param numElements The number of elements to print (-1 to print all).
 */
void printVector(std::string FirstText, const std::vector<double>& vec, int numElements = -1) {
    std::cout << FirstText;
    int count = 0;
    for (const auto& element : vec) {
        if (numElements == -1 || count < numElements) {
            std::cout << element << " ";
            count++;
        } else {
            break;
        }
    }
    std::cout << std::endl;
}

/**
 * Calculates the approximation error between two vectors of complex numbers.
 *
 * @param result The vector containing the calculated complex numbers.
 * @param expectedResult The vector containing the expected complex numbers.
 * @return The approximation error.
 * @throws config_error if the vectors have different sizes.
 * 
 */
double CalculateApproximationError(const std::vector<double>& result,
                                    const std::vector<double>& expectedResult) {
    
    // Using the infinity norm
    double maxError = 0;
    for (size_t i = 0; i < result.size(); ++i) {
        double error = std::abs(result[i] - expectedResult[i]);
        if (maxError < error)
            maxError = error;
    }

    double prec = std::log2(maxError);

    if(std::isfinite(prec)) {
        return prec;
    }
    else{
        return -1.0;
    }
        
}





int main(int argc, char* argv[]) {
    std::cout << "--------------------------------- PEARSON CORRELATION COEFFICIENTS ---------------------------------"
    << std::endl;
    CCParams<CryptoContextCKKSRNS> parameters;

    parameters.SetSecurityLevel(HEStd_NotSet);
    parameters.SetRingDim(1 << 10);

    #if NATIVEINT == 128
    usint scalingModSize = 78;
    usint firstModSize = 89;
    #else
    usint scalingModSize = 59;
    usint firstModSize = 60;
    #endif
    parameters.SetScalingModSize(scalingModSize);
    parameters.SetFirstModSize(firstModSize);

    // Choosing a higher degree yields better precision, but a longer runtime.
    uint32_t polyDegree = 512;
    uint32_t multDepth = 30;

    parameters.SetMultiplicativeDepth(multDepth);
    CryptoContext<DCRTPoly> cc = GenCryptoContext(parameters);
    cc->Enable(PKE);
    cc->Enable(KEYSWITCH);
    cc->Enable(LEVELEDSHE);
    // We need to enable Advanced SHE to use the Chebyshev approximation.
    cc->Enable(ADVANCEDSHE);

    auto keyPair = cc->KeyGen();
    // We need to generate mult keys to run Chebyshev approximations.
    cc->EvalMultKeyGen(keyPair.secretKey);
    cc->EvalRotateKeyGen(keyPair.secretKey, {1,2,3,-1,-2,-3});

    std::vector<std::vector<double>> RawInput = { {14.0,47.89746762162162,9.182932, 97.5,53.902815050857185, 14.932224832748549,25.70880986063563, 42.76932656905464},
        {11.0,47.89746762162162,10.008757825825825, 189.16666666666666,53.784931171127575, 70.36906413174,13.117850483332402, 16.90789035155603},
        {3.0,47.89747762162162,10.556805873873873, 250.0,91.8909357648358, 84.27259548336639,11.339274973688191, 29.204268409585215}
    };

    std::vector<double> RawPCor = CalcPlaintextPearsonCorr(RawInput);
    std::vector<Ciphertext<DCRTPoly>> CipherInput;
    size_t encodedLength = RawInput[0].size();

    /////////////////////////////////////////
    // ENCRYPTION
    /////////////////////////////////////////

    for(const auto& input : RawInput){
        Plaintext plaintext  = cc->MakeCKKSPackedPlaintext(input);
        auto ciphertext      = cc->Encrypt(keyPair.publicKey, plaintext);
        CipherInput.push_back(ciphertext);
    }

    double lowerBound = 0;
    double upperBound = 14000000;
    
    /////////////////////////////////////////
    // CALCULATION
    /////////////////////////////////////////

    // Step 1: calculate mean values
    auto csum = cc->EvalAddMany(CipherInput);
    auto MeanResult = cc->EvalMult(csum, 1.0/CipherInput.size());

    
    // Step 2: Pearson formula
    std::vector<Ciphertext<DCRTPoly>> numvec, denomvec1, denomvec2;

    for (long unsigned int i = 0; i < CipherInput.size(); i++){
        // numerator: sum((x_i - x_avg)*(y_i - y_avg))
        auto ciphxixavg = cc->EvalSub(CipherInput[i], MeanResult);
        auto ciphyiyavg = cc->EvalRotate(ciphxixavg, 1);
        numvec.push_back(cc->EvalMult(ciphxixavg, ciphyiyavg));

        // denominator: sqrt(sum((x_i - x_avg)²) * sum((y_i - y_avg)²))
        denomvec1.push_back(cc->EvalSquare(ciphxixavg));
        denomvec2.push_back(cc->EvalSquare(ciphyiyavg));
    }

    auto numerator = cc->EvalAddMany(numvec);
    auto denom1    = cc->EvalAddMany(denomvec1); //sum function
    auto denom2    = cc->EvalAddMany(denomvec2); //sum function
    auto denom3    = cc->EvalMult(denom1, denom2);
    
    // Scale denominator by 1E8
    denom3    = cc->EvalMult(denom3, 0.00000001);
    denom3    = cc->EvalChebyshevFunction([](double x) -> double { return std::sqrt(x); }, denom3, lowerBound, upperBound, polyDegree); // root square
    denom3    = cc->EvalMult(denom3, 10000);

    auto denom4 = cc->EvalDivide(denom3, lowerBound, upperBound, polyDegree); // makes 1/x
    auto result = cc->EvalMult(denom4, numerator);

    try
    {
        // Decrypt the encrypted mean result
        Plaintext resultPCor;

        cc->Decrypt(keyPair.secretKey, result, &resultPCor);
        resultPCor->SetLength(encodedLength);

        double prec = CalculateApproximationError(RawPCor, resultPCor->GetRealPackedValue());


        // Print the results of the mean calculation
        std::cout << "\nResults of Pearson correlation coefficient calculation" << std::endl;
        printVector("\tExpected result:\t", RawPCor, RawPCor.size());
        printVector("\tActual result:\t\t", resultPCor->GetRealPackedValue(), RawPCor.size());
        std::cout << "\tPrecision: " << prec << " bits" << std::endl;
    
    }
    catch(const std::exception& e)
    {
        // Handle decryption errors
        std::cerr << "Error while decrypting Pearson correlation coefficients results" << std::endl;
        std::cerr << e.what() << '\n';
    }
}

This variable defines an enormous interval. It does not make sense to have such an interval. I suggest you to empirically study the distribution of values right before the Chebyshev evaluation.

For example, you notice that the values are (almost) always in [4, 450]. Then, you normalize the values according to this min and max and run the Chebyshev approximation in [0, 1].

1 Like