This is the expected behavior since -1 mod 64 = 63. Zp is a class of residues and the representations {0,1,…,p-1} and {-p/2,…,-1,0,1,…p/2-1} are equivalent. You are observing the former representation. You can preprocess the decryption result to output the latter representation, i.e., -1 instead of 63.

Please read more about rings of integers and the modulo operation. You are working in Zp, not Z. You can not represent “both” -1 and 63 in Z_64, but -1 \equiv 63.

Hello, I wanted to add some additional context and clarification to @andreea.alexandru’s answer.

You can convert your result to a signed representation as follows (continuing from your code above):

// result = 63
int32_t signed_res = static_cast<int32_t>(result); // make sure to use wide enough integers
if (result >= p/2) // p/2 is your cut-off point, below it are positives, at and above it are negatives
signed_res -= static_cast<int32_t>(p)