Each cryptocontext has two (non-key-dependent) parts: the metadata and precomputed parameters. The metadata (like all lattice, polynomial, and other crypto-related parameters) get serialized as they are based on certain input parameters. The precomputed parameters get regenerated on the fly (when deserializing). The cryptocontext metadata can hypothetically get regenerated but it can be error-prone and very risky in practice (and even slow). Working with serialized cryptocontext metadata is certainly both better from robustness and security perspectives.
The message you got is related to the type check between cryptocontext and ciphertext. The pointer referring to the cryptocontext in the ciphertext does not match the pointer for the cryptocontext with which you are working.
In general, it is always good to serialize and deserialize cryptocontext metadata to guarantee expected/secure behavior. Recreating all core crypto parameters on the fly would be error-prone and potentially insecure.
If you want to reduce the time of deserialization (to avoid regenerating cryptocontext precomputations multiple times), you might want to read Deserializing Ciphertexts is slow