Homomorphic Encryption (HE) enables computations to be executed directly on encrypted data. As such, it is an auspicious solution for protecting the confidentiality of sensitive data without impeding its usability. However, HE does not provide any guarantees that the cryptographic material used has been honestly generated and that the computation was executed correctly on the encrypted data. Thus, even though many practical systems rely on HE to achieve strong privacy guarantees, they consider only an honest-but-curious threat model in their constructions. Although several efforts have been conducted to analyze and improve the security of HE-based systems against stronger threat models, these works have remained mostly theoretical and are still insufficient to be applicable to practical HE pipelines and real-life scenarios. Therefore, in our work, we propose and build solutions to protect HE pipelines against malicious adversaries and evaluate their performance over a wide range of use cases. We first propose VERITAS, an efficient solution that proves the correctness of homomorphic computations, without compromising the expressiveness of the HE scheme. Then, we introduce PELTA, a set of building blocks that secure HE pipelines in the multiparty setting. Our constructions can be used to verify, in a practical manner, the correctness of distributed operations without any compromise on the HE scheme. Finally, we propose CRISP to secure input verification and to prove correct encryption in settings where the client who encrypts the data is untrusted. All our constructions are a first step for evaluating the impact of the change of threat model in HE pipelines with real-life implementation constraints.