Buildings play a pivotal role in the ongoing worldwide energy transition, accounting for 30% of the global energy consumption. With traditional engineering solutions reaching their limits to tackle such large-scale problems, data-driven methods and Machine Learning (ML) tools are gaining momentum. In particular, Neural Networks (NNs) are becoming prominent, both for modeling tasks or as control policies in Deep Reinforcement Learning (DRL) agents. Despite their remarkable achievements, NNs however suffer from poor generalization to unseen data and may fail to adhere to the fundamental laws of physics. Consequently, the first part of this thesis focuses on merging physical insights into NNs, proposing the novel Physically Consistent Neural Network (PCNN) architecture. In PCNNs, a physics-inspired module leveraging established domain expertise runs in parallel to a black-box NN to ensure the model is aligned with the principles of physics. Applying PCNNs to multi-zone building thermal modeling, we prove that they are consistent with the laws of thermodynamics by design, as required, while simultaneously achieving state-of-the-art modeling performance among data-driven methods on a case study. The second part of this thesis starts by discussing the characteristics of an ideal building controller, identifying model-free DRL control policies as strong candidates. We then illustrate how DRL agents can not only significantly surpass baseline controllers but also achieve near-optimal performance. Finally, we propose to enforce expert-designed rules on DRL agents to avoid suboptimal decisions and accelerate learning. Collectively, these investigations on single-zone temperature case studies point toward the potential of DRL agents being deployed from scratch in buildings and autonomously acquiring near-optimal behaviors within complex environments in a reasonable amount of time, bypassing the need for engineering-heavy control solutions. Despite their versatile capabilities, however, the black-box nature of NNs may not be ideal in practice. To tackle this issue, the last part of this thesis focuses on using automatic backpropagation for System Identification (SI), extending beyond building-specific contexts. We introduce SIMBa, a general-purpose SI toolbox leveraging ML tools to identify structured linear state-space models from data. SIMBa facilitates the seamless incorporation of prior domain expertise while simultaneously ensuring model stability and achieving impressive performance across various SI tasks from both simulated and real-world data. Finally, we present one extension of SIMBa to identify irreversible port-Hamiltonian dynamics, creating nonlinear models that inherently adhere to the laws of thermodynamics and paving the way for the identification of general structured nonlinear systems through the power of backpropagation. Altogether, this thesis investigates diverse strategies to merge prior knowledge and ML techniques, encompassing both the adaptation of NNs to align with underlying physics and the utilization of automatic backpropagation to extract structured models from data. Overall, our results hint at the effectiveness of merging both worlds, leveraging the large-scale capabilities of ML tools to solve complex problems while anchoring their solutions in the foundational expertise of domain-specific knowledge.