Public policy evaluation has traditionally been conducted in a context constrained by legal, institutional and political red tape. The arrival of digitalisation adds a new layer to public policy evaluation processes. Assessing the quality of technologies, tools and processes in use requires that evaluators revise their skillset and methods. This chapter aims to illustrate the challenges that digitalisation entails for public policy evaluation and evaluators. First, we examine how the evaluation of digitalisation-related public policies differs from their "analog" counterparts using empirical findings from a Swiss local government project that introduced digital education in primary and secondary schools. Second, we discuss how the responsible implementation of Artificial Intelligence (AI) should be conducted and the impact of AI on practical evaluation. This chapter contributes to the burgeoning discussion of prospects of public policy evaluation amidst the proliferation of digitalisation and AI in the public sector.