The goal of this thesis is the development and the analysis of numerical methods for problems where the unknown is a curve on a smooth manifold. In particular, the thesis is structured around the three following problems: homotopy continuation, curve interpolation and integration of ordinary differential equations. To accommodate the manifold constraint, all the proposed methods feature as a central ingredient the concept of retractions, as extensively developed in the context of Riemannian optimization methods. A retraction can be thought as a generic device for crafting portions of manifold-constrained curves which are in general computationally cheaper to evaluate than the geodesics defining the Riemannian exponential map. Yet, the axiomatic definition of a retraction reveals to be rich enough for algorithms originally defined on Euclidean spaces to be adapted to the manifold setting using retractions while maintaining properties that are analogous to their Euclidean ancestor. We provide this type of analysis for the methods proposed in the thesis and we showcase the performance of the algorithms with experiments involving matrix manifolds, notably the fixed-rank matrix manifold. First, we consider a generalization of numerical continuation methods for their application to Riemannian optimization problems. In practice, we propose a retraction-based path-following numerical continuation algorithm for efficiently solving a sequence of Riemannian optimization problems of which the last is the actual problem of interest. Then, we address the problem of Hermite interpolation, whereby a sequence of manifold points are interpolated by a manifold curve whose velocity is prescribed at each interpolation point. For this, we introduce a generalization of the de Casteljau algorithm where suitably chosen retraction curves replace the straight lines of the original algorithm. Lastly, we tackle numerical integration of manifold-constrained ordinary differential equations, in particular for equations evolving on low-rank matrix manifolds encountered in the field of dynamical low-rank approximation. We derive two methods defined using retractions which exhibit second-order convergence of the approximation error with respect to the time integration step.