Diffusion magnetic resonance imaging is a non-invasive technique capable of quantifying the diffusion process of water molecules in living biological tissues. Its main application is the study of the local geometry and wiring pattern of the human brain white matter. A large number of neuroscience research studies and clinical applications have been conducted in the last decade. Many of these studies are based on different intra-voxel models of molecular diffusion, which in turn, require different sampling schemes to collect the data and fitting algorithms. In order to facilitate the widespread use of this technique, we developed a novel software called pHARDI. The purpose of pHARDI is twofold: (1) to provide in a single toolkit an extensive and diverse set of reconstruction methods for different sampling protocols, and (2) to accelerate the reconstruction process by means of high quality linear algebra libraries. The toolkit has a layer-based design allowing to parallelise the computations via multiple accelerators in a wide range of devices, including co-processors, multi-core CPU, and GPU devices.The experimental evaluation shows that pHARDI attains, on average, a speed-up of 8X over equivalent Matlab implementations.