Fragment-based screening by SPR enabled the discovery of chemical diverse fragment hits with millimolar binding affinities to the peptidyl-prolyl isomerase Cyclophilin D (CypD). The CypD protein crystal structures of 6 fragment hits provided the basis for subsequent medicinal chemistry optimization by fragment merging and linking yielding three different chemical series with either urea, oxalyl or amide linkers connecting millimolar fragments in the S1' and S2 pockets. We successfully improved the in vitro CypD potencies in the biochemical FP and PPIase assays and in the biophysical SPR binding assay from millimolar towards the low micromolar and submicromolar range by >1000-fold for some fragment derivatives. The initial SAR together with the protein crystal structures of our novel CypD inhibitors provide a suitable basis for further hit-to-lead optimization.