A fragment-based lead discovery approach was applied to Pyruvate Dehydrogenase Kinases (PDHKs) to discover inhibitors against the ATP binding site with novel chemotypes. X-ray fragment screening toward PDHK4 provided a fragment hit 1 with a characteristic interaction in a deep pocket of the ATP binding site. While known inhibitors utilize several water molecules in a deep pocket to form water-mediated hydrogen bond interactions, the fragment hit binds deeper in the pocket with a hydrophobic group. Displacement of a remaining water molecule in the pocket led to the identification of lead compound 7 with a notable improvement in inhibition potency. This lead compound possessed high ligand efficiency (LE) and showed decent selectivity profile. Two additional lead compounds 10 and 13 with new scaffolds with tricyclic and bicyclic cores were generated by merging structural information of another fragment hit 2. The characteristic interaction of these novel inhibitors in a deep pocket provides new structural insights about PDHKs ATP binding site and opens a novel direction for the development of PDHKs inhibitors.