Histone deacetylase (HDAC) is a validated target for pursuing anticancer agents. However, obtaining a selective inhibitor against a given HDAC member remains a significant challenge. We report here the use of 1-hydroxypyridine-2-thione (1HPT) as a key pharmacophore for zinc-binding can result in highly selective HDAC inhibitors. 1HPT-6-carboxylic acid exhibits selective inhibition of HDAC6 with an IC50 of 150 nM that corresponds to a remarkable 0.9 ligand efficiency. Two analogs with simple amino acids shows nearly 600-fold selectivity among the eleven zinc-dependent HDACs. At low micromolar concentration these compounds inhibit the growth of HDAC8-overexpressing chronic myelogenous leukemia cells and specific form of acute myelogenous leukemia cells. Their potential mode of binding was examined by molecular docking and their stability was assessed in mouse and human plasma. Together the results suggest 1HPT analogs exhibit promising therapeutic potential for further development as anticancer agents to treat leukemia.