Zinc-dependent histone deacetylases (HDACs) are important epigenetic regulators that have become important drug targets for treating cancer. Although five HDAC inhibitors have been approved for treating several cancers, there is still a huge demand on discovering new HDAC inhibitors to explore the therapeutic potentials for treating solid tumor cancers. Substrate mimics are a powerful rational design approach for the development of potent inhibitors. Here we describe the rational design, synthesis, biological evaluation, molecular docking and in vivo efficacy study of a class of HDAC inhibitors using N<sup>ε</sup>-acetyl lysine mimics that are derived from cysteine. As a result, compounds 7a, 9b and 13d demonstrated pan-HDAC inhibition and broad cytotoxicity against several cancer cell lines, comparable to the approved HDAC inhibitor SAHA. Furthermore, 13d significantly inhibited tumor growth in a A549 xenograft mice model without any obvious weight loss, supporting that the cysteine-derived acetyl lysine mimics are promising HDAC inhibitors with therapeutic potentials for treating cancer.