Inhibition of histone deacetylase (HDAC) has been regarded as a potential therapeutic approach for treatment of multiple diseases including cancer. Based on pharmacophore model of HDAC inhibitors, a series of quinoline-based N-hydroxycinnamamides and N-hydroxybenzamides were designed and synthesized as potent HDAC inhibitors. All target compounds were evaluated for their in vitro HDAC inhibitory activities and anti-proliferative activities and the best compound 4a surpass Vorinostat in both enzymatic inhibitory activity and cellular anti-proliferative activity. In terms of HDAC isoforms selectivity, compounds 4a exhibited preferable inhibition for class I HDACs, especially for HDAC8, the IC50 value (442 nM) was much lower than that of Vorinostat (7468 nM). Subsequently, we performed class I & IIa HDACs whole cell enzyme assay to evaluate inhibitory activity in whole cell context. Compounds 4a and 4e displayed much better cellular activity for class I HDACs than that for class IIa HDACs, which indicated that 4a and 4e might be potent class I HDAC inhibitors. Meanwhile, flow cytometry analysis showed that compound 4a and 4e can promote cell apoptosis in vitro.