We report here the strategy used in our research group to find a new class of histone deacetylase (HDAC) inhibitors. A series of N-substituted 4-alkylpiperazine and 4-alkylpiperidine hydroxamic acids, corresponding to the basic structure of HDAC inhibitors (zinc binding moiety-linker-capping group) has been designed, prepared, and tested for HDAC inhibition. Linker length and aromatic capping group connection were systematically varied to find the optimal geometric parameters. A new series of submicromolar inhibitors was thus identified, which showed antiproliferative activity on HCT-116 colon carcinoma cells.