In the light of known HDAC inhibitors, 33 carboxylic acid derivatives were tested to understand the structural requirements for HDAC inhibition activity. Several modifications were applied to develop the structure-activity relationships of carboxylic acid HDAC inhibitors. HDAC inhibition activities were investigated in vitro by using HeLa nuclear extract in a fluorimetric assay. Molecular docking was also carried out for the human HDAC8 enzyme in order to predict inhibition activity and the 3D poses of inhibitor-enzyme complexes. Of these compounds, caffeic acid derivatives such as chlorogenic acid and curcumin were found to be highly potent compared to sodium butyrate, which is a well-known HDAC inhibitor.