Several human diseases are associated with aberrant epigenetic pathways mediated by histone deacetylases (HDACs), especially HDAC6, a class IIb HDACs, which has emerged as an attractive target for neurodegenerative and autoimmune disease therapeutics. In a previous study, we developed the novel HDAC6-selective inhibitor 9a ((E)-N-hydroxy-4-(2-styrylthiazol-4-yl)butanamide) and showed that it has anti-sepsis activity in vivo. In this study, we conducted structure-activity relationship (SAR) studies to optimize the activity and selectivity of HDAC6, synthesizing its derivatives with various aliphatic linker sizes and cap structures. We identified 6u ((E)-N-hydroxy-3-(2-(4-fluorostyryl)thiazol-4-yl)propanamide), which has nanomolar inhibition activity and a 126-fold selectivity for HDAC6 over HDAC1. Through the docking analyses of 6u against HDAC subtypes, we revealed the importance of the optimal aliphatic linker size, as well as the electronic substituent effect and rigidity of the aryl cap group. Thus, we suggest a new rationale for the design of HDAC6-selective inhibitors.