Histone deacetylase 6 (HDAC6) catalyzes the removal of an acetyl group from lysine residues of several non-histone proteins. Here we report the preparation of thiazole-, oxazole-, and oxadiazole-containing biarylhydroxamic acids by a short synthetic procedure. We identified them as selective HDAC6 inhibitors by investigating the inhibition of recombinant HDAC enzymes and the protein acetylation in cells by Western blotting (tubulin vs histone acetylation). The most active compounds exhibited nanomolar potency and high selectivity for HDAC6. For example, an oxazole hydroxamate inhibits HDAC6 with an IC50 of 59 nM and has a selectivity index of >200 against HDAC1 and HDAC8. This is the first report showing that the nature of a heterocycle directly connected to a zinc binding group (ZBG) can be used to modulate subtype selectivity and potency for HDAC6 inhibitors to such an extent. We rationalize the high potency and selectivity of the oxazoles by molecular modeling and docking.