Small molecules featuring a hydroxamic acid or a benzamide zinc binding group (ZBG) are the most thoroughly studied histone deacetylase (HDAC) inhibitors. However, concerns about the pharmacokinetic liabilities of the hydroxamic acid moiety and potential metabolic toxicity of the aniline portion of benzamide HDAC inhibitors have stimulated research efforts aimed at discovering alternative ZBGs. Here we report the 2-(oxazol-2-yl)phenol moiety as a novel ZBG that can be used to produce compounds that are potent HDAC inhibitors. A series of analogues with this novel ZBG have been synthesized, and these analogues exhibit selective inhibition against HDAC1 as well as the class IIb HDACs (HDAC6 and HDAC10). Compound 10 possesses an IC50 value of 7.5 μM in the MV-4-11 leukemia cell line, and induces a comparable amount of acetylated histone 3 lysine 9 (H3K9) and p21Waf1/CIP1 as 0.5 μM of SAHA. Modeling of compound 10 in the active site of HDAC2 demonstrates that the 2-(oxazol-2-yl)phenol moiety has a zinc-binding pattern similar to benzamide HDAC inhibitors.