Inhibition of histone deacetylase 6 (HDAC6) has emerged as a promising therapeutic strategy for the treatment of cancer, chemotherapy-induced peripheral neuropathy, and neurodegenerative disease. The recent X-ray crystal structure determination of HDAC6 enables an understanding of structural features directing affinity and selectivity in the active site. Here, we present the X-ray crystal structures of five HDAC6-inhibitor complexes that illuminate key molecular features of the inhibitor linker and capping groups that facilitate and differentiate binding to HDAC6. In particular, aromatic and heteroaromatic linkers nestle within an aromatic cleft defined by F583 and F643, and different aromatic linkers direct the capping group toward shallow pockets defined by the L1 loop, the L2 loop, or somewhere in between these pockets. These results expand our understanding of factors contributing to the selective inhibition of HDAC6, particularly regarding interactions that can be targeted in the region of the L2 pocket.