Based on the classical pharmacophore structure model of HDAC inhibitors, a total of 24 new substituted purine hydroxamic acid derivatives were designed, synthesized, and evaluated as HDAC inhibitors. The preliminary enzyme assay showed that most compounds can effectively inhibit HDAC1 and 2. The potent compounds, such as 5r, 5w, and 5x, showed obvious anti-proliferative activities in eight cancer cell lines. In addition, the three compounds also had high affinity and good selectivity to HDAC6 compared with SAHA. The results suggested that 2,6-disubstituted purine hydroxamate derivatives could be used as lead compounds to develop novel anti-cancer agents.