A series of JMJD2A inhibitors had been designed by analyzing the binding mode of 5-carboxy-8-hydroxyquinoline (5-carboxy-8-HQ) with JMJD2A. The inhibitory activity of the synthesized compounds against JMJD2A was determined, followed by docking simulations to understand the structure-activity relationships. Compounds with potent JMJD2A inhibitory activity demonstrated outstanding selectivity for JMJD2A over PHD2. Several potent compounds were selected to evaluate their anti-proliferative activity on tumor cell lines. Among them, compound 6p displayed the best anti-proliferative activity. Based on these in vitro biological data, seven compounds were chosen to determine their physicochemical properties. Compound 6p displayed good aqueous solubility and better permeability than 5-carboxy-8-HQ. Our data recognized that compound 6p could be considered as a starting point for development of new JmjC inhibitors.