Novel pyrrole and pyrrolopyrimidine scaffold-based sulfonamides were designed and synthesized. The carbonic anhydrase (CA) inhibition ability of all derivatives was assessed against the human (h) cytosolic isoforms hCA I and II and the transmembrane, tumor-associated isoforms hCA IX and XII. Some of these sulfonamides were 6-8 fold more potent than the reference drug acetazolamide (AZA, Ki = 5.7 nM)) against hCA XII showing subnanomolar activity. The in vitro cytotoxicity of these derivatives was evaluated against MCF-7, where some derivatives were more cytotoxic than doxorubicin (IC50 = 8.02 μM) displaying IC50 values between 6.46 and 7.56 μM. Docking of these sulfonamides with CA XII was performed and their binding modes were comparable with that of AZA.