Cyclic nucleotide phosphodiesterase type 5 (PDE5) is a prime drug target for treating the diseases associated with a lower level of the cyclic guanosine monophosphate (cGMP), which is a specific substrate for PDE5 hydrolysis. Here we report a series of novel PDE5 inhibitors with the new scaffold of the monocyclic pyrimidin-4(3H)-one ring developed using the structure-based discovery strategy. In total, 37 derivatives of the pyrimidin-4(3H)-ones, were designed, synthesized, and evaluated for their inhibitory activities to PDE5, resulting in 25 compounds with IC50 ranging from 1 to 100 nM and 11 compounds with IC50 ranging from 1 to 10 nM. Compound 5, 5,6-diethyl-2-[2-n-propoxy-5-(4-methyl-1-piperazinylsulfonyl)phenyl]pyrimid-4(3H)-one, the most potent compound, has an excellent IC50 (1.6 nM) in vitro and a good efficacy in a rat model of erection. It thus provides a potential candidate for the further development into a new drug targeting PDE5.