As the continuous rise in the incidence of antibiotic resistance, it is urgent to develop novel chemical scaffolds with antibacterial activities to control the spread of resistance to conventional antibiotics. In this study, a series of phenylthiazole and phenylthiophene pyrimidindiamine derivatives were designed and synthesized by modifying the hit compound (N-isobutyl-N-((4-methyl-2-phenylthiazol-5-yl)methyl) pyrimidine-2,4-diamine) and their antibacterial activities were evaluated both in vitro and in vivo. Among the tested compounds, compound 14g (N-((5-(3-bromophenyl)thiophen-2-yl)methyl)-N-isobutylpyrimidine-2,4-diamine) displayed the best antibacterial activities, which was not only capable of inhibiting E. coli and S. aureus growth at concentrations as low as 2 and 3 μg/mL in vitro, but also efficacious in a mice model of bacteremia in vivo. Unlike conventional antibiotics, compound 14g was elucidated to mainly destroy the bacterial cell membrane, with the dissipation of membrane potential and leakage of contents, ultimately leading to cell death. The destruction of cell structure is challenging to induce bacterial resistance, which suggested that compound 14g may be a kind of promising alternatives to antibiotics against bacteria.