Three novel series of 1,2,3-triazole and 1,3,4-oxadiazole derivatives of imatinib were prepared and evaluated in vitro for their cytostatic effects against a human chronic myeloid leukemia (K562), acute myeloid leukemia (HL60), and human leukemia stem-like cell line (KG1a). The structure-activity relationship was analyzed by determining the inhibitory rate of each imatinib analog. Benzene and piperazine rings were necessary groups in these compounds for maintaining inhibitory activities against the K562 and HL60 cell lines. Introducing a trifluoromethyl group significantly enhanced the potency of the compounds against these two cell lines. Surprisingly, some compounds showed significant inhibitory activities against KG1a cells without inhibiting common leukemia cell lines (K562 and HL60). These findings suggest that these compounds are able to inhibit leukemia stem-like cells.