Glycogen synthase kinase 3β (GSK-3β) plays a key role in insulin metabolizing pathway and therefore inhibition of the enzyme might provide an important therapeutic approach for treatment of insulin resistance and type 2 diabetes. Recently, discovery of ATP noncompetitive inhibitors is gaining importance not only due to their generally increased selectivity but also for the potentially subtle modulation of the target. These kinds of compounds include allosteric modulators and substrate competitive inhibitors. Here we reported two benzothiazinone compounds (BTO), named BTO-5h (IC50=8 μM) and BTO-5s (IC50=10 μM) as novel allosteric modulator and substrate competitive inhibitor of GSK-3β, respectively. Their different action modes were proved by kinetic experiments. Furthermore, BTO-5s was selected to check the kinases profile and showed little or even no activity to a panel of ten protein kinases at 100 μM, indicating it has good selectivity. Docking studies were performed to give suggesting binding modes which can well explain their impacts on the enzyme. Moreover, cell experiments displayed both compounds reduced the phosphorylation level of glycogen synthase in an intact cell, and greatly enhanced the glucose uptake in both HpG2 and 3T3-L1 cells. All of these results suggested BTO-5s and BTO-5h maybe have potentially therapeutic value for anti-diabetes. The results also offer a new scaffold for designing and developing selective inhibitors with novel mechanisms of action.