Inhibition of FBPase is considered a promising way to reduce hepatic gluconeogenesis and therefore could be a potential approach to treat type 2 diabetes. Herein we report the discovery of a series of purine phosphonic acids as AMP mimics targeting the AMP site of FBPase, which was achieved using a structure-guided drug design approach. These non-nucleotide purine analogues inhibit FBPase in a similar manner and with similar potency as AMP. More importantly, several purine analogues exhibited potent cellular and in vivo glucose-lowering activities, thus achieving proof-of-concept for inhibiting FBPase as a drug discovery target. For example, compounds 4.11 and 4.13 are as equipotent as AMP with regard to FBPase inhibition. Furthermore, compound 4.11 inhibited glucose production in primary rat hepatocytes and significantly lowered blood glucose levels in fasted rats.