With the aim of discovering a novel class of non-phosphorus-based fructose-1,6-bisphosphatase (FBPase) inhibitors, a series of 2,5-diphenyl-1,3,4-oxadiazoles were synthesized based on the hit compound (1) resulting from a high-throughput screening (HTS). Structure-activity relationship (SAR) studies led to the identification of several compounds with comparable inhibitory activities to AMP, the natural allosteric inhibitor of FBPase. Notably, compound 22 and 27b, bearing a terminal carboxyl or 1H-tetrazole, demonstrated remarkable inhibition to gluconeogenesis (GNG). In addition, both inhibition and binding mode to the enzyme were investigated by enzymatic kinetics and in silico experiments for representative compounds 16 and 22.