Harpagide (1) and harpagoside (2) are two iridoid glycosides existing in many medicinal plants. Although they are believed to be the main bioactive compounds related to the anti-inflammatory efficacy of these plants, the mechanisms of their anti-inflammatory activities remain unclear. The results of our present study showed that 1 and 2 had no effects on inhibitions of cyclooxygenase (COX)-1/2 enzyme activity, tumor necrosis factor-α (TNF-α) release, and nitric oxide (NO) production in vitro. However, the hydrolyzed products of 1 and 2 with β-glucosidase treatment showed a significant inhibitory effect on COX-2 activity at 2.5-100 μM in a concentration-dependent manner. Our further study revealed that the hydrolyzed 2 product was structurally the same as the hydrolyzed 1 product (H-harpagide (3)). The structure of 3 was 2-(formylmethyl)-2,3,5-trihydroxy-5-methylcyclopentane carbaldehyde, with a backbone similar to prostaglandins and COX-2 inhibitors such as celecoxib. All of them have a pentatomic ring with two adjacent side chains. The result of molecular modeling and docking study showed that 3 could bind to the COX-2 active domain well through hydrophobic and hydrogen-bonding interactions, whereas 1 and 2 could not, implying that the hydrolysis of the glycosidic bond of 1 and 2 is a pre-requisite step for their COX-2 inhibitory activity.