Triggered urn models for frequently asked questions (FAQ)

Abstract

We investigate a nonclassic urn model with triggers that increase the number of colors. The scheme has emerged as a model for web services that set up frequently asked questions (FAQ). We present a thorough asymptotic analysis of the FAQ urn scheme in generality that covers a large number of special cases, such as Simon urn. For instance, we consider time dependent triggering probabilities. We identify regularity conditions on these probabilities that classify the schemes into those where the number of colors in the urn remains almost surely finite or increases to infinity and conditions that tell us whether all the existing colors are observed infinitely often or not. We determine the rank curve, too. In view of the broad generality of the trigger probabilities, a spectrum of limit distributions appears, from central limit theorems to Poisson approximation, to power-laws, revealing connections to Heap's exponent and Zipf's law. A combinatorial approach to the Simon urn is presented to indicate the possibility of such exact analysis, which is important for short-term predictions. Extensive simulations on real datasets (from Amazon sales) as well as computer-generated data clearly indicate that the asymptotic and exact theory developed agrees with practice.