User-expertise modeling with empirically derived probabilistic implication networks

The application of user-expertise modeling for adaptive interfaces is confronted with a number of difficult challenges, namely, efficiency and reliability, the cost-benefit ratio, and the practical usability of user modeling techniques. We argue that many of these obstacles can be overcome by standard, automatic means of performing knowledge assessment. Within this perspective, we present the basis of a probabilistic user modeling approach, the POKS technique, which could serve as a standard user-expertise modeling tool. The POKS technique is based on the cognitive theory of knowledge structures: a formalism for the representation of the order in which we learn knowledge units (KU). The technique permits the induction of knowledge structures from a small number of empirical data cases. It uses an evidence propagation scheme within these structures to infer an individual's knowledge state from a sample of KU. The empirical induction technique is based, in part, on statistical hypothesis testing over conditional probabilities that are determined by the KUs' learning order. Experiments with this approach show that the technique is successful in partially inferring an individual's knowledge state, either through the monitoring of a user's behavior, or through a selective questioning process. However, the selective process, based on entropy minimization, is shown to be much more effective in reducing the standard error score of knowledge assessment than random sampling.