Compute Information Retrieval measures.

Functions to compute Information Retrieval measures for pairwise loss for a single group. The function returns the respective metric, or a negative value if it is undefined for the given group.

Usage

gbm.roc.area(obs, pred)

gbm.conc(x)

ir.measure.conc(y.f, max.rank = 0)

ir.measure.auc(y.f, max.rank = 0)

ir.measure.mrr(y.f, max.rank)

ir.measure.map(y.f, max.rank = 0)

ir.measure.ndcg(y.f, max.rank)

perf.pairwise(y, f, group, metric = "ndcg", w = NULL, max.rank = 0)

Arguments

obs: Observed value.
pred: Predicted value.
x: Numeric vector.
y, y.f, f, w, group, max.rank: Used internally.
metric: What type of performance measure to compute.

Value

The requested performance measure.

Details

For simplicity, we have no special handling for ties; instead, we break ties randomly. This is slightly inaccurate for individual groups, but should have only a small effect on the overall measure.

gbm.conc computes the concordance index: Fraction of all pairs (i,j) with i<j, x[i] != x[j], such that x[j] < x[i]

If obs is binary, then gbm.roc.area(obs, pred) = gbm.conc(obs[order(-pred)]).

gbm.conc is more general as it allows non-binary targets, but is significantly slower.

References

C. Burges (2010). "From RankNet to LambdaRank to LambdaMART: An Overview", Microsoft Research Technical Report MSR-TR-2010-82.

Author

Stefan Schroedl