Performance of cross-validation and likelihood based strategies to select tuning factors for penalized estimation

Abstract

Using simulation, the efficacy of penalized maximum likelihood estimation of genetic covariances when employing different strategies to determine the necessary tuning parameter is investigated. It is shown that errors in estimating the tuning factor from the data using cross-validation can reduce the percentage reduction in average loss at modest sample sizes from 70% or more to 60% or less. Mild penalization by limiting the change in likelihood is shown to perform well and to yield choices which are highly correlated with those based on the population parameters. Likelihood based selection of the tuning parameter is recommended as a simple and effective alternative to cross-validation.