Package {APML0}

Augmented and Penalized Minimization Method L0

Description

Fit linear, logistic and Cox models regularized with L0, lasso (L1), elastic-net (L1 and L2), or net (L1 and Laplacian) penalty, and their adaptive forms. The package solves the L0 penalty problem by simultaneously selecting regularization parameters and performing hard-thresholding or selecting the number of non-zeros. This augmented and penalized minimization method provides an approximation solution to the L0 penalty problem but runs as fast as L1 regularization. A one-step coordinate descent algorithm exploits sparsity and handles very high dimensional data efficiently.

Details

Main function: APML0
Print method: print.APML0

Author(s)

Xiang Li, Shanghong Xie, Donglin Zeng and Yuanjia Wang
Maintainer: Xiang Li spiritcoke@gmail.com

References

Li, X., Xie, S., Zeng, D., Wang, Y. (2018). Efficient l0-norm feature selection based on augmented and penalized minimization. Statistics in Medicine, 37(3), 473–486. doi:10.1002/sim.7526

Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J. (2011). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1), 1–122.

Friedman, J., Hastie, T., Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1). doi:10.18637/jss.v033.i01

Examples

set.seed(1213)
N=100; p=30; p1=5
x=matrix(rnorm(N*p), N, p)
beta=rnorm(p1)
xb=x[, 1:p1] %*% beta
y=rnorm(N, xb)
fiti=APML0(x, y, penalty="Lasso", nlambda=10)
fiti2=APML0(x, y, penalty="Lasso", nlambda=10, nfolds=10)

Fit a Model with Various Regularization Forms

Description

Fit linear, logistic and Cox models regularized with L0, lasso (L1), elastic-net (L1 and L2), or net (L1 and Laplacian) penalty, and their adaptive forms, such as adaptive lasso / elastic-net and net adjusting for signs of linked coefficients.

It solves the L0 penalty problem by simultaneously selecting regularization parameters and performing hard-thresholding (or selecting the number of non-zeros). This augmented and penalized minimization method provides an approximation solution to the L0 penalty problem and runs as fast as L1 regularization.

The function uses a one-step coordinate descent algorithm and runs extremely fast by taking into account the sparsity structure of coefficients. It can handle very high dimensional data.

Usage

APML0(x, y, weights=NULL, family=c("gaussian", "binomial", "cox"),
penalty=c("Lasso","Enet", "Net"),
Omega=NULL, alpha=1.0, lambda=NULL, nlambda=50, rlambda=NULL,
wbeta=rep(1,ncol(x)), sgn=rep(1,ncol(x)), KK=log(nrow(x)),
nfolds=1, foldid=NULL, cvll=TRUE, icutB=TRUE, ncutB=10,
ifast=TRUE, isd=FALSE, iysd=FALSE, ifastr=TRUE, keep.beta=FALSE,
thresh=1e-6, maxit=1e+5, threshC=1e-5, maxitC=1e+2, threshP=1e-5)

Arguments

Details

A one-step coordinate descent algorithm is applied for each lambda. For family = "cox", ifastr = TRUE uses an efficient risk-set update and may stop before all nlambda values are evaluated; use ifastr = FALSE to force evaluation of all values.

x is always standardized internally and estimates are returned on the original scale. For family = "gaussian", y is centered (no intercept is returned).

L0 variable selection is always performed in addition to the regularized fit. Without cross-validation, the support is selected by a BIC-type criterion nzero \times KK - 2 \times \mathrm{logLik} (see KK). With cross-validation, selection uses hard-thresholding (icutB = TRUE) or the number of non-zeros (icutB = FALSE).

Value

An object of S3 class "APML0" containing:

Warning

The function may return NULL if no valid lambda values are found.

Author(s)

Xiang Li, Shanghong Xie, Donglin Zeng and Yuanjia Wang
Maintainer: Xiang Li spiritcoke@gmail.com

References

Li, X., Xie, S., Zeng, D., Wang, Y. (2018). Efficient l0-norm feature selection based on augmented and penalized minimization. Statistics in Medicine, 37(3), 473–486. doi:10.1002/sim.7526

Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J. (2011). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1), 1–122.

Friedman, J., Hastie, T., Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1). doi:10.18637/jss.v033.i01

See Also

print.APML0

Examples

###  Linear model  ###
set.seed(1213)
N=100; p=30; p1=5
x=matrix(rnorm(N*p), N, p)
beta=rnorm(p1)
xb=x[, 1:p1] %*% beta
y=rnorm(N, xb)

fiti=APML0(x, y, penalty="Lasso", nlambda=10)
fiti2=APML0(x, y, penalty="Lasso", nlambda=10, nfolds=10)


###  Logistic model  ###
set.seed(1213)
N=100; p=30; p1=5
x=matrix(rnorm(N*p), N, p)
beta=rnorm(p1)
xb=x[, 1:p1] %*% beta
y=rbinom(n=N, size=1, prob=1.0/(1.0+exp(-xb)))

fiti=APML0(x, y, family="binomial", penalty="Lasso", nlambda=10)
fiti2=APML0(x, y, family="binomial", penalty="Lasso", nlambda=10, nfolds=10)


###  Cox model  ###
set.seed(1213)
N=100; p=30; p1=5
x=matrix(rnorm(N*p), N, p)
beta=rnorm(p1)
xb=x[, 1:p1] %*% beta
ty=rexp(N, exp(xb))
td=rexp(N, 0.05)
tcens=ifelse(td < ty, 1, 0)
y=cbind(time=ty, status=1-tcens)

fiti=APML0(x, y, family="cox", penalty="Lasso", nlambda=10)
fiti2=APML0(x, y, family="cox", penalty="Lasso", nlambda=10, nfolds=10)

Print an APML0 Object

Description

Print a summary of results along the regularization path.

Usage

## S3 method for class 'APML0'
print(x, digits = 4, ...)

Arguments

Details

Prints the model family and penalty type, the fit table along the lambda path, and, when cross-validation with L0 selection was performed, the fit0 table for the selected solution.

Value

Called for its side effect (printing). Returns x invisibly.

Author(s)

Xiang Li, Shanghong Xie, Donglin Zeng and Yuanjia Wang
Maintainer: Xiang Li spiritcoke@gmail.com

See Also

APML0

Examples

set.seed(1213)
N=100; p=30; p1=5
x=matrix(rnorm(N*p), N, p)
beta=rnorm(p1)
xb=x[, 1:p1] %*% beta
y=rnorm(N, xb)
fiti=APML0(x, y, penalty="Lasso", nlambda=10, nfolds=10)
print(fiti)