A closer look at Knockoff on heavy correlation design
Lei Sun
2018-02-20
Last updated: 2018-02-23
Code version: 2f0b137
Introduction
Knockoff has 3 steps.
Generate knockoff variables, which keep the same correlation structure as original variables but has no effect on the response.
Generate test statistics such that these statistics tend to be large positive number for non-null variables but iid positive or negative for null variables.
Find a cutoff threshold for those test statistics to control the estimated FDR under \(q\).
The default knockoff::knockoff.filter function uses SDP construction in step 1 and LASSO-related statistics in step 2. However, we’ve found that Knockoff coded in this way failed to control FDR in simulations when variables are generated such that \(\hat\beta\) has heavy average absolute pairwise correlation, which seems to contradict Thoerem 2 in the Knockoff paper. Now we take a closer look to see what went wrong.
In step 1, we use two construction methods: equi and sdp. sdp is believed to be more powerful. In step 2, we use two statistics: marginal and lasso-related. lasso-related is believed to be more powerful.
n <- 3e3
p <- 1e3
k <- 50
d <- 7
q <- 0.1Fixed \(X\) Knockoffs: 1000 simulation trials
Model \(X\) Knockoffs: 100 simulation trials
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Warning: doMC is not installed. Without parallelization, the statistics
will be slower to compute
Session information
sessionInfo()R version 3.4.3 (2017-11-30)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.3
Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] ggplot2_2.2.1 reshape2_1.4.3 knockoff_0.3.0
loaded via a namespace (and not attached):
[1] Rcpp_0.12.14 knitr_1.19 magrittr_1.5 munsell_0.4.3
[5] colorspace_1.3-2 rlang_0.1.6 stringr_1.2.0 plyr_1.8.4
[9] tools_3.4.3 grid_3.4.3 gtable_0.2.0 git2r_0.21.0
[13] htmltools_0.3.6 yaml_2.1.16 lazyeval_0.2.1 rprojroot_1.3-2
[17] digest_0.6.14 tibble_1.4.1 evaluate_0.10.1 rmarkdown_1.8
[21] labeling_0.3 stringi_1.1.6 pillar_1.0.1 compiler_3.4.3
[25] scales_0.5.0 backports_1.1.2 This R Markdown site was created with workflowr