CASH Simulation, Part 4: Smaller, faster exploratory simulations – Average pFDP

Using smaller, faster, exploratory simulations

Instead of doing a simulation with \(1K\) runs, each run with \(10K\) samples, we are downsizing it so that simulations can be done faster while still show a pattern. Right now, \(100\) runs, each run with \(5K\) samples seem adequate.

We don’t want the sample size to be too small. Small sample sizes appear to work less well with Gaussian derivatives fitting.

We are experimenting with different effect size distributions, and see how the average pFDP look with all data sets across all nominal \(q\) value levels.

Frequentist pFDR accuracy with all data sets

source("../code/gdash_lik.R")

z.mat = readRDS("../output/z_null_liver_777.rds")
se.mat = readRDS("../output/sebetahat_null_liver_777.rds")

Simulation

cashSim(z.mat, se.mat,
        nsim = 100, ngene = 1000,
        g.pi = c(0.8, 0.2), g.sd = c(0, 1), relative_to_noise = TRUE)

Simulation

cashSim(z.mat, se.mat,
        nsim = 100, ngene = 1000,
        g.pi = c(0.8, 0.2), g.sd = c(0, 2), relative_to_noise = TRUE)

Simulation

cashSim(z.mat, se.mat,
        nsim = 100, ngene = 1000,
        g.pi = c(0.8, 0.2), g.sd = c(0, 1), relative_to_noise = FALSE)

Simulation

cashSim(z.mat, se.mat,
        nsim = 100, ngene = 1000,
        g.pi = c(0.8, 0.1, 0.1), g.sd = c(0, 1, 2), relative_to_noise = TRUE)

Simulation

cashSim(z.mat, se.mat,
        nsim = 100, ngene = 1000,
        g.pi = c(0.8, 0.1, 0.1), g.sd = c(0, 1, 2), relative_to_noise = FALSE)

Simulation

cashSim(z.mat, se.mat,
        nsim = 100, ngene = 1000,
        g.pi = c(0.9, 0.1), g.sd = c(0, 1), relative_to_noise = TRUE)

Simulation

cashSim(z.mat, se.mat,
        nsim = 100, ngene = 1000,
        g.pi = c(0.9, 0.1), g.sd = c(0, 2), relative_to_noise = TRUE)

Simulation

cashSim(z.mat, se.mat,
        nsim = 100, ngene = 1000,
        g.pi = c(0.9, 0.1), g.sd = c(0, 1), relative_to_noise = FALSE)

Simulation

cashSim(z.mat, se.mat,
        nsim = 100, ngene = 1000,
        g.pi = c(0.9, 0.05, 0.05), g.sd = c(0, 1, 2), relative_to_noise = TRUE)

Simulation

cashSim(z.mat, se.mat,
        nsim = 100, ngene = 1000,
        g.pi = c(0.9, 0.05, 0.05), g.sd = c(0, 1, 2), relative_to_noise = FALSE)

Simulation

cashSim(z.mat, se.mat,
        nsim = 100, ngene = 1000,
        g.pi = c(0.85, 0.10, 0.05), g.sd = c(0, 1, 2), relative_to_noise = TRUE)

Simulation

cashSim(z.mat, se.mat,
        nsim = 100, ngene = 1000,
        g.pi = c(0.85, 0.10, 0.05), g.sd = c(0, 1, 2), relative_to_noise = FALSE)

Session information

sessionInfo()

R version 3.4.2 (2017-09-28)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.1

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] ashr_2.1-27       Rmosek_7.1.3      PolynomF_0.94     cvxr_0.0.0.9400  
[5] REBayes_0.85      Matrix_1.2-11     SQUAREM_2017.10-1 EQL_1.0-0        
[9] ttutils_1.0-1    

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.13      knitr_1.17        magrittr_1.5     
 [4] MASS_7.3-47       pscl_1.5.2        doParallel_1.0.11
 [7] lattice_0.20-35   foreach_1.4.3     stringr_1.2.0    
[10] tools_3.4.2       parallel_3.4.2    grid_3.4.2       
[13] git2r_0.19.0      iterators_1.0.8   htmltools_0.3.6  
[16] yaml_2.1.14       rprojroot_1.2     digest_0.6.12    
[19] gmp_0.5-13.1      codetools_0.2-15  evaluate_0.10.1  
[22] rmarkdown_1.6     stringi_1.1.5     compiler_3.4.2   
[25] backports_1.1.1   truncnorm_1.0-7