Hi everyone,

I'm having a hard time merging my conceptual approach to bootstrapping Elastic Net modeling, and would appreciate insight or opinions.

I have built three cross-validated ENet models with continuous predictor variables (severity of different symptoms) and 16S microbiome data, such that my outcome selects which taxa are most relevant to the severity of each symptom. I used a nested model to tune parameters and a 10-fold CV to select my final taxa as my primary models.

I want to utilize some method to capture the variance in selected features and settled on the bootstrapping method of Variable Inclusion Probability (VIP), which generates the probability of feature inclusion over 200 models with random seeds using 90% of the sample. Unlike with linear models, this bootstrapping generates a resulting percentage as opposed to an adjusted significance value, confidence interval, and standard error. However, now I am stuck on how to integrate the VIP with the taxa that were and were not selected.

I have seen the VIP used where the selected features and their coefficients generated from the primary ENet are basically ignored and further analyses move forward with features who have a VIP > specified threshold, say 50 or 70% (fairly arbitrary). This method is straightforward and utilizes the feature selection aspect of ENet, primarily the LASSO-like component/penalization, but ignores the RIDGE-like component by throwing out the adjusted coefficients. It has also been shown that the VIP has a higher Type-I error rate for false positives, meaning that a feature not selected by the primary ENet model might erroneously have a high VIP. So, on top of not wanting to ignore the coefficients, I don't want to include taxa that weren't chosen by the CV ENet models.

So, I see two different ways forward. The first is to report the VIP values for transparency; these taxa were selected and there is a variable amount of confidence in each taxa. The second is to report the VIP for the selected taxa AND to only move forward with further analyses with the selected taxa who also have a VIP> a certain threshold.

I'm not new to microbiome research, but I am to these type of analyses; I welcome any suggestions, even about other methods of bootstrapping. Thanks!