This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Removing columns from multiple files in R

I have written a code for removing columns from multiple text files.

file_list <- list.files(pattern = ".txt")
 dfList <- lapply(file_list, function(f) {
     df <- read.table(f, header=TRUE, sep="\t", stringsAsFactors=FALSE)
     df <- df[grep("(adj.P.Val|P.Value|t|B)", names(df), invert = TRUE)]
 })

 finaldf <- do.call(rbind, dfList)

But i need multiple files as it is. rbind merges all the file. How to remove only columns from all the files keeping as it is? Thanks in advance

r microarray
• 3,964 views • link
updated by GenoMax GenoMax • written by smrutimayipanda • smrutimayipanda

After the lapply command you should end up with a list of data.frames that each only have the selected columns. What did you want to do with this list exactly?

• 0 views • link
written by rpolicastro rpolicastro

I want to remove columns from the files and keep it as it is.i want them separate only

• 0 views • link
written by smrutimayipanda • smrutimayipanda

So do you want to save them as new files then?

• 0 views • link
written by rpolicastro rpolicastro

2 answers

I use the walk function here from the tidyverse library purrr because we don't care about returning a list from the loop.

library("purrr")
library("dplyr")

file_list <- list.files(pattern = "\\.txt$")

walk(file_list, function(f) {
  file_name <- paste0("filtered_", basename(f))
  read.table(f, header=TRUE, sep="\t", stringsAsFactors=FALSE) %>%
    select(-adj.P.Val, -P.Value, -t, -B) %>%
    write.table(file_name, sep="\t", col.names=TRUE, row.names=FALSE, quote=FALSE)
})
• 0 views • link
written by rpolicastro rpolicastro

but it is printing same columns and it is not printing whole file. what to do ?

ID  adj.P.Val   P.Value t   B   logFC   Gene.symbol
8156043 7.92e-08    4.54e-12    -35.8   17.12755    -1.87   PSAT1
8114249 7.92e-08    1.63e-11    -31.5   16.233  -2  CXCL14
7912975 9.5e-08 2.45e-11    -30.3   15.93407    -1.33   ALDH4A1
• 0 views • link
updated by zx8754 zx8754 • written by smrutimayipanda • smrutimayipanda

I edited my answer so that it should work based on your regex. If it doesn't work, can you post the names of the columns that you want to remove here?

• 0 views • link
written by rpolicastro rpolicastro

"adj.P.Val" "P.Value" "t" "B"

Thanks its working. The previous one is also working

• 0 views • link
written by smrutimayipanda • smrutimayipanda

Can you edit this to only include the column names you want to remove?

• 0 views • link
written by rpolicastro rpolicastro

I edited my answer, check to see if it works.

• 0 views • link
written by rpolicastro rpolicastro

Hello

I am trying to use this code to delete specific columns in multiple csv files. I changed the .txt to .csv and also I changed my desired column names that I need to be omitted but it says that Error: Can't subset columns that don't exist. x Column Event doesn't exist. I checked with different column's names but I receive the same error each time. what should I do?

Thank you

• 0 views • link
written by paramount.amin • paramount.amin

You have solved the problem but you are binding the dataframes in the last call by producing finaldf. Therefore, you will get all data frames together. What you need to do is to write every data frame in the list to seperated files for example or work with them separately. So you will need something like dfList[[1]] to get the reduced contents of the first dataframe/file:

print(dfList[[1]])
• 0 views • link
written by Ali T. A. • Ali T. A.

Log in to answer this question.

More posts like this