Stat Bandit

Musings on statistics, computation and data research

A word of warning about grep, which and the like

I’ve often selected columns or rows of a data frame using grep or which, based on some property. That is inherently sound, but the trouble comes when you wish to remove rows or columns based on that grep or which call, e.g.,

dat <- dat[,-grep('\\.1', names(dat))]

which would remove columns with a .1 in the name. This is fine the first time around, but if you forget and re-run the code, grep('\\.1',names(dat)) gives a vector of length 0, and hence dat becomes a data.frame with 0 columns. The function which also has similar pitfalls, as demonstrated in a recent R-help posting by David Winsemius. I find a more reliable method is to do

dat <- dat[,setdiff(1:ncol(dat),grep('\\.1',names(dat)))]

which will always give the right number of columns. Other suggestions for getting around this issue are welcomed in the comments.

About these ads

3 responses to “A word of warning about grep, which and the like

  1. Dhruv M July 14, 2011 at 9:29 AM

    use grepl instead.

    e.g.
    colSlice <- function(dat,pattern,…){
    dat[,grepl(pattern,names(dat),...)]
    }

  2. Ben Bolker July 14, 2011 at 2:20 PM

    In general (beyond “grep” and “which”) you can avoid this sort of problem by creating new temporary variables rather than overwriting the existing variable. This is a minor pain because you end up with variables dat1, dat2, dat3, … representing the steps of the analysis, but may be worth it in the long run.

  3. Hans Róbert Hlynsson July 14, 2011 at 6:12 PM

    I always just use grepl like Dhruv recommends.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 391 other followers

%d bloggers like this: