Querying Databases in R

earl — Fri, 14 Aug 2009 16:00:36 +0000

One of the first things you’ll want to do in R is set it up to talk to databases. The easiest way to do this is using ODBC, via package RODBC.

To get the package, run

> install.packages(RODBC)

Once you have RODBC installed, you call it in R as follows. But it’s very simple: a bit of setup, then sqlQuery will run your sql and return the results in a data frame.

library(RODBC)

db <- odbcConnect( dsn='your dsn name' )
sql <- 'select page_id, count(*) as cnt
           from document_ads
           group by page_id
           having count(*) > 1'

results <- sqlQuery(db, sql, errors=T, rows_at_time=1024)
str(results)
'data.frame':	282432 obs. of  2 variables:
 $ page_id: int  17646774 17115332 17606022 15899428 17099174 17283774 8604200 16315025 17259751 17283270 ...
 $ cnt            : int  489 1119 132 113 148 200 112 121 1135 633 ...

On Windows, you setup the DSNs in the ODBC Data Sources inside the control panel; on MacOS, mysql includes a program called ODBC Administrator; on linux, you'll have to install unixODBC .

Also, it's often convenient to write code that caches your query results, particularly if the query takes a while. I've found that the easiest thing to do is write the results into a data file and check for the file existence like such:

filename <- 'query cache.RData'
if (!file.exists(filename)){
   # don't have a cached copy so run the query
   library(RODBC)
   [snip]
   query1 <- sqlQuery(db, sql, errors=T, rows_at_time=1024)

   # save the query results for the future
   save(list=c('query1', 'sql'), file=filename)
   rm(list=c('query1', 'sql') )
}
load(file=filename)

Examining Data Frames — head and tail

earl — Sun, 02 Aug 2009 07:30:39 +0000

head and tail, for those familiar with the unix command line, are two very handy utilities for looking at data frames. Along with str, which displays the structure of a data frame, they help you look at your data:

> d <- data.frame(mean=rep(1:10,5), val = rnorm(n=50, mean=rep(1:10,5)))
> d <- d[ order(d$mean), ]
>
> str(d)
'data.frame':	50 obs. of  2 variables:
 $ mean: int  1 1 1 1 1 2 2 2 2 2 ...
 $ val : num  2.303 2.222 -1.153 1.795 -0.232 ...
>
> head(d)
   mean        val
1     1  2.3026422
11    1  2.2216371
21    1 -1.1533163
31    1  1.7945563
41    1 -0.2318763
2     2 -0.4994239
>
> tail(d)
   mean       val
49    9  8.462525
10   10 10.437314
20   10 10.815264
30   10 10.218853
40   10  9.754245
50   10  9.596825
>

If you are familiar with data frames, you’ll know that head(d) is no different than displaying the first 6 rows via subsetting, eg, d[1:6, ] but it saves some typing. tail, on the other hand, saves us from either asking first how many rows a data frame has with nrow, or typing a mess: d[ (nrow(d)-5):nrow(d), ]

> nrow(d)
[1] 50
> d[45:50,]
   mean       val
49    9  8.462525
10   10 10.437314
20   10 10.815264
30   10 10.218853
40   10  9.754245
50   10  9.596825
> d[ (nrow(d)-5):nrow(d), ]
   mean       val
49    9  8.462525
10   10 10.437314
20   10 10.815264
30   10 10.218853
40   10  9.754245
50   10  9.596825
>

Stochastic Nonsense » data frame

Querying Databases in R

Examining Data Frames — head and tail