/doc/source/comparison_with_r.rst
ReStructuredText | 479 lines | 334 code | 145 blank | 0 comment | 0 complexity | 67224ca59de884b90ae5c618a2320834 MD5 | raw file
Possible License(s): BSD-3-Clause, Apache-2.0
- .. currentmodule:: pandas
- .. _compare_with_r:
- .. ipython:: python
- :suppress:
- import pandas as pd
- import numpy as np
- options.display.max_rows=15
- Comparison with R / R libraries
- *******************************
- Since ``pandas`` aims to provide a lot of the data manipulation and analysis
- functionality that people use `R <http://www.r-project.org/>`__ for, this page
- was started to provide a more detailed look at the `R language
- <http://en.wikipedia.org/wiki/R_(programming_language)>`__ and its many third
- party libraries as they relate to ``pandas``. In comparisons with R and CRAN
- libraries, we care about the following things:
- - **Functionality / flexibility**: what can/cannot be done with each tool
- - **Performance**: how fast are operations. Hard numbers/benchmarks are
- preferable
- - **Ease-of-use**: Is one tool easier/harder to use (you may have to be
- the judge of this, given side-by-side code comparisons)
- This page is also here to offer a bit of a translation guide for users of these
- R packages.
- Base R
- ------
- Slicing with R's |c|_
- ~~~~~~~~~~~~~~~~~~~~~
- R makes it easy to access ``data.frame`` columns by name
- .. code-block:: r
- df <- data.frame(a=rnorm(5), b=rnorm(5), c=rnorm(5), d=rnorm(5), e=rnorm(5))
- df[, c("a", "c", "e")]
- or by integer location
- .. code-block:: r
- df <- data.frame(matrix(rnorm(1000), ncol=100))
- df[, c(1:10, 25:30, 40, 50:100)]
- Selecting multiple columns by name in ``pandas`` is straightforward
- .. ipython:: python
- df = DataFrame(np.random.randn(10, 3), columns=list('abc'))
- df[['a', 'c']]
- df.loc[:, ['a', 'c']]
- Selecting multiple noncontiguous columns by integer location can be achieved
- with a combination of the ``iloc`` indexer attribute and ``numpy.r_``.
- .. ipython:: python
- named = list('abcdefg')
- n = 30
- columns = named + np.arange(len(named), n).tolist()
- df = DataFrame(np.random.randn(n, n), columns=columns)
- df.iloc[:, np.r_[:10, 24:30]]
- |aggregate|_
- ~~~~~~~~~~~~
- In R you may want to split data into subsets and compute the mean for each.
- Using a data.frame called ``df`` and splitting it into groups ``by1`` and
- ``by2``:
- .. code-block:: r
- df <- data.frame(
- v1 = c(1,3,5,7,8,3,5,NA,4,5,7,9),
- v2 = c(11,33,55,77,88,33,55,NA,44,55,77,99),
- by1 = c("red", "blue", 1, 2, NA, "big", 1, 2, "red", 1, NA, 12),
- by2 = c("wet", "dry", 99, 95, NA, "damp", 95, 99, "red", 99, NA, NA))
- aggregate(x=df[, c("v1", "v2")], by=list(mydf2$by1, mydf2$by2), FUN = mean)
- The :meth:`~pandas.DataFrame.groupby` method is similar to base R ``aggregate``
- function.
- .. ipython:: python
- from pandas import DataFrame
- df = DataFrame({
- 'v1': [1,3,5,7,8,3,5,np.nan,4,5,7,9],
- 'v2': [11,33,55,77,88,33,55,np.nan,44,55,77,99],
- 'by1': ["red", "blue", 1, 2, np.nan, "big", 1, 2, "red", 1, np.nan, 12],
- 'by2': ["wet", "dry", 99, 95, np.nan, "damp", 95, 99, "red", 99, np.nan,
- np.nan]
- })
- g = df.groupby(['by1','by2'])
- g[['v1','v2']].mean()
- For more details and examples see :ref:`the groupby documentation
- <groupby.split>`.
- |match|_
- ~~~~~~~~~~~~
- A common way to select data in R is using ``%in%`` which is defined using the
- function ``match``. The operator ``%in%`` is used to return a logical vector
- indicating if there is a match or not:
- .. code-block:: r
- s <- 0:4
- s %in% c(2,4)
- The :meth:`~pandas.DataFrame.isin` method is similar to R ``%in%`` operator:
- .. ipython:: python
- s = pd.Series(np.arange(5),dtype=np.float32)
- s.isin([2, 4])
- The ``match`` function returns a vector of the positions of matches
- of its first argument in its second:
- .. code-block:: r
- s <- 0:4
- match(s, c(2,4))
- The :meth:`~pandas.core.groupby.GroupBy.apply` method can be used to replicate
- this:
- .. ipython:: python
- s = pd.Series(np.arange(5),dtype=np.float32)
- pd.Series(pd.match(s,[2,4],np.nan))
- For more details and examples see :ref:`the reshaping documentation
- <indexing.basics.indexing_isin>`.
- |tapply|_
- ~~~~~~~~~
- ``tapply`` is similar to ``aggregate``, but data can be in a ragged array,
- since the subclass sizes are possibly irregular. Using a data.frame called
- ``baseball``, and retrieving information based on the array ``team``:
- .. code-block:: r
- baseball <-
- data.frame(team = gl(5, 5,
- labels = paste("Team", LETTERS[1:5])),
- player = sample(letters, 25),
- batting.average = runif(25, .200, .400))
- tapply(baseball$batting.average, baseball.example$team,
- max)
- In ``pandas`` we may use :meth:`~pandas.pivot_table` method to handle this:
- .. ipython:: python
- import random
- import string
- baseball = DataFrame({
- 'team': ["team %d" % (x+1) for x in range(5)]*5,
- 'player': random.sample(list(string.ascii_lowercase),25),
- 'batting avg': np.random.uniform(.200, .400, 25)
- })
- baseball.pivot_table(values='batting avg', columns='team', aggfunc=np.max)
- For more details and examples see :ref:`the reshaping documentation
- <reshaping.pivot>`.
- |subset|_
- ~~~~~~~~~~
- .. versionadded:: 0.13
- The :meth:`~pandas.DataFrame.query` method is similar to the base R ``subset``
- function. In R you might want to get the rows of a ``data.frame`` where one
- column's values are less than another column's values:
- .. code-block:: r
- df <- data.frame(a=rnorm(10), b=rnorm(10))
- subset(df, a <= b)
- df[df$a <= df$b,] # note the comma
- In ``pandas``, there are a few ways to perform subsetting. You can use
- :meth:`~pandas.DataFrame.query` or pass an expression as if it were an
- index/slice as well as standard boolean indexing:
- .. ipython:: python
- df = DataFrame({'a': np.random.randn(10), 'b': np.random.randn(10)})
- df.query('a <= b')
- df[df.a <= df.b]
- df.loc[df.a <= df.b]
- For more details and examples see :ref:`the query documentation
- <indexing.query>`.
- |with|_
- ~~~~~~~~
- .. versionadded:: 0.13
- An expression using a data.frame called ``df`` in R with the columns ``a`` and
- ``b`` would be evaluated using ``with`` like so:
- .. code-block:: r
- df <- data.frame(a=rnorm(10), b=rnorm(10))
- with(df, a + b)
- df$a + df$b # same as the previous expression
- In ``pandas`` the equivalent expression, using the
- :meth:`~pandas.DataFrame.eval` method, would be:
- .. ipython:: python
- df = DataFrame({'a': np.random.randn(10), 'b': np.random.randn(10)})
- df.eval('a + b')
- df.a + df.b # same as the previous expression
- In certain cases :meth:`~pandas.DataFrame.eval` will be much faster than
- evaluation in pure Python. For more details and examples see :ref:`the eval
- documentation <enhancingperf.eval>`.
- zoo
- ---
- xts
- ---
- plyr
- ----
- ``plyr`` is an R library for the split-apply-combine strategy for data
- analysis. The functions revolve around three data structures in R, ``a``
- for ``arrays``, ``l`` for ``lists``, and ``d`` for ``data.frame``. The
- table below shows how these data structures could be mapped in Python.
- +------------+-------------------------------+
- | R | Python |
- +============+===============================+
- | array | list |
- +------------+-------------------------------+
- | lists | dictionary or list of objects |
- +------------+-------------------------------+
- | data.frame | dataframe |
- +------------+-------------------------------+
- |ddply|_
- ~~~~~~~~
- An expression using a data.frame called ``df`` in R where you want to
- summarize ``x`` by ``month``:
- .. code-block:: r
- require(plyr)
- df <- data.frame(
- x = runif(120, 1, 168),
- y = runif(120, 7, 334),
- z = runif(120, 1.7, 20.7),
- month = rep(c(5,6,7,8),30),
- week = sample(1:4, 120, TRUE)
- )
- ddply(df, .(month, week), summarize,
- mean = round(mean(x), 2),
- sd = round(sd(x), 2))
- In ``pandas`` the equivalent expression, using the
- :meth:`~pandas.DataFrame.groupby` method, would be:
- .. ipython:: python
- df = DataFrame({
- 'x': np.random.uniform(1., 168., 120),
- 'y': np.random.uniform(7., 334., 120),
- 'z': np.random.uniform(1.7, 20.7, 120),
- 'month': [5,6,7,8]*30,
- 'week': np.random.randint(1,4, 120)
- })
- grouped = df.groupby(['month','week'])
- print grouped['x'].agg([np.mean, np.std])
- For more details and examples see :ref:`the groupby documentation
- <groupby.aggregate>`.
- reshape / reshape2
- ------------------
- |meltarray|_
- ~~~~~~~~~~~~~
- An expression using a 3 dimensional array called ``a`` in R where you want to
- melt it into a data.frame:
- .. code-block:: r
- a <- array(c(1:23, NA), c(2,3,4))
- data.frame(melt(a))
- In Python, since ``a`` is a list, you can simply use list comprehension.
- .. ipython:: python
- a = np.array(list(range(1,24))+[np.NAN]).reshape(2,3,4)
- DataFrame([tuple(list(x)+[val]) for x, val in np.ndenumerate(a)])
- |meltlist|_
- ~~~~~~~~~~~~
- An expression using a list called ``a`` in R where you want to melt it
- into a data.frame:
- .. code-block:: r
- a <- as.list(c(1:4, NA))
- data.frame(melt(a))
- In Python, this list would be a list of tuples, so
- :meth:`~pandas.DataFrame` method would convert it to a dataframe as required.
- .. ipython:: python
- a = list(enumerate(list(range(1,5))+[np.NAN]))
- DataFrame(a)
- For more details and examples see :ref:`the Into to Data Structures
- documentation <basics.dataframe.from_items>`.
- |meltdf|_
- ~~~~~~~~~~~~~~~~
- An expression using a data.frame called ``cheese`` in R where you want to
- reshape the data.frame:
- .. code-block:: r
- cheese <- data.frame(
- first = c('John', 'Mary'),
- last = c('Doe', 'Bo'),
- height = c(5.5, 6.0),
- weight = c(130, 150)
- )
- melt(cheese, id=c("first", "last"))
- In Python, the :meth:`~pandas.melt` method is the R equivalent:
- .. ipython:: python
- cheese = DataFrame({'first' : ['John', 'Mary'],
- 'last' : ['Doe', 'Bo'],
- 'height' : [5.5, 6.0],
- 'weight' : [130, 150]})
- pd.melt(cheese, id_vars=['first', 'last'])
- cheese.set_index(['first', 'last']).stack() # alternative way
- For more details and examples see :ref:`the reshaping documentation
- <reshaping.melt>`.
- |cast|_
- ~~~~~~~
- In R ``acast`` is an expression using a data.frame called ``df`` in R to cast
- into a higher dimensional array:
- .. code-block:: r
- df <- data.frame(
- x = runif(12, 1, 168),
- y = runif(12, 7, 334),
- z = runif(12, 1.7, 20.7),
- month = rep(c(5,6,7),4),
- week = rep(c(1,2), 6)
- )
- mdf <- melt(df, id=c("month", "week"))
- acast(mdf, week ~ month ~ variable, mean)
- In Python the best way is to make use of :meth:`~pandas.pivot_table`:
- .. ipython:: python
- df = DataFrame({
- 'x': np.random.uniform(1., 168., 12),
- 'y': np.random.uniform(7., 334., 12),
- 'z': np.random.uniform(1.7, 20.7, 12),
- 'month': [5,6,7]*4,
- 'week': [1,2]*6
- })
- mdf = pd.melt(df, id_vars=['month', 'week'])
- pd.pivot_table(mdf, values='value', index=['variable','week'],
- columns=['month'], aggfunc=np.mean)
- Similarly for ``dcast`` which uses a data.frame called ``df`` in R to
- aggregate information based on ``Animal`` and ``FeedType``:
- .. code-block:: r
- df <- data.frame(
- Animal = c('Animal1', 'Animal2', 'Animal3', 'Animal2', 'Animal1',
- 'Animal2', 'Animal3'),
- FeedType = c('A', 'B', 'A', 'A', 'B', 'B', 'A'),
- Amount = c(10, 7, 4, 2, 5, 6, 2)
- )
- dcast(df, Animal ~ FeedType, sum, fill=NaN)
- # Alternative method using base R
- with(df, tapply(Amount, list(Animal, FeedType), sum))
- Python can approach this in two different ways. Firstly, similar to above
- using :meth:`~pandas.pivot_table`:
- .. ipython:: python
- df = DataFrame({
- 'Animal': ['Animal1', 'Animal2', 'Animal3', 'Animal2', 'Animal1',
- 'Animal2', 'Animal3'],
- 'FeedType': ['A', 'B', 'A', 'A', 'B', 'B', 'A'],
- 'Amount': [10, 7, 4, 2, 5, 6, 2],
- })
- df.pivot_table(values='Amount', index='Animal', columns='FeedType', aggfunc='sum')
- The second approach is to use the :meth:`~pandas.DataFrame.groupby` method:
- .. ipython:: python
- df.groupby(['Animal','FeedType'])['Amount'].sum()
- For more details and examples see :ref:`the reshaping documentation
- <reshaping.pivot>` or :ref:`the groupby documentation<groupby.split>`.
- .. |c| replace:: ``c``
- .. _c: http://stat.ethz.ch/R-manual/R-patched/library/base/html/c.html
- .. |aggregate| replace:: ``aggregate``
- .. _aggregate: http://finzi.psych.upenn.edu/R/library/stats/html/aggregate.html
- .. |match| replace:: ``match`` / ``%in%``
- .. _match: http://finzi.psych.upenn.edu/R/library/base/html/match.html
- .. |tapply| replace:: ``tapply``
- .. _tapply: http://finzi.psych.upenn.edu/R/library/base/html/tapply.html
- .. |with| replace:: ``with``
- .. _with: http://finzi.psych.upenn.edu/R/library/base/html/with.html
- .. |subset| replace:: ``subset``
- .. _subset: http://finzi.psych.upenn.edu/R/library/base/html/subset.html
- .. |ddply| replace:: ``ddply``
- .. _ddply: http://www.inside-r.org/packages/cran/plyr/docs/ddply
- .. |meltarray| replace:: ``melt.array``
- .. _meltarray: http://www.inside-r.org/packages/cran/reshape2/docs/melt.array
- .. |meltlist| replace:: ``melt.list``
- .. meltlist: http://www.inside-r.org/packages/cran/reshape2/docs/melt.list
- .. |meltdf| replace:: ``melt.data.frame``
- .. meltdf: http://www.inside-r.org/packages/cran/reshape2/docs/melt.data.frame
- .. |cast| replace:: ``cast``
- .. cast: http://www.inside-r.org/packages/cran/reshape2/docs/cast