Repeated measures ANOVA using Python

A common method in experimental psychology is within-subjects designs. One way to analysis the data collected using within-subjects designs are using repeated measures ANOVA. I recently wrote a post on how to conduct a repeated measures ANOVA using Python and rpy2. I wrote that post since the great Python package statsmodels do not include repeated measures ANOVA. However, the approach using rpy2 requires R statistical environment installed.  Recently, I found a python library called pyvttbl whith which you can do within-subjects ANOVAs.  Pyvttbl enables you to create multidimensional pivot tables, process data and carry out statistical tests. Using the method anova on pyvttbl’s DataFrame we can carry out repeated measures ANOVA using only Python.

Why within subject designs?

There are, at least, two of the advantages using within-subjects design. First, more information is obtained from each subject in a within-subjects design compared to a between-subjects design. Each subject is measured in all conditions, whereas in the between-subjects design, each subject is typically measured in one or more but not all conditions. A within-subject design thus requires fewer subjects to obtain a certain level of statistical power. In situations where it is costly to find subjects this kind of design is clearly better than a between-subjects design. Second, the variability in individual differences between subjects is removed from the error term. That is, each subject is his or her own control and extraneous error variance is reduced.

Repeated measures ANOVA in Python

Installing pyvttbl

pyvttbl can be installed using pip:

If you are using Linux you may need to add ‘sudo’ before the pip command. This method installs pyvttbl and, hopefully, any missing dependencies.

Python script

I continue with simulating a response time data set. If you have your own data set you want to do your analysis on you can use the method “read_tbl” to load your data from a CSV-file.

Conducting the repeated measures ANOVA with pyvttbl is pretty straight forward. You just take the pyvttbl DataFrame object and use the method anova. The first argument is your dependent variable (e.g. response time), and you specify the column in which the subject IDs are (e.g., sub=’Sub_id’). Finally, you add your within subject factor(s) (e.g., wfactors). wfactors take a list of column names containing your within subject factors. In my simulated data there is only one (e.g. ‘condition’).

Tests of Within-Subjects Effects

Measure: rt
Source   Type III Sum of Squares ε df MS F Sig. η2G Obs. SE of x̄ ±95% CI λ Obs. Power
condition Sphericity Assumed 4209536.428 1.000 4209536.428 309.093 0.000 4.165 40.000 19.042 37.323 317.019 1.000
Greenhouse-Geisser 4209536.428 1.000 1.000 4209536.428 309.093 0.000 4.165 40.000 19.042 37.323 317.019 1.000
Huynh-Feldt 4209536.428 1.000 1.000 4209536.428 309.093 0.000 4.165 40.000 19.042 37.323 317.019 1.000
Box 4209536.428 1.000 1.000 4209536.428 309.093 0.000 4.165 40.000 19.042 37.323 317.019 1.000
Error(condition) Sphericity Assumed 531140.646 39.000 13618.991                
Greenhouse-Geisser 531140.646 1.000 39.000 13618.991                
Huynh-Feldt 531140.646 1.000 39.000 13618.991                
Box 531140.646 1.000 39.000 13618.991                

As can be seen in the output table the Sum of Squares used is Type III which is what common statistical software use when calculating ANOVA (the F-statistic) (e.g., SPSS or R-packages such as ‘afex’ or ‘ez’). The table further contains correction in case our data violates the assumption of Sphericity (which in the case of only 2 factors, as in the simulated data, is nothing to worry about). As you can see we also get generalized eta squared as effect size measure and 95 % Confidence Intervals. It is stated in the docstring for the class Anova that standard Errors and 95% confidence intervals are calculated according to Loftus and Masson (1994). Furthermore, generalized eta squared allows comparability across between-subjects and within-subjects designs (see, Olejnik & Algina, 2003).

Conveniently, if you ever want to transform your data you can add the argument transform. There are several options here; log or log10, reciprocal or inverse, square-root or sqrt, arcsine or arcsin, and windsor10. For instance, if you want to use log-transformation you just add the argument “transform=’log'” (either of the previously mentioned methods can be used as arguments in string form):

Using pyvttbl we can also analyse mixed-design/split-plot (within-between) data. Doing a split-plot is easy; just add the argument “bfactors=” and a list of your between-subject factors. If you are interested in one-way ANOVA for independent measures see my newer post: Four ways to conduct one-way ANOVAS with Python.

Finally, I created a function that extracts the F-statistics, Mean Square Error, generalized eta squared, and the p-value the results obtained with the anova method.  It takes a factor as a string, a ANOVA object, and the values you want to extract. Keys for your different factors can be found using the key-method (e.g., aov.keys()).

Note, the table with the results in this post was created with the private method _within_html. To create an HTML table you will have to import SimpleHTML:

That was all. There are at least one downside with using pyvttbl for doing within-subjects analysis in Python (ANOVA). Pyvttbl is not compatible with Pandas DataFrame which is commonly used. However, this may not be a problem since pyvttbl, as we have seen, has its own DataFrame method. There are also a some ways to aggregate and visualizing data using Pyvttbl. Another downside is that it seems like Pyvttbl no longer is maintained.


Loftus, G.R., & Masson, M.E. (1994). Using confidence intervals in  within-subjects designs. The Psychonomic Bulletin & Review, 1(4),  476-490.
Olejnik, S., & Algina, J. (2003). Generalized eta and omega squared statistics: measures of effect size for some common research designs. Psychological Methods, 8(4), 434–47.

12 thoughts on “Repeated measures ANOVA using Python

  1. Dear Erik,

    thanks for this excellent blog post! I would like to point out that there is also relatively new, but actively maintained RM-ANOVA support in the mne-python package, see The data have to be transformed into a certain shape to use this function though; but this should be straightforward using pandas. It would be great if you could cover this approach in future post, as I think it might help many people making the final switch to Python.

    All the best,


    1. Hey Richard,

      Glad you liked my post.

      Thank you for mentioning mne-python package. It seems like I’ve missed it.
      I will have a look at it later today (travelling atm).

      Again, thank you for pointing it out.
      Have a nice day,


      1. Hi Erik,

        great you like my suggestion 🙂 I actually discovered mne’s RM-ANOVA through this github issue re statsmodels:

        And by the way, although the docstring of mne-python’s f_mway_rm() states it expects a 3D array as input, the code will handle ordinary 2D (“behavioral”) data just fine:

        It would be great to have a side-by-side comparison of the results from this RM-ANOVA implementation and possibly afex or JASP (which, to my knowledge, internally uses afex). Let me know if I can help you with this in any way!

        (NB: I’m by no means a stats guru, but simply a neuroscientist/psychologist who happens to be willing to analyze his data 😉 That’s why I’ve been very hesitant to move away from afex so far, because I have the impression that Henrik Singmann, the author of afex — who also happens to be a psychologist! — made very “sane” decision as to how the package should behave, e.g. it allows me to replicate existing SPSS analyses etc., so I can assume that “I’m doing it right” with great confidence.)

        Thanks again,


        1. Great suggestions. I have actually been thinking on comparing R-packages and Python packages. Maybe I’ll reach out to you.

          I am no stats guru either… However, I enjoy programming (may sound strange but I find it relaxing to write code) and I’ve found that I understand statistical concepts better when using R (or Python).

          I tend to mess things up in GUI interfaces such as SPSS (e.g., forgetting which boxes I checked in last time). Doing stats in R (or Python) make it easier to reproduce the same analysis. In my two first studies of my thesis I use R (i.e., afex and lsmeans). Afex is really great.

  2. Hi,

    Thanks for your post.

    I’m able to run your example without an issue, but when I try to run it with a second within subject factor, I get the error below;

    Exception: (‘data has %d conditions; design only %d’, 2, 4)

    I’ve pasted is the script that causes the error below. As you can see, the only change that I have made to your example is to add a second within subject factor (which is simply a duplicate of your single within subject factor), such that now the data contains condition1 and condition2 instead of only condition. Have you been able to run your example with multiple within subject factors?

    Here is the script;
    from numpy.random import normal
    import pyvttbl as pt
    from collections import namedtuple

    N = 40
    P = [‘c1’, ‘c2’]
    rts = [998, 511]
    mus = rts * N

    Sub = namedtuple(‘Sub’, [‘Sub_id’, ‘rt’, ‘condition1’, ‘condition2’])
    df = pt.DataFrame()
    for subid in xrange(0, N):
    for i, condition in enumerate(P):
    df.insert(Sub(subid + 1, normal(mus[i], scale=112., size=1)[0], condition, condition)._asdict())

    aov = df.anova(‘rt’, sub=’Sub_id’, wfactors=[‘condition1’, ‘condition2’])


    Thanks, Phil

  3. Hello Erik,

    thank you for your article!
    I am trying to use your guide for running a repeated measures analyisis for my experiment. I have the data in a CSV file which I imported using the method read_table. However, I am getting issues when running the anova method.
    Here my script:

    df = pt.DataFrame()
    df.read_tbl(‘GEQ_DATA_ANALYSIS_CORE_CSV_SUMMARY_p.csv’,delimiter =’;’)
    aov = df.anova(‘FLOW’, sub=’P_ID’, wfactors=[‘SETUP’])

    The error I get is:

    TypeError Traceback (most recent call last)
    in ()
    1 df = pt.DataFrame()
    2 df.read_tbl(‘GEQ_DATA_ANALYSIS_CORE_CSV_SUMMARY_p.csv’,delimiter =’;’)
    —-> 3 aov = df.anova(‘FLOW’, sub=’P_ID’, wfactors=[‘SETUP’])
    4 print(aov)

    C:\Program Files\Anaconda2\lib\site-packages\pyvttbl\base.pyc in anova(self, dv, sub, wfactors, bfactors, measure, transform, alpha)
    1973 aov=stats.Anova()
    1974, dv, sub=sub, wfactors=wfactors, bfactors=bfactors,
    -> 1975 measure=measure, transform=transform, alpha=alpha)
    1976 return aov

    C:\Program Files\Anaconda2\lib\site-packages\pyvttbl\stats\_anova.pyc in run(self, dataframe, dv, wfactors, bfactors, sub, measure, transform, alpha)
    709 if len(wfactors)!=0 and len(bfactors)==0:
    –> 710 self._within()
    712 if len(wfactors)==0 and len(bfactors)!=0:

    C:\Program Files\Anaconda2\lib\site-packages\pyvttbl\stats\_anova.pyc in _within(self)
    1131 for e in xrange(1,Ne+1):
    1132 # code effects so we can build contrasts
    -> 1133 cw = self._num2binvec(e,Nf)
    1134 efs = asarray(factors)[Nf-1-where(asarray(cw)==2.)[0][::-1]]
    1135 r=self[tuple(efs)] # unpack dictionary

    C:\Program Files\Anaconda2\lib\site-packages\pyvttbl\stats\_anova.pyc in _num2binvec(self, d, p)
    1238 d=floor(d/2.)
    -> 1240 return list(array(list(zeros((p-len(b))))+b)+1.)
    1242 ## def output2html(self, fname, script=”):

    TypeError: ‘float’ object cannot be interpreted as an index

    Which are the requirements for the CSV file ? How does it has to be formatted ?

    Thank you in advance!



Leave a Reply

Your email address will not be published. Required fields are marked *