Last updated on January 13, 2020
In this post, we will learn how to use Pandas get_dummies() method to create dummy variables in Python. Dummy variables (or binary/indicator variables) are often used in statistical analyses as well as in more simple descriptive statistics. Towards the end of the post, there’s a link to a Jupyter Notebook containing all Pandas get_dummies() examples.
Dummy Coding for Regression Analysis
One statistical analysis in which we may need to create dummy variables in regression analysis. In fact, regression analysis requires numerical variables and this means that when we, whether doing research or just analyzing data, wishes to include a categorical variable in a regression model, supplementary steps are required to make the results interpretable.
In these steps, categorical variables in the data set are recoded into a set of separate binary variables (dummy variables). Furthermore, this re-coding is called “dummy coding” and involves the creation of a table called contrast matrix. Dummy coding can be done automatically by statistical software, such as R, SPSS, or Python.
What is Categorical Data?
In this section, of the creating dummy variables in Python guide, we are going to answer the question about what categorical data is. Now, in statistics, a categorical variable (also known as factor or qualitative variable) is a variable that takes on one of a limited, and most commonly a fixed number of possible values. Furthermore, these variables are typically assigning each individual, or another unit of observation, to a particular group or nominal category. For example, gender is a categorical variable.
What is a Dummy Variable?
Now, the next question we are going to answer before working with Pandas get_dummies, is “what is a dummy variable?”. Typically, a dummy variable (or column) is one which has a value of one (1) when a categorical event occurs (e.g., an individual is male) and zero (0) when it doesn’t occur (e.g., an individual is female).
Obviously, we need to have Pandas installed to use the get_dummies() method. Pandas can be installed using pip or conda, for instance. If we want to install Pandas using condas we type
conda install pandas. On the other hand, if we want to use pip, we type
pip install pandas. Note, it is typically suggested that Python packages are installed in virtual environments. Pipx can be used to install Python packages directly in virtual environments and if we want to install, update, and use Python packages we can, as in this post, use conda or pip.
Example Data to Dummy Code
In this Pandas dummies tutorial, we will use the Salaries dataset, which contains the 2008-09 nine-month academic salary for Assistant Professors, Associate Professors, and Professors in a college in the U.S.
Import Data in Python using Pandas
Now, before we start using Pandas get_dummies() method, we need to load pandas and import the data.
import pandas as pd data_url = 'http://vincentarelbundock.github.io/Rdatasets/csv/carData/Salaries.csv' df = pd.read_csv(data_url, index_col=0) df.head()
Of course, data can be stored in multiple different file types. For instance, we could have our data stored in .xlsx, SPSS, SAS, or STATA files. See the following tutorials to learn more about importing data from different file types:
- Learn how to read Excel (.xlsx) files using Python and Pandas
- Read SPSS files using Pandas in Python
- Import (Read) SAS files using Pandas
- Read STATA files in Python with Pandas
Creating Dummy Variables in Python
In this section, we are going to use pandas get_dummies() to generate dummy variables in Python. First, we are going to work with the categorical variable “sex”. That is, we will start with dummy coding in Python with a categorical variable with two levels.
Second, we are going to generate dummy variables in Python with the variable “rank”. That is, in that dummy coding in Python example we are going to work with a factor variable with three levels.
How to Make Dummy Variables in Python with Two Levels
In this section, we are going to create a dummy variable in Python using Pandas get_dummies method. Specifically, we will generate dummy variables for a categorical variable with two levels (i.e., male and female).
In this create dummy variables in Python post, we are going to work with Pandas get_dummies(). As can be seen, in the image above we can change the prefix of our dummy variables, and specify which columns that contain our categorical variables.
First Dummy Coding in Python Example:
In the first Python dummy coding example below, we are using Pandas get_dummies to make dummy variables. Note, we are using a series as data and, thus, get two new columns named Female and Male.
In the code, above, we also printed the first 5 rows (using Pandas head()). We will now continue and use the columns argument. Here we input a list with the column(s) we want to create dummy variables from. Furthermore, we will create the new Pandas dataframe containing our new two columns.
More Python Dummy Coding Examples:
df_dummies = pd.get_dummies(df, columns=['sex']) df_dummies.head()
In the output (using Pandas head()), we can see that Pandas get_dummies automatically added “sex” as prefix and underscore as prefix separator. If we, however, want to change the prefix as well as the prefix separator we can add these arguments to Pandas get_dummies():
df_dummies = pd.get_dummies(df, prefix='Gender', prefix_sep='.', columns=['sex']) df_dummies.head()
Remove Prefix and Separator from Dummy Columns
In the next Pandas dummies example code, we are going to make dummy variables in Python but we will set the prefix and the prefix_sep arguments so that we the column name will be the factor levels (categories):
df_dummies = pd.get_dummies(df, prefix=, prefix_sep='', columns=['sex']) df_dummies.head()
How to Create Dummy Variables in Python with Three Levels
In this section, of the dummy coding in Python tutorial, we are going to work with the variable “rank”. That is, we will create dummy variables in Python from a categorical variable with three levels (or 3 factor levels). In the first dummy variable in Python code example below, we are working with Pandas get_dummies() the same way as we did in the first example.
That is, we put in a Pandas Series (i.e., the column with the variable) as the only argument and then we only got a new dataframe with 3 columns (i.e., for the 3 levels).
Create a Dataframe with Dummy Coded Variables
Of course, we want to have the dummy variables in a dataframe with the data. Again, we do this by using the columns argument and a list with the column that we want to use:
df_dummies = pd.get_dummies(df, columns=['rank']) df_dummies.head()
In the image above, we can see that Pandas get_dummies() added “rank” as prefix and underscore as prefix separator. Next, we are going to change the prefix and the separator to “Rank” (uppercase) and “.” (dot).
df_dummies = pd.get_dummies(df, prefix='Rank', prefix_sep='.', columns=['rank']) df_dummies.head()code>
Now, we may not need to have a prefix or a separator and, as in the previous Pandas create dummy variables in Python example, want to remove these. To accomplish this, we just add empty strings to the prefix and prefix_sep arguments:
df_dummies = pd.get_dummies(df, prefix='', prefix_sep='', columns=['rank'])code>
Creating Dummy Variables in Python for Many Columns
In the final Pandas dummies example, we are going to dummy code two columns. Specifically, we are going to add a list with two categorical variables and get 5 new columns that are dummy coded. This is, in fact, very easy and we can follow the example code from above:
Creating Multiple Dummy Variables Example Code:
df_dummies = pd.get_dummies(df, prefix='', prefix_sep='', columns=['rank', 'sex']) df_dummies.head()
Finally, if we want to add more columns, to create dummy variables from, we can add that to the list we add as a parameter to the columns argument. See this notebook for all code examples in this tutorial about creating dummy variables in Python.
Conclusion: Dummy Coding in Python
In this post, we have learned how to do dummy coding in Python using Pandas get_dummies() method. More specifically, we have worked with categorical data with two levels, and categorical data with three levels. Furthermore, we have learned how to add and remove prefixes from the new columns created in the dataframe.