A diurnal cycle is anything that occurs daily. In atmospheric chemistry, a diurnal profile displays the trends in a given gas concentration over a period of time at each time throughout the day (makes perfect sense when you see the plot). They are great for visualizing how a certain gas changes in the atmosphere throughout the day (think ozone near a huge interstate interchange).

Plotting them is pretty easy with python and pandas! Here is how you do it:

Read in your data

The pandas library comes with several functions to easily import data from csv, excel, or txt files. There are also several others, including reading from json which will come in handy when I finally get around to working with the EPA API's. I will show how to read in some data from an excel file, since that is currently what I have to work with.

  1. Import some things

    import pandas as pd
    import matplotlib
    from matplotlib import dates as d
    import datetime as dt
    import matplotlib.pyplot as plt
    import numpy as np

  2. Define some variables that we may later want to change

    filename = 'path to your file'
    sheetname = 'sheet where your data is located'
    skiprows = 'number of rows to skip at top of the file'
    index_col = 'column number where your datetimes are'

  3. Read in your data

    data = pd.read_excel(filename, sheetname, skiprows = skiprows, index_col = index_col)

Munge your data

To plot a diurnal profile, we are going to need to group our data in order to perform some statistical analysis of the data. It is important to define a resolution you are satisfied with when you perform the next step; I am fine with minute resolution as seen below.

  1. Create a new column that contains timestamps in the form H:m

    data['Time'] = data.index.map(lambda x: x.strftime("%H:%M"))

  2. Group the data by its new Time column and perform some statistics

    data = data.groupby('Time').describe().unstack()

    At this point, we have a grouped dataframe with columns containing all the statistics you could ever want (and all we need for our analysis).

  3. Set our new index to be of the datetime format for easy plotting

    data.index = pd.to_datetime(data.index.astype(str))

Plot your data

Now that our data is properly munged, we can go ahead and plot (fun!). The first plot we will create is a simple diurnal trend showing the mean concentration of the gas (or particle!) throughout the day. The second plot will build on the first one to include the inner and outer quartiles with shading in between (pretty!).

Simple Mean

  1. Set up your plot

    fig, ax = plt.subplots(1, figsize=(12,6))
    ax.set_title('Diurnal Profile', fontsize=14)
    ax.set_ylabel('Gas Concentrations', fontsize=14, weight='bold')
    ax.set_xlabel('Time of Day', fontsize=14)
  2. Plot

    ax.plot(data.index, data[column_name]['mean'], 'g', linewidth=2.0)
  3. Make the axes pretty and show!

    ticks = ax.get_xticks()
    ax.set_xticks(np.linspace(ticks[0], d.date2num(d.num2date(ticks[-1]) + dt.timedelta(hours=3)), 5))
    ax.set_xticks(np.linspace(ticks[0], d.date2num(d.num2date(ticks[-1]) + dt.timedelta(hours=3)), 25), minor=True)
    ax.xaxis.set_major_formatter(matplotlib.dates.DateFormatter('%I:%M %p'))

You should end up with something like this:

Mean, quartiles, and shading

To add the inner and outer quartiles, there are only a few more lines of code (woohoo!)

  1. Add inner and outer quartile lines

    ax.plot(data.index, data[column_name]['75%'], color='g')
    ax.plot(data.index, data[column_name]['25%'], color='g')
  2. Add shading in between quartiles and the mean

    ax.fill_between(data.index, data[column_name]['mean'], data[column_name]['75%'], alpha=.5, facecolor='g')
    ax.fill_between(data.index, data[column_name]['mean'], data[column_name]['25%'], alpha=.5, facecolor='g')

After adding these lines, you should end up with an even more beautiful plot as seen below:


Python is awesome. Pandas is super-awesome. Science is the greatest! I will upload this code as a github gist as soon as I get around to it. Also, I plan on adding this to the (someday) awesome atmospy python library for the atmospheric sciences.