Create a financial dashboard with Flask

This is the second post of a three-part series on how to create a financial dashboard with Flask. If you missed the first post, you can check it out here. In the last post, we created a skeleton financial dashboard with fake market data. We could see some potential, but without real data, the dashboard is not very useful. Time to pump in some actual data.

Adding real market data

There are several market data providers which give limited access for free. For example:

However, we are not going to use any of these. Today, we will be using Yahoo Finance in our dashboard to show us useful market information, which is completely free. It's known Yahoo Finance has some issues, but for a simple personal dashboard, it will suffice. However, you should probably consider alternatives if you are trading with real money as data issues can be very costly.

We will be using the yfinance package which implements functions to download data from Yahoo Finance. We can leverage the package to get the data for us and we will only need to pipe the data into our existing functions. We can't use the package if it is not installed, so the first step is to add it to our virtual environment with poetry.

poetry add yfinance

Add package to our poetry virtual environment

Let's get some data. This new function will get data from Yahoo Finance and later it will substitute our fake_data function in our dashboard.

import yfinance as yf

def get_price_data(symbols, start_date = None, end_date = None):

    if end_date is None:
        end_date = datetime.date.today()

    if start_date is None:
        start_date = end_date

    start_date = parse_date(start_date)
    end_date = parse_date(end_date)

    start_date += datetime.timedelta(days = 1) # YFinance API auto subtracts 1 day
    end_date += datetime.timedelta(days = 1)   # we want end_date to be inclusive

    data = yf.download(
        symbols,
        start = start_date,
        end = end_date,
        actions = True,
        group_by = 'ticker'
    )

    # - Format data

    data = (
        data
        .melt(var_name = ['symbol', 'field'], ignore_index = False)
        .reset_index()
        .pivot(['Date', 'symbol'], 'field', 'value')
        .reset_index()
        .pipe(
            lambda df: df.rename(
                columns = {col : col.lower().replace(" ", "_") for col in df.columns}
            )
        )
        .sort_values('date', ascending = False)
    )

    return data

Get data from Yahoo Finance --- data.py

This function should be written in our data.py file. There's nothing too special about it, just a few curveballs the yfinance API throw at us which I mentioned in the comments. The variable symbols are the Yahoo symbols we will get the data for.

Because we are manipulating dates, it is much easier if we work with datetime objects instead of strings. However, sometimes is more convenient to pass dates as strings if we are quickly working in the console. To do this effectively, the code above uses a parse_date function which translates strings into dates. We have not written this function yet, let's do it now.

Parsing dates

Parsing dates is a simple task with the datetime module. If you have never done it, you can check the documentation here but probably it is very similar to other programming languages you may have worked with. So, why do we need to write a specific function to parse the dates?

Well, we don't, but we can it make a bit easier to work with. One thing that annoys me when dealing with dates and strings is to remember the date format. I will adopt the convention the date will be provided in the YEAR, MONTH, DAY format. However, I see a lot of date strings which are similar but still require different date format. For example:

2021-04-01
2021/04/01
2021_04_01

All of these have years followed by months, followed by days. It would be nice if we could write a function that parse all of these cases. And that's exactly what our parse_date function will do. Its purpose is simply to give more flexibility when working with dates.

import re

def parse_date(date):

    if isinstance(date, datetime.datetime):
        return date.date()

    if isinstance(date, datetime.date):
        return date

    if isinstance(date, str):
        date = re.sub(r'[-_/]', '', date)
        date = datetime.datetime.strptime(date, "%Y%m%d")
        return date.date()

    raise TypeError(f"Cannot parse_date for type {type(date).__name__}")

parse_date --- data.py

If the object date is already a date object, we return it. If it is a datetime object, we strip the time from it. Else, we use a regular expression to remove dashes, underscores or slashes.

Now we can parse all of these date combinations.

assert parse_date(20210401)   == datetime.date(2021,4,1)
assert parse_date(2021-04-01) == datetime.date(2021,4,1)
assert parse_date(2021_04_01) == datetime.date(2021,4,1)
assert parse_date(2021/04/01) == datetime.date(2021,4,1)
assert parse_date(2021_04/01) == datetime.date(2021,4,1)
assert parse_date(2021/04-01) == datetime.date(2021,4,1)

parsing dates

Calculating returns

Let's create a helper function to calculate N-day returns. This will come in handy when plotting our data.

def calc_return(price_data, index = 1):

    if index < 1:
        raise ValueError("index must be greater than zero")

    def ret(pr, index):

        last  = pr['date'].iloc[0]
        first = pr['date'].iloc[min(index, len(pr))]
        df = pr.loc[pr['date'].isin([last, first])]

        return (
            df.loc[:, ['date', 'symbol', 'adj_close']]
            .pivot('symbol', 'date', 'adj_close')
            .assign(
                ret  = lambda df: df[last] / df[first] - 1,
                date = last
            )
        )

    df = pd.concat(
        [ret(tbl, index) for _, tbl in price_data.groupby('symbol')],
        sort = False
    )

    return df[['date', 'ret']]

calculating returns

Calculating returns is straightforward, but in this case, we are taking extra care to make sure we are computing it with the correct dates. This is especially important when dealing with assets from different regions and time zones. For example, Japanese and US equities. The data provider may have already obtained the close prices for Japan while the US is still trading. In this case, we can have a date mismatch in our calculation.

We solve this by creating an inner function (in the case above ret) and applying the calculation for each symbol. As a side bonus, this also takes care of holidays in case we are running for a date in which it is a holiday in one region but not in the other.

Data.py

Now our data.py file should be complete. If you have been following along, it should like this. Next, we will update our front-end.

Updating our dashboard

Time to pump the market data to our dashboard. First, we need to define which symbols we will display in our dashboard. I will use 5 ETFs:

Next, we need to define how many days we will display the data. In my case, I will display the last 90 days of data. To simplify, I am defining the data extraction in the home function in the index.py file.


# All the previous imports go here...

from .data import get_price_data

@app.route("/")
def home():

    symbols = ['SPY', 'EZU', 'IWM', 'EWJ', 'EEM']
    end = datetime.date.today()
    start = end - datetime.timedelta(days = 30 * 3)

    df = get_price_data(symbols, start_date = start, end_date = end)

    # More continues below
    # ...

data definition --- index.py

However, 90 days of ETF prices would be too much data to show in our table. Let's summarise the results in our table to display only some useful metrics about our data.

Updating our table

Instead of displaying all the ETF prices, let's display only the:

most recent price
most recent return
the monthly return

This will give us a brief and yet meaningful summary of our ETFs. If we wish to analyse the performance over the entire 90 days, we can do so by analysing the chart. To do so, we will use our helper function calc_return which we created above.


# All the previous imports go here...

from .data import get_price_data
from .data import calc_return

@app.route("/")
def home():

    symbols = ['SPY', 'EZU', 'IWM', 'EWJ', 'EEM']
    end = datetime.date.today()
    start = end - datetime.timedelta(days = 30 * 3)

    df = get_price_data(symbols, start_date = start, end_date = end)

    ret_d01 = calc_return(df, index = 1)
    ret_d21 = calc_return(df, index = 21)
    prices  = (
        df[['symbol', 'date', 'adj_close']]
        .rename(
            columns = {'adj_close' : 'price'}
        )
        .merge(ret_d01.reset_index(), how = 'inner', on = ['date', 'symbol'])
        .rename(columns = {'ret' : 'daily_return'})
        .merge(ret_d21.reset_index(), how = 'inner', on = ['date', 'symbol'])
        .rename(columns = {'ret' : 'monthly_return'})
    )

    # More continues below
    # ...

summarising price data --- index.py

Above I am using the convention that a month has 21 business days, but you can change it to any day count convention you prefer. Because we had already done the heavy lifting with calc_return, we just need to use our helper function and do some data manipulation to make it look nice.

Alright, things are looking good but our table is a bit dull. Let's style it a bit. Pandas provides some incredible styling options to customize our HTML table. This post is not about pandas, so I will not go in-depth about how to customize the HTML table. I encourage you to check out the documentation and all the amazing things you could do with it.

I will style the table as follows:

positive returns show in green
negative returns show in red
add a per cent symbol to returns
style numbers with 2 decimal places

Our final version of the table looks like this.


# All the previous imports go here...

from .data import get_price_data
from .data import calc_return

@app.route("/")
def home():

    symbols = ['SPY', 'EZU', 'IWM', 'EWJ', 'EEM']
    end = datetime.date.today()
    start = end - datetime.timedelta(days = 30 * 3)

    df = get_price_data(symbols, start_date = start, end_date = end)
    _plot = (
        df
        .loc[:, ['date', 'symbol', 'adj_close']]
        .pivot('date', 'symbol', 'adj_close')
        .pipe(plot)
    )
    _plot = customize_chart(_plot)

    try:
        chart = export_svg(_plot)
    finally:
        plt.close()

    ret_d01 = calc_return(df, index = 1)
    ret_d21 = calc_return(df, index = 21)
    prices  = (
        df[['symbol', 'date', 'adj_close']]
        .rename(
            columns = {'adj_close' : 'price'}
        )
        .merge(ret_d01.reset_index(), how = 'inner', on = ['date', 'symbol'])
        .rename(columns = {'ret' : 'daily_return'})
        .merge(ret_d21.reset_index(), how = 'inner', on = ['date', 'symbol'])
        .rename(columns = {'ret' : 'monthly_return'})
    )

    # - styling

    return_cols = ['daily_return', 'monthly_return']
    def ret_color(x):
        color = 'tomato' if x < 0 else 'lightgreen'
        return 'color: %s' % color

    prices = (
        prices
        .sort_values('monthly_return', ascending = False)
        .assign(date = lambda df: df['date'].dt.strftime("%Y-%m-%d"))
        .pipe(format_data_frame)
        .format("{:,.2f}", subset = ['price'])
        .applymap(ret_color, subset = return_cols)
        .format("{:+,.2%}", subset = return_cols)
    )

    return render_template(
        "index.html",
        prices = prices.render(),
        chart = chart.getvalue().decode('utf8')
    )

styling our table --- index.py

Now our table should be looking like this. Much nicer!

symbol	date	price	daily_return	monthly_return
SPY	2021-04-14	411.45	-0.34%	+4.13%
EZU	2021-04-14	48.07	-0.02%	+3.53%
EWJ	2021-04-14	69.55	-0.19%	+0.35%
EEM	2021-04-14	53.72	+0.51%	-0.74%
IWM	2021-04-14	223.32	+0.99%	-4.56%

Updating our chart

The last step is to update our chart. We already have the skeleton ready, we only need to pass the correct data. I am also changing the y-axis to display the values in percentage terms. This way we can easily compare our ETFs relative performance over time.

Our final index.py file looks like this. Note I combined all the imports at the top to make it easier to read.


import datetime
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
import matplotlib.ticker as mtick # - needs this to customize y-axis
from flask import current_app, render_template
from .data import (
    calc_return,
    format_data_frame,
    get_price_data
)

from .util import css_variables
from io import BytesIO

app = current_app
matplotlib.use('Agg')


def plot(prices):

    prices = (
        prices
        .sort_index()
        .apply(np.log)
        .diff()
        .fillna(0.0)
        .cumsum()
        .apply(np.exp)
        .apply(lambda x: x - 1)
    )

    return prices.plot()


def export_svg(chart):
    output = BytesIO()
    chart.get_figure().savefig(output, format = "svg")
    return output


def customize_chart(chart):
    fig = plt.gcf()
    css = css_variables()

    fig.set_facecolor(css['color_1'])
    chart.set_xlabel(None)
    chart.set_ylabel("Cumulative return", color = css['color_2'])
    chart.tick_params(color = css['color_2'], labelcolor = css['color_2'], which = "both")
    chart.set_facecolor(css['color_1'])
    for s in chart.spines:
        chart.spines[s].set_color(css['color_2'])

    chart.yaxis.set_major_formatter(mtick.PercentFormatter(1.0)) # --- Y-axis in percentage terms
    return chart


@app.route("/")
def home():

    symbols = ['SPY', 'EZU', 'IWM', 'EWJ', 'EEM']
    end = datetime.date.today()
    start = end - datetime.timedelta(days = 30 * 3)

    df = get_price_data(symbols, start_date = start, end_date = end)

    # --- pass our new data to our plot

    _plot = (
        df
        .loc[:, ['date', 'symbol', 'adj_close']]
        .pivot('date', 'symbol', 'adj_close')
        .pipe(plot)
    )
    _plot = customize_chart(_plot)

    try:
        chart = export_svg(_plot)
    finally:
        plt.close()

    ret_d01 = calc_return(df, index = 1)
    ret_d21 = calc_return(df, index = 21)
    prices  = (
        df[['symbol', 'date', 'adj_close']]
        .rename(
            columns = {'adj_close' : 'price'}
        )
        .merge(ret_d01.reset_index(), how = 'inner', on = ['date', 'symbol'])
        .rename(columns = {'ret' : 'daily_return'})
        .merge(ret_d21.reset_index(), how = 'inner', on = ['date', 'symbol'])
        .rename(columns = {'ret' : 'monthly_return'})
    )

    # - styling

    return_cols = ['daily_return', 'monthly_return']
    def ret_color(x):
        color = 'tomato' if x < 0 else 'lightgreen'
        return 'color: %s' % color

    prices = (
        prices
        .sort_values('monthly_return', ascending = False)
        .assign(date = lambda df: df['date'].dt.strftime("%Y-%m-%d"))
        .pipe(format_data_frame)
        .format("{:,.2f}", subset = ['price'])
        .applymap(ret_color, subset = return_cols)
        .format("{:+,.2%}", subset = return_cols)
    )

    return render_template(
        "index.html",
        prices = prices.render(),
        chart = chart.getvalue().decode('utf8')
    )

final index.py

Now our chart is looking like the example below. If something does not look right, you can check the final code of part 2 here.

Now it is your turn

In this post, we developed our dashboard to use market data from Yahoo Finance, styled our table and updated our chart. Now we can use this dashboard to track the relative performance of stocks, ETFs, currencies, etc, easily. Now it is up to you! How can you improve this dashboard to suit your needs? For example, when we hover over our table, we lose the red/green colours we added in our styling. How could you change the code to prevent this from happening?

Our dashboard is useful, but it is a bit boring. It does not do anything, it just shows data. It would be nice if we could interact with it to display the data we want to see, not only a fixed set of symbols. That's exactly what we will be doing in part 3 of this Flask series. Stay tuned!

✌️