case study · scheduled etl pipeline

The efficient frontier of coffee & cotton.

Harry Markowitz's insight (1952): a portfolio's risk is not the average of its parts — when assets barely move together, blending them cancels risk out. This study applies that lens to two real ICE commodities and locates the mix that minimizes risk. It runs on a pipeline built end to end for the study, which reruns after every market close.

Yahoo FinanceICE futures · KC=F / CT=F→ Python ETLextract · transform · load→ SQLite · CSV · JSONanalytical store + exports→ This pageself-contained front-end
Orchestrated by a scheduled GitHub Actions workflow — rerun after each ICE close, auto-commit, auto-deploy.
Pythonpandas · numpySQL · SQLiteGitHub Actionsinteractive SVG
real data · ICE futures  ·  coffee KC=F · cotton CT=F  ·  2023-10-06 → 2026-10-06  ·  last refresh 2026-10-07 01:40

Latest 10 sessions

closing prices in US¢/lb · regenerated on every pipeline run
date coffee (KC)Δ cotton (CT)Δ
2026-10-06303.20+3.64%80.92+4.91%
2026-10-05292.55+1.32%77.13+2.63%
2026-10-02288.75+0.17%75.15+1.51%
2026-10-01288.25-0.84%74.03-1.02%
2026-09-30290.70+0.43%74.79-5.40%
2026-09-29289.45+0.24%79.06+0.19%
2026-09-28288.75+3.64%78.91-0.75%
2026-09-25278.60+1.18%79.51+0.76%
2026-09-24275.35-0.20%78.91-0.98%
2026-09-23275.90+1.32%79.69-0.20%
1

The two assets

Realized over the period — annualized return and risk (volatility), plus how much they co-move.
Coffee (Arabica)KC · ICE
Return p.a.31.9%
Risk (vol) p.a.38.9%

The higher-return, higher-risk asset — a sustained rally over the window, with the widest daily swings.

Cotton (No.2)CT · ICE
Return p.a.0.3%
Risk (vol) p.a.23.1%

Traded sideways: minimal return over the period, but the lower-volatility series.

Correlationdaily returns
0.04

Effectively zero — the two markets move independently. This is the condition under which diversification reduces portfolio risk.

2

Every possible mix — the risk × return frontier

Each point on the curve is a coffee/cotton blend, from 0% to 100% coffee (hover any point for its exact mix). The curve bends left: combining the two reaches lower risk than either asset alone.
37%18%26%24%16%30%5%36%-5%42%Risk — annualized volatility (%)Annualized return (%)0% coffee / 100% cotton — risk 23.1% · return 0.3%5% coffee / 95% cotton — risk 22.1% · return 1.9%10% coffee / 90% cotton — risk 21.3% · return 3.5%15% coffee / 85% cotton — risk 20.7% · return 5.0%20% coffee / 80% cotton — risk 20.3% · return 6.6%25% coffee / 75% cotton — risk 20.2% · return 8.2%30% coffee / 70% cotton — risk 20.3% · return 9.8%35% coffee / 65% cotton — risk 20.6% · return 11.3%40% coffee / 60% cotton — risk 21.2% · return 12.9%45% coffee / 55% cotton — risk 22.0% · return 14.5%50% coffee / 50% cotton — risk 23.0% · return 16.1%55% coffee / 45% cotton — risk 24.1% · return 17.7%60% coffee / 40% cotton — risk 25.4% · return 19.2%65% coffee / 35% cotton — risk 26.8% · return 20.8%70% coffee / 30% cotton — risk 28.4% · return 22.4%75% coffee / 25% cotton — risk 30.0% · return 24.0%80% coffee / 20% cotton — risk 31.6% · return 25.6%85% coffee / 15% cotton — risk 33.4% · return 27.2%90% coffee / 10% cotton — risk 35.2% · return 28.7%95% coffee / 5% cotton — risk 37.0% · return 30.3%100% coffee / 0% cotton — risk 38.9% · return 31.9%Efficient frontierdominated — same risk, less returnCotton alone — risk 23.1% · return 0.3%100% Cotton0.3% ret · 23.1% riskCoffee alone — risk 38.9% · return 31.9%100% Coffee31.9% ret · 38.9% riskMinimum-variance mix — 25% coffee / 75% cotton · risk 20.2% · return 8.3%Minimum-variance mix25% coffee · 20.2% risk
View the frontier as a table
CoffeeCottonRisk (vol p.a.)Return p.a.
0%100%23.1%0.3%
5%95%22.1%1.9%
10%90%21.3%3.5%
15%85%20.7%5.0%
20%80%20.3%6.6%
25%75%20.2%8.2%
30%70%20.3%9.8%
35%65%20.6%11.3%
40%60%21.2%12.9%
45%55%22.0%14.5%
50%50%23.0%16.1%
55%45%24.1%17.7%
60%40%25.4%19.2%
65%35%26.8%20.8%
70%30%28.4%22.4%
75%25%30.0%24.0%
80%20%31.6%25.6%
85%15%33.4%27.2%
90%10%35.2%28.7%
95%5%37.0%30.3%
100%0%38.9%31.9%

Finding — the diversification gain

25% / 75%coffee / cotton (min-variance mix)
20.2%portfolio risk (vs 23.1% cotton alone)
−13%lower risk than cotton alone
+8.0 ppmore return than cotton alone

Holding 25% coffee and 75% cotton carries 20.2% risk — below cotton on its own (23.1%) — while returning 8.3% versus cotton's 0.3%. Adding a slice of the riskier asset made the portfolio both safer and more profitable than the safe asset alone: because the two barely correlate (0.04), their day-to-day shocks offset. That is diversification, quantified on real market data.

3

Why the inputs look the way they do

The two series behind the frontier: the price paths that set each asset's return & risk, and the rolling correlation that keeps them independent.

Price paths — coffee vs cotton (shared US¢/lb scale)

Coffee's rally drives its high return and high risk; cotton stays flat and low-volatility.
CoffeeCotton
46235625014438Oct/23May/24Dec/24Jul/25Mar/26Oct/26

Rolling 63-day correlation of daily returns

Hovers near zero throughout — the diversification assumption holds over time.
0.300.150.01-0.13-0.270Jan/24Jul/24Feb/25Aug/25Mar/26Sep/26
4

How this study was built

A production-shaped data pipeline: automated capture, a relational store, and a self-contained web page rendered from the same data.
Pythonpandas · numpy · yfinance

End-to-end ETL (pipeline.py): extracts 3 years of daily ICE futures, computes returns, annualized volatility, drawdown, rolling correlation and the closed-form minimum-variance weights; renders this page as pure SVG (build_dashboard.py).

SQL · SQLiteanalytical store

The pipeline loads prices, daily metrics and correlation into a relational store with an analytical view; queries.sql documents 10 production-style queries — joins, window functions, time aggregations, CASE logic.

Interactive SVG datavizno chart library

Charts are rendered as SVG — server-side by build_dashboard.py for the initial paint, then enhanced with hand-written JS: a frontier explorer, a date-range filter and a hover crosshair. No chart library, no runtime dependency, and the frontier ships a table twin for accessibility.

GitHub Actionsscheduled automation

A cron workflow runs the pipeline after every ICE close (weekdays), commits the refreshed data and redeploys this page via GitHub Pages — the study refreshes with no server to maintain.

Method. Annualized return = mean daily return × 252; risk = standard deviation of daily returns × √252; ρ = realized correlation of the two daily-return series. The frontier sweeps the coffee weight from 0→100% using the two-asset portfolio variance with the real ρ; the minimum-variance weight is the closed-form two-asset solution. The efficient frontier is the branch above that point — every mix below it is dominated (same risk, less return). All figures computed from real ICE futures prices (Yahoo Finance). Educational study — not investment advice.