System Requirements Document

Cotton Intelligence & Price Forecasting Platform

Version: v1.0

Table of Contents

1. Executive Summary
2. Project Objectives
3. Cotton Procurement Calendar & Buying Windows
4. Data Sources & Live Pipelines
5. Price Forecasting Engine
6. Dashboard Modules
   • 6.1 Today’s Price Forecast
   • 6.2 Global Supply & Demand
   • 6.3 Crop Progress & Weather
   • 6.4 Trade & Positioning
   • 6.5 Macro Drivers & Synthetic Competitiveness
   • 6.6 India Cotton Intelligence
   • 6.7 Competing Crops
   • 6.8 Backtest & Model Performance
7. Functional Requirements
8. Non-Functional Requirements
9. Out of Scope
10. Assumptions & Dependencies

1. Executive Summary

The client requires a proprietary, AI-driven Cotton Intelligence Platform that consolidates global market data, crop intelligence, speculative positioning, macro-economic signals, and Indian domestic data into a single decision-support dashboard. This platform is powered by a production-grade price forecasting model that generates a 365-day daily forecast for ICE Cotton No. 2 futures (CT1).

The platform will ingest data from over 15 live, automated data pipelines, covering sources such as ICE futures, USDA WASDE, CFTC Commitment of Traders, Cotlook, NOAA weather, USDA NASS crop progress, ICAC, CAI India, DXY, WTI crude oil, PSF prices, and competing crop markets. All pipelines will be fully automated with no manual intervention required for routine data refresh.

The forecasting engine will be built on a Prophet + LSTM ensemble model trained on a complete multi-factor feature set, producing a daily price forecast curve across a 1-year horizon with 80% and 95% confidence intervals. The dashboard will surface this forecast alongside the full market intelligence context that drives it, enabling the procurement team to make informed, data-backed cotton sourcing decisions, particularly during the critical October to January buying window.

2. Project Objectives

• Build a live, automated data ingestion platform covering all major cotton market signals across global and Indian markets.
• Develop a production-grade ML forecasting engine that predicts ICE Cotton No. 2 daily prices for the next 365 days, refreshed automatically on each trading day close.
• Deliver a multi-module intelligence dashboard that contextualizes the forecast with supply & demand fundamentals, crop health, speculative positioning, macro drivers, Indian domestic data, and competing crop dynamics.
• Provide country-wise granularity on WASDE supply data including production, consumption, and ending stocks with month-on-month change tracking, referenced against each country’s own cotton marketing year.
• Integrate Indian cotton market data (Kapas arrivals, pressing numbers, MCX/CAI spot prices) as a dedicated module to support domestic procurement decisions.
• Quantify synthetic fibre competitiveness through PSF and crude oil relationships to support demand-side forecasting.
• Track competing crop acreage and price dynamics in both the USA and India to model supply-side planting decisions.
• Deliver a fully validated model with backtest performance metrics and ongoing accuracy monitoring.

3. Cotton Procurement Calendar & Buying Windows

Cotton is a seasonal commodity, and procurement decisions are concentrated in specific windows that align with crop harvests, arrivals, and market liquidity cycles. The platform is designed to be most actionable during these windows. The most important buying period is October through January, when the bulk of the Indian domestic crop arrives in the market and physical cotton is most actively priced and traded.

a. Key Cotton Marketing Years by Country

Each major cotton-producing country runs its own cotton marketing year aligned to its harvest and trade cycle. All data on the platform is presented against the relevant country’s cotton year.

• India: Oct – Sep; Harvest Window: Oct – Jan (Kharif)
• USA: Aug – Jul; Harvest Window: Sep – Dec
• Brazil: Apr – Mar; Harvest Window: Mar – Jun
• Australia: May – Apr; Harvest Window: Mar – May
• China: Sep – Aug; Harvest Window: Sep – Nov
• Pakistan: Aug – Jul; Harvest Window: Sep – Dec
• Uzbekistan: Sep – Aug; Harvest Window: Sep – Nov

b. The October–January Procurement Window

For Indian textile mills, the October to January window is the single most important period of the year for cotton procurement. This is when the domestic crop arrives in the market, spot prices are most competitive, and forward buying decisions for the rest of the year are shaped. The platform is specifically designed to support decision-making during this window.

• October: India crop arrivals begin; new cotton year starts.
• November: Peak arrival season; USDA WASDE revises crop estimates.
• December: India arrivals peak; US crop almost fully harvested.
• January: Arrival pace begins to slow; USDA final crop estimates.

Outside this primary window, two secondary events are important: the Brazilian harvest (March–June) and the US and India planting season (April–June).

4. Data Sources & Live Pipelines

All data pipelines will be fully automated via Apache Airflow DAGs. No manual intervention will be required for routine data refresh. Each pipeline includes data validation, SLA monitoring, automatic retry on failure (3 attempts with exponential back-off), and failure alerting to the operations team within 15 minutes of any SLA breach. A data-freshness timestamp will be visible on every dashboard module.

a. ICE Cotton Futures (CT1) — Daily Price Data

• Source: ICE Exchange / Barchart.com / Investing.com / Quandl
• Data Includes: Daily OHLCV, Open interest, ICE Certified Stock levels, 20-day rolling price volatility, Contract curve.
• Refresh: Daily, automatically triggered within 30 minutes of ICE settlement.

b. Cotlook Benchmarks

• Source: Cotlook (existing subscription)
• Data Includes: Cotlook A-Index, Cotlook production-to-use ratio (StOU), USDA vs Cotlook StOU divergence tracker, Cotlook A-Index vs ICE CT1 spread chart.
• Refresh: Daily for A-Index; monthly for StOU on Cotlook report release.

c. Monthly WASDE Analysis (USDA)

• Source: USDA PSD Online API
• Data Includes: World and country-wise production, consumption, ending stocks, imports, and exports.
• Refresh: Ingested automatically on WASDE release day, live within 2 hours of USDA publication.

d. ICAC Data (International Cotton Advisory Committee)

• Source: ICAC published reports / USDA cross-reference
• Data Includes: ICAC world production, consumption, and trade estimates, ICAC world stock-to-use ratio, ICAC price outlook reports and market commentary.
• Refresh: Monthly on ICAC report publication.

e. CFTC Commitment of Traders (COT) Report

• Source: CFTC.gov — disaggregated COT report, bulk download
• Data Includes: Managed Money (Speculative) positions, Commercial Hedgers positions, Swap Dealers positions, Other Reportables positions, Total open interest breakdown, Managed money positioning percentile rank.
• Refresh: Weekly, every Friday after CFTC publication (within 4 hours).

f. USDA Weekly Export Sales

• Source: USDA Foreign Agricultural Service (FAS) API
• Data Includes: Net weekly US cotton export sales by destination country, Cumulative season-to-date sales vs USDA seasonal export target, Top destination countries chart, 4-week rolling average export pace.
• Refresh: Weekly, every Thursday after USDA FAS publication.

g. Cotton Sowing & Plantation Data (USDA NASS)

• Source: USDA National Agricultural Statistics Service (NASS) API
• Data Includes: US cotton planted percentage, Good/Excellent condition rating, Harvested percentage, Planting pace vs 5-year average.
• Refresh: Weekly, every Monday after USDA NASS publication.

h. Weather Forecasts & Historical Anomalies

• Source: Copernicus ERA5 (historical) / NOAA GFS via Open-Meteo (forecast) / NOAA CPC (ENSO)
• Data Includes: ERA5 historical rainfall anomaly, NOAA GFS 15-day precipitation forecast, ENSO / Oceanic Niño Index (ONI).
• Refresh: Daily for ERA5 and NOAA GFS; monthly for ENSO ONI.

i. DXY US Dollar Index

• Source: FRED API (Federal Reserve Economic Data) / Investing.com / Barchart.com
• Data Includes: US Dollar Index (DXY) daily level, 20-day and 90-day DXY trend chart, DXY 20-day change.
• Refresh: Daily.

j. WTI Crude Oil Prices

• Source: EIA API / Investing.com / Barchart.com
• Data Includes: WTI Crude Oil daily price, Cotton / WTI price ratio, WTI 90-day trend chart with 20-day moving average.
• Refresh: Daily.

k. PSF (Polyester Staple Fibre) Prices & Synthetic Competitiveness

• Source: ICIS / PCI Wood Mackenzie / trade publications
• Data Includes: PSF weekly price, Cotton / PSF price ratio, 3-line normalised comparison chart, Synthetic substitution risk index.
• Refresh: Weekly.

l. CAI India Cotton Data

• Source: Cotton Association of India (CAI) published reports
• Data Includes: State-wise weekly Kapas arrivals, Cotton pressing / ginning numbers, CAI monthly crop size estimate and closing stock projection, Carry-over stock vs prior season.
• Refresh: Weekly for arrivals/pressing; monthly for crop estimate.

m. India Spot Cotton Prices

• Source: CAI / MCX / Agmarknet
• Data Includes: MCX cotton futures daily price, Physical spot prices by major market centre, India spot vs ICE CT1 basis chart.
• Refresh: Daily.

n. Competing Crops Data (USA & India)

• Source: CBOT / Yahoo Finance / Quandl (USA); DACFW / Agmarknet / USDA (India)
• Data Includes: USA CBOT front-month prices, USA USDA planted area by crop and season, USA Cotton/soybean and cotton/corn price ratio, India Kharif sown area by crop, India MSP comparison table, India Key relationships.
• Refresh: Daily for futures prices; weekly for sowing area during season; annual for USDA/DACFW area estimates.

o. Global Production & Consumption Trends

• Data Includes: World production vs consumption by season with surplus/deficit overlay.
• Refresh: Monthly, auto-triggered on USDA WASDE release day.

5. Price Forecasting Engine

The forecasting engine produces a daily ICE Cotton No. 2 (CT1) price forecast for the next 365 calendar days, refreshed automatically within 30 minutes of each ICE daily close. The engine uses a Prophet + LSTM ensemble architecture trained on a multi-factor feature set covering price history, supply fundamentals, speculative positioning, macro signals, crop conditions, and Indian domestic data.

a. Model Architecture

• Prophet (Trend + Seasonality): Captures yearly and weekly seasonality, US and India cotton year calendar effects, WASDE release date effects, and the long-run structural trend in cotton prices.
• LSTM (Multi-variate): Learns non-linear relationships and lagged dependencies across the full engineered feature set. Input window: 63 trading days. Architecture: 3 stacked LSTM layers with dropout regularisation and dense output layer.
• Ensemble Blending: Weighted average of Prophet and LSTM outputs; weights optimised on a held-out validation set using minimum MAE criterion; re-optimised quarterly.
• Confidence Intervals: Bootstrapped 80% and 95% confidence bands for all forecast horizons via Monte Carlo simulation on model residuals.
• Forecast Output: 365 daily price predictions (point forecast + upper/lower confidence bands), a 90-day fan chart for the dashboard, and scenario paths (bull/base/bear) based on feature sensitivity analysis.
• Refresh Trigger: Airflow DAG triggered by ICE EOD data arrival event; full forecast regenerated and served to the dashboard within 30 minutes of CT1 daily settlement.
• Model Versioning: Every trained model stored in versioned model registry with training date, validation MAE, and feature importance snapshot. Rollback to prior version possible within minutes.

b. Feature Engineering Inputs

• Price & Momentum: ICE CT1 - 5d, 10d, 21d, 63d price lags; 14-day RSI; 20-day rolling volatility; price vs 50/200-day moving average deviation; Bollinger band width.
• Supply Fundamentals: USDA WASDE / ICAC / Cotlook - World stock-to-use ratio, ending stocks MoM delta, production MoM delta, consumption MoM delta, USDA-Cotlook-ICAC three-way divergence.
• Speculative Positioning: CFTC COT - Managed money net position, 52-week positioning percentile rank, WoW change, open interest growth rate.
• Macro & FX: FRED / EIA / ICIS - DXY level and 20d change, WTI price and 20d change, Cotton/WTI ratio, Cotton/PSF ratio.
• Crop & Weather: USDA NASS / ERA5 / NOAA - US good/excellent rating, planted % vs 5-year average, rainfall anomaly per region (weighted composite), ENSO phase encoding.
• India Domestic: CAI / MCX - Kapas arrivals YoY percentage, India spot premium/discount to ICE (basis), pressing pace vs prior season.
• Competing Crops: CBOT / USDA / DACFW - Cotton/soy price ratio, cotton/corn price ratio, US and India cotton planted area vs prior year.
• Export Demand: USDA FAS - Weekly net export sales, cumulative as % of USDA target, 4-week rolling export pace.

c. Forecast Outputs on Dashboard

• 90-day fan chart: Last 90 days of historical price plus 90-day forward forecast with 80% confidence band.
• Full 365-day forecast curve: Daily predicted price with upper and lower confidence bounds — interactive and downloadable as CSV.
• Scenario price paths: Bull case, base case, bear case at 30-day, 90-day, 180-day, and 365-day horizons.
• Rolling 20-day MAE tracker: Live model accuracy vs recorded actuals, updated daily.

6. Dashboard Modules

The platform dashboard comprises eight modules. All modules display a data-freshness timestamp and source attribution. Charts are interactive with zoom, hover tooltips, and downloadable PNG/CSV export.

6.1 Today’s Price Forecast

• Current ICE CT1 settle price (last close) with date stamp.
• 90-day historical price chart plus 90-day forward forecast fan chart with 80% confidence interval.
• Full 365-day forecast curve with confidence bands (interactive, zoomable, CSV downloadable).
• Scenario price paths: Bull / base / bear at 30-day, 90-day, 180-day, and 365-day horizons.
• 20-day rolling volatility chart — current market turbulence indicator.
• ICE Cotton OHLCV candlestick chart (last 90 days) with volume bars.
• Recent prediction history table: Last 30 trading days of predicted vs actual price with absolute error.

6.2 Global Supply & Demand

• World production vs consumption by country and season: Identifies net producers vs net consumers and provides a structural supply/demand picture.
• Country-wise production trend (last 10 cotton years): Highlights Brazil growth, US fluctuation, India vs China production trajectory.
• Country-wise mill consumption trend: Emphasizes China and India dominance; changes in Chinese state reserve policy as a significant demand variable.
• Global trade flows: Details top exporters (Brazil, USA, Australia) vs top importers (China, Bangladesh, Vietnam, Turkey) with annual volumes and market share trend.
• World Stock-to-Use Ratio (USDA): Bar chart by cotton marketing season, last 15 seasons — colour-coded (below 50% tight, above 65% ample).
• Country-wise WASDE delta table: Top 10 countries, production/consumption/ending stocks, current vs prior month estimate, MoM delta with traffic-light colouring — all figures shown against each country’s own cotton year.
• Country-wise closing stocks panel: Brazil, India, USA, China, Pakistan, Australia, Uzbekistan — current season vs prior season bar chart.
• Cotlook production-to-use ratio: 6-month rolling window alongside USDA StOU.
• USDA vs Cotlook vs ICAC StOU divergence tracker: Three-way view of leading supply estimates.
• Cotlook A-Index vs ICE CT1 (last 6 months): Basis chart — physical world vs futures.
• Refresh: Monthly on WASDE and ICAC publication.

6.3 Crop Progress & Weather

• US planting pace vs 5-year average: Weekly percentage points ahead or behind.
• USDA NASS crop progress: Planted %, good/excellent condition rating, harvested % — seasonal line chart vs prior year.
• Rainfall anomaly by region: Historical % deviation from 5-year mean, last 90 days, for 6 growing regions.
• 15-day rainfall forecast overlay: NOAA GFS forward precipitation forecast by region on the same chart as historical anomaly.
• ENSO / ONI index: Current phase, monthly value, 3-month outlook with impact note for Indian monsoon and key growing regions.

6.4 Trade & Positioning

• Full CFTC COT disaggregated table: Managed Money, Commercial Hedgers, Swap Dealers, Other Reportables — long/short/net with WoW delta column.
• Managed money net position: 52-week weekly bar chart.
• Commercial hedger net position line chart: Increasing commercial longs signal physical buying.
• Total open interest trend: Rising OI with rising price confirms trend conviction.
• Positioning percentile rank (5-year): Where managed money net currently stands relative to history.
• USDA weekly export sales: Net sales by week, cumulative vs USDA seasonal target, top destination countries.

6.5 Macro Drivers & Synthetic Competitiveness

• DXY 90-day trend chart: Stronger USD is bearish for cotton exports.
• WTI Crude Oil 90-day trend: Higher crude raises polyester cost, supports cotton demand.
• Cotton/WTI price ratio (90-day window).
• PSF price series: Weekly INR/kg and USD/tonne, last 12 months.
• Cotton/PSF price ratio with 90-day mean reference line.
• 3-line normalised comparison chart: Cotton vs PSF vs WTI crude (all rebased to 100 at window start).
• Synthetic substitution risk index: Composite signal based on Cotton/PSF and Cotton/WTI ratios vs 2-year averages.

6.6 India Cotton Intelligence

• Weekly Kapas arrivals by state: Stacked bar chart, season-to-date vs prior 3 seasons (Oct–Sep cotton year).
• Cotton pressing / ginning pace: Weekly pressing numbers, cumulative vs prior year.
• India spot cotton prices: MCX futures and physical spot by market centre (Rajkot, Akola, Guntur, Adoni) — daily.
• India spot vs ICE CT1 basis chart (6 months): Premium/discount of Indian physical to ICE futures.
• CAI monthly crop estimate and closing stock projection: India cotton balance sheet panel.
• Carry-over stock vs prior season YoY comparison.

6.7 Competing Crops

• USA normalised price index: Cotton, soybean, corn, wheat front-month futures rebased to 100 at season start — relative attractiveness for farmers.
• USA acreage: USDA planted area for cotton vs soybean, corn, wheat — latest season vs prior 3 seasons.
• Cotton/soybean and cotton/corn price ratio charts with historical average reference line.
• India kharif acreage dashboard: State-wise sown area for cotton, soybean, groundnut, maize, rice, pulses (Maharashtra, Gujarat, Telangana) vs 5-year normal — weekly during sowing season.
• India MSP comparison table: Current season MSP for cotton vs all major competing kharif crops — ratio highlights farmer switching incentive.
• Key competition notes by region: Maharashtra (soy/groundnut), Telangana/AP (rice/maize), Marathwada (tur dal), US Mid-South (soy/corn), Texas (wheat/corn).

6.8 Backtest & Model Performance

• Model accuracy summary: MAE for near-term, 1-week, and 1-month horizons across all historical cutoffs.
• Predicted vs actual scatter chart: Coloured by absolute error magnitude.
• Absolute error per cutoff bar chart with 3¢ go-live gate reference line.
• Direction accuracy: Percentage of forecasts where predicted direction matched actual direction.
• Rolling 20-day MAE (live): Model drift indicator, updated daily.
• Full backtest results table: Every historical cutoff with last close, predicted price, actual price, and absolute error — downloadable.

7. Functional Requirements

8. Non-Functional Requirements

• Performance: Dashboard page load < 3 seconds. API response < 500ms for pre-computed data. Forecast refresh complete within 30 minutes of ICE CT1 daily settle.
• Data Freshness SLAs: ICE price: within 30 min of EOD settle. WASDE: within 2 hours of USDA release. CFTC COT: within 4 hours of Friday publication. NASS: within 2 hours of Monday release. Weather: daily by 08:00 IST. CAI India: within 24 hours of publication.
• Availability: 99.5% uptime SLA for the dashboard (measured monthly). Planned maintenance windows communicated 48 hours in advance. Auto-scaling handles peak concurrent users without degradation.
• Security: All API keys stored in AWS Secrets Manager with automated rotation. HTTPS enforced across all endpoints. Role-based access control: admin and read-only tiers. No raw data or model weights exposed in frontend responses.
• Scalability: Backend on ECS Fargate with auto-scaling. TimescaleDB with hypertable partitioning for time-series performance. S3 data lake designed to support 10+ years of historical data without architectural changes.
• Observability: CloudWatch dashboards for all pipeline and API metrics. Data-freshness timestamp visible on every dashboard module. Rolling MAE chart monitors model drift daily. Structured logging for all pipeline runs.
• Auditability: Every forecast logged with timestamp, model version, and input feature snapshot. Full WASDE ingestion history retained. Immutable raw data lake preserves original source data. Model registry retains all trained versions.
• Browser Support: Chrome, Firefox, Safari — latest 2 versions each. Responsive layout supporting desktop and tablet viewports (minimum 1024px width).

9. Out of Scope

The following are explicitly outside the scope of this platform. These exclusions apply regardless of how market conditions evolve during the project:

a. Operational Exclusions

• ERP or procurement system integration — no direct connection to the client’s internal systems.
• Automated trade execution, order placement, or hedging workflow automation.
• Real-time intraday tick-by-tick ICE price data — EOD settlement data only.
• Currency hedging analytics or INR/USD forward rate modelling.
• Commodity markets beyond cotton and its defined substitutes and competing crops.
• Native mobile application (iOS or Android) — the web dashboard is responsive but a dedicated app is not in scope.

b. Geopolitical & Exogenous Event Forecasting

The platform is a data-driven quantitative intelligence tool. It does not attempt to predict, model, or provide signals on events that are inherently unpredictable or outside the domain of market and agronomic data. The following are explicitly out of scope:

• Geopolitical events: trade war escalations, tariff policy changes, US-China trade relations, sanctions, or the impact of statements by political figures (including but not limited to policy announcements by heads of state).
• Pandemic or public health crises (e.g. COVID-19 type disruptions to supply chains or mill demand).
• Extreme weather events or natural disasters beyond what is captured in the 15-day NOAA GFS forecast — long-range drought prediction, cyclone track forecasting, or flood modelling are not within scope.
• Force majeure events: port strikes, war, civil unrest, or infrastructure failures in producing or consuming countries.
• Central bank policy surprises or currency crisis events that cause sudden DXY dislocations beyond the model’s training distribution.
• Regulatory changes in cotton trading, export bans, or domestic price controls by any government (e.g. India’s cotton export policy changes).
• Predictions of Chinese state reserve buying or selling decisions, which are announced without public advance notice and frequently override market fundamentals.

Where such events occur, the model’s confidence intervals will widen (as volatility rises), and the dashboard will reflect the resulting market data. The platform is a tool to support informed judgment — it is not a replacement for it in extraordinary circumstances.

10. Assumptions & Dependencies

• The client to provide or confirm access to CAI India data — either via existing subscription or by confirming scraping rights on CAI public releases.
• PSF price data will be sourced from ICIS or PCI Wood Mackenzie — client to advise preferred vendor or confirm EMB to procure independently.
• ICE CT1 end-of-day data will be sourced via Yahoo Finance, Barchart.com, or Investing.com; if the client requires a direct ICE data feed, this will be scoped separately.
• DACFW and Agmarknet India crop data is publicly available; if access restrictions arise, USDA attaché reports will be used as an alternative with prior agreement.
• A minimum of 5 years of labelled historical data is available across all pipeline sources for model training and validation.
• Client UAT feedback cycles will be completed within 5 business days of each milestone delivery to avoid sprint delays.