Introducing ChronoGraph: A Real-World Graph-Based Multivariate Time Series Dataset for Research and Benchmarking

ChronoGraph is a new, real-world graph-based multivariate time series dataset designed for research and benchmarking. It offers several key advantages over existing datasets, providing a robust platform for developing and evaluating time series forecasting models.

Direct Access, Explicit Schema, and Licensing: The ChronoGraph Data Blueprint

Dataset: N=1,500 nodes; F=8; T=300; E=4; dynamic adjacency; avg degree ~4.1.
Splits: train steps 0–239; validation 240–269; test 270–299; supports rolling-origin and multi-horizon forecasting.
Storage: features.parquet (N × F × T); edges.parquet (per-edge-type COO lists); metadata.json and schema.json describe evolution and features.
Access: download at https://chronograph-dataset.org/downloads/chronograph_v1.0.tar.gz; no login; versioned releases (v1.0, v1.1, …).
Licensing/Citations: CC-BY-4.0 data; MIT code/notebooks; cite ChronoGraph v1.0 (DOI 10.1000/chronograph.2025.01); BibTeX in README.
Provenance/Quality: dataset_notes.txt documents provenance, missing-value handling, normalization, and quality flags; data dictionary provided.
Documentation/Support: dataset_schema.json, example_notebooks.ipynb, usage_guides.md, API reference, and a changelog.
Technical prerequisites: Python 3.9+, PyTorch/PyTorch Geometric or DGL backends; notebooks cover single-step and multi-step forecasting.
Reproducibility: reproducibility package includes env.yml, Dockerfile, and a script to set seeds and recreate splits.

Reproducibility-First Baselines: Configurations, Evaluation Protocols, and How to Reproduce

Having clearly defined baselines is essential for fair and interpretable comparison. Below are the concrete configurations for three representative baselines, followed by the evaluation protocol, reproducibility artifacts, and how we attribute and cite the results.

Baseline Configurations

Evaluation Protocol

Rolling-origin (walk-forward) evaluation with a fixed window length of 12 steps. The model is retrained or updated as new data becomes available, simulating a real forecasting workflow. For each horizon (1, 3, 6), report: RMSE, MAE, MAPE. Robustness is assessed by aggregating results across 3 random seeds and presenting the mean ± standard deviation (mean ± stddev).

Reproducibility Artifacts

ChronoGraph-benchmarks repository containing reproducibility scripts, an environment.yml, and a Dockerfile for consistent environments. data_split.json encodes the train/val/test splits used for evaluation. seed_setup.sh fixes random seeds to ensure deterministic runs where possible.

Citation and Attribution

Baseline results are produced using ChronoGraph’s official splits to ensure fair comparison. Include model implementation references in the project README and in the CITATION.cff file to enable proper attribution and reproducibility.

Quick Reproducibility Checklist for Baselines

Reproducibility isn’t optional when comparing baselines; it’s the baseline itself. Use this concise checklist to lock in ChronoGraph v1.0 evaluations and make fair, transparent comparisons.

• Ensure the exact ChronoGraph v1.0 splits
• Set global seeds for numpy, PyTorch, and Python’s random module to fixed values at the start of every run (for example, seed = 42).
• Load data_split.json from the repository root before each evaluation to guarantee identical train/val/test splits across runs.
• If you perform multiple runs or folds, re-seed and re-load the split state before each one to avoid carryover effects.
• Use the provided environment configuration
• Create and activate the environment from environment.yml (e.g., conda env create -f environment.yml and conda activate the environment).
• Verify PyTorch and related package versions align with the baseline implementations. A quick check is to print the version (e.g., import torch; print(torch.__version__)) and compare against the baseline reference.
• Document any deviations in dependencies and adjust reproducibility notes accordingly.
• Document and share seeds, hyperparameters, and evaluation scripts
• Maintain a run configuration file (config.json or config.yaml) that records seeds, hyperparameters, and paths to evaluation scripts.
• At program start, lock randomness across libraries: random.seed(seed), np.random.seed(seed), torch.manual_seed(seed). If using CUDA, torch.cuda.manual_seed_all(seed) and consider torch.backends.cudnn.deterministic = True and torch.backends.cudnn.benchmark = False.
• Log and publish the exact seeds, hyperparameters, and the exact scripts used to produce the results alongside the results themselves.
• Publish results in a unified JSON format

Provide a single JSON record per evaluation with fields for horizon, metrics, stds, and a short narrative on deviations. Include the following metrics: RMSE, MAE, MAPE, plus their standard deviations. Include a brief narrative describing any deviations from the expected results or baseline behavior (data drift, minor implementation differences, etc.).

End-to-End Tutorial: Quick Start with ChronoGraph

Getting ChronoGraph up and running quickly doesn’t require reading the entire paper. This streamlined guide walks you from a clean environment to reproducible results in eight concrete steps, with exact commands where provided.

Step-by-Step Quick-Start

1. Environment setup: Install Python 3.9 or newer, create a conda environment from the included environment.yml, and activate it. Check Python version: python --version (should be 3.9+). Create the environment: conda env create -f environment.yml. Activate the environment (replace chronograph with your environment name if needed): conda activate chronograph.
2. Download ChronoGraph: Fetch the ChronoGraph dataset archive, extract it, and verify the essential files exist. Download: curl -L https://chronograph-dataset.org/downloads/chronograph_v1.0.tar.gz -o chronograph_v1.0.tar.gz. Extract: tar -xzf chronograph_v1.0.tar.gz. Verify contents: ls -1 features.parquet edges.parquet metadata.json schema.json.
3. Data preparation: Run the included preprocess.py to convert parquet files into torch tensors and to generate train/val/test splits from data_split.json; outputs go to data/chronograph_processed/. Process data: python preprocess.py. Output location: data/chronograph_processed/.
4. Train a baseline: Train a baseline model (DCRNN) using the prepared data. This example sets horizons to 1, 3, and 6, with common training hyperparameters. Command: python train.py --model dcrnn --data data/chronograph_processed/ --horizons 1 3 6 --epochs 500 --lr 0.001 --batch 32 --seed 42.
5. Evaluation: Evaluate predictions against ground truth and generate both a results summary and visual plots. Evaluation: python evaluate.py --preds preds/ --labels labels/ --metrics RMSE MAE MAPE --seed 42. Outputs: results.json and a plots/ directory with time-series forecast plots.
6. Reproduce additional horizons: Repeat training and evaluation for horizons [1, 3, 6] and compare results. Export a consolidated report for reporting. Re-run training/evaluation for the same horizons as Step 4/5 as needed, ensuring seeds are fixed for comparability. Consolidated results: results_table.csv (e.g., by running a tool that merges per-horizon results from results.json into a single CSV).
7. Extensions: Swap in alternative baselines (Graph WaveNet or Temporal Graph Transformer) using the provided configs, and compare performance across horizons. Graph WaveNet: python train.py --model graph_wavenet --config configs/graph_wavenet.yaml --data data/chronograph_processed/ --horizons 1 3 6. Temporal Graph Transformer: python train.py --model temporal_graph_transformer --config configs/temporal_graph_transformer.yaml --data data/chronograph_processed/ --horizons 1 3 6.
8. Notebooks: Open the included interactive walkthrough to explore ChronoGraph visually and interactively. Notebook: example_notebooks/ChronoGraph Quick Start.ipynb.

Comparison Table: ChronoGraph vs. Prior Datasets in Real-World Graph-Based Time Series Benchmarking

Pros and Cons for Researchers

Pros

• Clear, explicit dataset schema
• Direct downloadable access with no login
• Clear licensing and citation guidelines
• Ready-to-use baseline configurations and evaluation protocols
• End-to-end tutorials and notebooks
• Supports multi-horizon forecasting on dynamic graphs
• Reproducibility artifacts (envs, Dockerfile, splits)

Cons

• Larger dataset size demands modest computational resources (RAM and GPU)
• Initial setup requires checking environment compatibility and correct parquet/dataset paths
• Learning curve for new users to align baselines with ChronoGraph’s splits and evaluation protocol

Related Video Guide