Autoregressive Diffusion Tested for Market Data Generation
multimodal robotics vector-db
| Source: HN | Original article
Researchers explore applying autoregressive diffusion models—previously successful for images, video, and robotics—to generate synthetic market data.
A Jane Street research intern has built an autoregressive diffusion model that attempts to synthesize raw market‑order‑book activity. Inspired by recent work on autoregressive image generation without vector quantisation, the intern—identified only as Kavish—treated each order‑book event (arrival time, price, size) as a continuous signal, analogous to video or audio streams, and applied flow‑matching and atom‑smoothing techniques to train the diffusion process.
The experiment showed that a fully continuous formulation struggles to capture the “jagged” characteristics of real‑world trading data. Market activity exhibits sharp discontinuities such as clustering of orders at round numbers and the frequent “pennying” behaviour where prices move by a single tick rather than the smoother two‑tick shifts seen elsewhere. While the model generated plausible sequences, it fell short of reproducing these discrete spikes accurately.
The work matters because realistic synthetic market data could accelerate the development and testing of algorithmic trading strategies, risk‑management tools, and regulatory stress‑tests without exposing proprietary or sensitive information. Demonstrating that diffusion models—already successful in image, video, robotics and audio domains—can be extended to financial time series opens a new research frontier, but also highlights the need for hybrid approaches that respect the inherent discreteness of market microstructure.
Future steps will likely focus on integrating discrete handling mechanisms, such as quantised diffusion or mixed continuous‑discrete architectures, and on benchmarking the synthetic output against live order‑book streams. Observers will watch for follow‑up publications from Jane Street or academic collaborators that refine the technique and assess its practical utility for trading desks and market‑data providers.
Sources
Back to AIPULSEN