Second-order consistency for learning chaotic dynamics via randomized Jacobian matching
Short-horizon accuracy does not ensure that a learned chaotic system has correct long-time dynamics. Trajectory (zeroth-order) matching constrains vector-field values, and Jacobian (first-order) matching constrains local tangent dynamics, but neither determines how the Jacobian varies away from supervised states, so a model can be locally accurate while drifting toward spurious attractors and distorting long-time statistics. We show that second-order supervision mitigates these failures. Because forming full Hessian tensors is computationally prohibitive in high dimensions, we propose model-constrained randomized Jacobian matching, which compares the Jacobians of the true and learned vector fields at randomly perturbed inputs. A Taylor expansion shows that the expected randomized Jacobian loss decomposes into the Jacobian mismatch plus a Hessian mismatch scaled by the noise variance, implicitly enforcing second-order consistency at $O(d^2)$ memory cost without forming the $O(d^3)$ Hessian tensor. In Lorenz 63 with minimal temporal supervision, second-order supervision reduces invariant-measure error and Lyapunov-spectrum MSE, and recovers the constant Hessian norm of the true bilinear field. Across five training seeds, explicit Hessian matching produces catastrophic Lyapunov outliers for four of five seeds, whereas randomized Jacobian matching produces none among 5,000 on-attractor rollouts, and it attains the largest threshold in a directional capture scan. In coupled Lorenz 96, first-order methods enter spurious high-amplitude regimes as forcing increases, while second-order methods retain accurate marginals. Randomized Jacobian matching costs about the same as explicit Hessian matching on Lorenz 63 and 40% less on Lorenz 96, with no reference Hessian evaluations during training.