High-Dimensional Inference under Dynamics and Complexity: Applications in Biology and Finance