Since naturalistic behaviors are, in nature, to be long horizon and highly none-linear, none-gaussian with respect to their goals, this sequence modeling problem would be particularly challenging as we want to generate long biomechanically realistic sequences from modeling the progression of these goal sequences that has only partial observables (keypoints).
To this end, advised by professor Scott Linderman at Stanford University, I am doing sequence modeling using Mimic-mjx, which provide us with observables that was not available previously, to develop state space models (i.e. AR-HMM, SLDS, KF, EKF, etc) and deep state space models (i.e. S4, S5, Mamba, etc).
Codebase (Closed repo for now, will open later)