Virtual Neural Lab - Tracking

The brain captures such nature of information processing, capturing some ways that just happen to work. We are not trying to mimic the brain, but try to capture such natural way of processing by using inspirations from brain. For an organism to exist today, it must be able to survive to pass on its genes. We believe that creating motion, interacting with the world, and making "output" is what the brain evolved to do: it is what keeps us alive to pass the genes. Hence, we think that the capability to describe motions and behavior is really powerful; it is an integration of all computations from sensory to supervisory signals.

I am working as an research intern at Talmo's Lab in the Salk Institute. We collaborate with Department of Organismic and Evolutionary Biology at Harvard University, Department of Biology at the University of Washington, and Mila - Quebec AI Institute, aiming to use generative inverse kinetmatics imitation learning and goal-directed deep reinforcement learning to create highly efficient and parallelizable systems for computational models of the motor cortex.

Abstract Deep Imitation Learning Idea (curtesy of Talmo's VNL slides)

We believe that inverse kinematics imitation injects the basis layer alignment with biology to support more abstratc expolration of representations of the brain in artificial agents. Thus, we aim to create pipelines with architectures and learning algorithms that are capable of generalizing and continual learning of low-level skills that are transferable for multiple higher-level task-driven goals, trying to get closer to what the "real brain" is capable of doing.

Deep imitation learning illustration (curtesy of VNL research strategy)

Particularly we are interested in the motion of bio-mechanically realistic rodent model as we can use them (along with many motion captured CV data) to perhaps form an neural represenattion of how control is been done in the brain. For the speed and computability, we use gpu based data-parrallelism training using Jax/Brax/Flax (Mimic-MJX) or cpu-based multi-node task-parralelism distributed training using Ray/Acme (VNL-Ray). For illustration purpose, here are some recent results that our team produced.

A MJX trained PPO agent (left) and a Mujoco trained DMPO agent (right).

More analysis of the learned representation (through a topological lens) was down in this supplementary project