antifold

Research

Physics Aware Flies: Generative Dynamics on SE(3)

August 28, 2024 • Research • Simulation

How symmetry, noise and reward shape motion when treated as group dynamics rather than vector space operations.

🎮 Experience the Interactive Simulation

Generative dynamics on SE(3) reveal how symmetry, noise and reward shape motion

Abstract

Motion in three dimensions is naturally expressed on the Lie group $SE(3)$, the set of rigid transforms combining rotations $SO(3)$ with translations $\mathbb{R}^3$. Here we present an interactive swarm in which each agent ("fly") evolves by sampling infinitesimal on-manifold increments and applying them by left multiplication. A small generative prior supplies candidate increments; a GAN-like critic assigns rewards tied to user-placed attractors, repellents and a diffusing scalar "plume". Selecting and reinforcing high-reward proposals produces structured transport while preserving the symmetries of space.

Main

The pose of each agent is a homogeneous transform

$$T=\begin{pmatrix}R & t \\ 0 & 1\end{pmatrix}\in SE(3),\quad R\in SO(3),\ t\in\mathbb{R}^3.$$

Updates are performed in the tangent space $\mathfrak{se}(3)$: we draw a twist $\xi=[\rho,\omega]\in\mathbb{R}^6$ and advance

$$T_{t+1} \;=\; \exp(\xi_t)\,T_t.$$

The exponential map uses the classical Rodrigues form for rotations,

$$R(\omega)=\exp(\widehat{\omega})=I+\frac{\sin\theta}{\theta}\widehat{\omega}+\frac{1-\cos\theta}{\theta^2}\widehat{\omega}^2,\quad \theta=\|\omega\|,$$

and a coupled translation via

$$\exp\!\begin{pmatrix}\widehat{\omega}&\rho\\0&0\end{pmatrix} = \begin{pmatrix} R(\omega) & V(\omega)\rho\\ 0 & 1 \end{pmatrix},\quad V(\omega)=I+\frac{1-\cos\theta}{\theta^2}\widehat{\omega}+\frac{\theta-\sin\theta}{\theta^3}\widehat{\omega}^2.$$

In this formulation, orientation and position evolve coherently and never leave the manifold; there are no parameterization artefacts.

Candidate increments are drawn from a mixture prior with four physically interpretable components. A Haar-uniform term supplies orientation noise that is truly uniform on $SO(3)$—a symmetry-respecting baseline for exploration. A goal-seeking term concentrates proposals toward the nearest user-placed attractor (implemented as a von-Mises–Fisher-like directional sampler). A plume-following term aligns proposals with the gradient of a diffusing, decaying scalar field seeded by clicks. A small exploration term contributes body-frame jitter. Visual "ghost" segments in the scene show these one-step proposals, making the prior itself observable.

A lightweight critic converts a proposed next state $T'$ into a scalar reward,

$$R(T') \;=\; w_g\,\phi_{\mathrm{goal}}(T') + w_p\,\phi_{\mathrm{plume}}(T') - w_r\,\phi_{\mathrm{repel}}(T') - w_w\,\phi_{\mathrm{wall}}(T') - w_c\,\phi_{\mathrm{collision}}(T'),$$

favoring proximity to attractors and strong plume signal, while penalizing walls, repellents and crowding. At each step, the agent selects the highest-reward proposal (a one-step free-energy descent) and updates its mixture weights by a small policy-gradient correction,

$$\Delta \alpha_c \;\propto\; \eta\,\big(\mathbf{1}\{c=c^\star\}-\pi_\alpha(c)\big)\,R(T^\star),$$

where $\alpha$ are component logits and $\pi_\alpha$ their softmax. Components that repeatedly yield good moves grow in prevalence; those that do not recede. This "GAN-like" loop is intentionally modest: it shapes behaviour without heavy function approximation, and it leaves the manifold geometry intact.

The resulting dynamics resemble Langevin motion on a symmetry manifold: symmetric noise (Haar on $SO(3)$) plus potential-driven drift derived from user-defined fields. The white box acts as a boundary condition; reflections model elastic encounters. The plume field provides an intuitive language for exogenous information—clicks inject localized sources into a diffusion equation that then spreads and decays. Because all updates are computed in $\mathfrak{se}(3)$ and applied via the exponential map, the simulation respects the kinematics of rigid bodies used in robotics, vision and molecular modelling.

Key Observations

Several observations follow. First, geometry matters: using $SE(3)$ directly avoids the biases and singularities of Euclidean parameterizations, and the Haar component prevents covert frame preferences. Second, structure can be light: a small set of task-aligned directions, blended with symmetry-respecting noise, already produces rich transport. Third, the prior should be visible: rendering proposals exposes failure modes (e.g. collapsed headings or wall hugging) and makes tuning ethical—users can see what the model intends to do before it acts. Finally, fields are a programmable substrate: attractors, repellents and plumes offer a compact way to sculpt behaviour without specifying policies explicitly.

Although the present implementation is pedagogical, the ingredients align with practice: manifold-correct motion models for tracking, proposal priors for 6-DoF pose estimation, and energy-shaping controllers for navigation. The interactive setting highlights the unifying theme: symmetry, noise and reward can be composed cleanly when the state space is treated as a group, not a vector space.

Technical Implementation

The simulation demonstrates several key technical concepts:

Applications and Extensions

This framework has implications for:

🎮 Interactive Exploration

Experience the concepts firsthand in our interactive simulation. Click to add attractors (gold), repellents (red), and plumes (blue). Watch how the ghost segments reveal the generative prior's proposals, and observe how the flies adapt their behavior through online learning.