r/ControlTheory • u/bruno_pinto90 • Jul 04 '26
Technical Question/Problem Perception Aware MPC
Hello all,
I am sharing a small NMPC project I've been building as part of a larger thesis on drone target-tracking. Check here https://github.com/brunopinto900/argus/tree/master
This is simulation of 12-state quadrotor (position, velocity, ZYX Euler angles, body rates), 4 inputs (collective thrust + commanded body rates, matching a PX4-style body-rate setpoint interface). Formulated as a NONLINEAR_LS OCP in acados, code-generated to C and called from C++.
Alongside the usual position/velocity tracking terms, there's a soft constraint keeping a ground target inside the camera's FOV cone.
The OCP's prediction model uses a first-order lag for the inner rate loop, but the plant simulator runs the actual second-order dynamics.
Result in the gif: 2m-radius circle at 1.5m altitude, 3m/s, camera locked on a static ground target, ~0.1m steady-state XY tracking error.
•
u/jnez71 Jul 05 '26
Did you learn anything interesting from building and/or running this?
→ More replies (0)
•
u/nikiexpo Jul 05 '26
Quite cool!
Regarding the fov cone soft constraint is handled, I assume you just minimise the deviation from the centre of your cone projection to your target. It’s pretty practical, but in this case the soft constraint is always active even when the target is in frame. You can construct a constraint that only activates if a condition is met such as “penalise deviation if the target leaves the frame”, they are called complimentarity constraints.
This is just an interesting tidbit, I’m sure it’s not practical to attempt to solve an OCP with complimentarity conditions in real-time.
Cool project though, good luck!!
→ More replies (0)
•
u/LordDan_45 Jul 05 '26
It'd be a nice touch to see how this fares against disturbances, given that you are using a camera
→ More replies (0)
•
•
u/hs123go Jul 04 '26
It appears to be a strict subset of PAMPC (Falanga 2018). The perception term is a scalar FOV-cone membership penalty, whereas PAMPC uses the full pinhole projection.
Another worrying sign is that the navigation task (2m circle, 1.5m altitude, hardcoded target at origin) synergizes with perception-awareness a-la PX4 Orbit mode. PAMPC's racing task is harder since it makes perception compete with agile flight, so the solver must trade them off by tuning.
On the upside, modern C++ encapsulation of an acados MPC is honest engineering work, but it's already solved by Agilicious (Foehn 2022).
Generally, "Perception-aware aerial tracking" is well-trodden ground in the autonomous drone racing domain, with the classics (PAMPC, agilicious) providing good baseline analytic solutions, and later RL-based works and Skydreamer (MAVLAB) tackling harder real-world configurations. I'd say your work is good for an early chapter in your thesis, but the remaining 60-70% needs to address harder problems.
→ More replies (0)
•
•
0
u/[deleted] Jul 04 '26
[deleted]