When BindsTo= takes your MAVLink router down with it

A companion computer can drop telemetry without any radio fault. If the MAVLink router’s unit is bound to another unit, the router stops when that unit dies, and range control hears nothing.

The journal shows the pattern within milliseconds, but only the unit files can confirm it. Here is how to read the first and check the second.

What the journal shows

The example is INC-0142, an example from the Foxborne example dataset, not a field report. On a Q4 recon quad, the kernel killed perception_node for running out of memory at 14:32:04.118 UTC. On the same companion computer, mavlink-router.service carries MAVLink between the flight controller’s UART and the mesh radio.

14:32:04.118  kernel   Out of memory: Killed process 2213 (perception_node) total-vm:11873248kB, anon-rss:6823516kB, file-rss:10240kB, shmem-rss:0kB, UID:1001 pgtables:14720kB oom_score_adj:0
14:32:04.121  systemd  perception.service: A process of this unit has been killed by the OOM killer.
14:32:04.133  systemd  perception.service: Main process exited, code=killed, status=9/KILL
14:32:04.134  systemd  perception.service: Failed with result 'oom-kill'.
14:32:04.139  systemd  Stopping MAVLink router...
14:32:04.152  systemd  Stopped MAVLink router.

Read the last two lines closely. systemd wrote them itself, as PID 1, with UNIT=mavlink-router.service, and they name the unit by its Description=, as systemd’s status messages do.¹ A router that crashed would log “Main process exited”, as perception did.

This router did not crash. systemd stopped it 34 ms after the kernel’s kill line and 19 ms after it recorded the exit of perception.service. The monotonic timestamps agree: 1,152,938,000 minus 1,152,904,000 µs is 34 ms.

All of these lines come from one journal on one boot, so the intervals do not rest on clock sync between machines. The next piece of evidence comes from another machine.

Range control last heard the vehicle at 14:32:03.912. The next heartbeat was due at 04.912, 0.76 s after the router stopped, and never arrived. Nothing did until 14:32:10.874, 6.962 s after the last one.

PX4 saw the same gap from its side: telemetry_status on TELEM2 reported no ground station heartbeat at 14:32:06.510. It recorded no failsafe, since the gap was shorter than COM_DL_LOSS_T, 10 s on this vehicle.

Three dependencies, three behaviors

systemd has three settings that tie one unit’s life to another’s. They differ in exactly the case that matters here: a unit that dies without anyone stopping it. Written on the router, pointing at perception:¹

• Requires= stops or restarts the router when perception is explicitly stopped or restarted.
• PartOf= does the same, limited to stops and restarts that systemd carries out.
• BindsTo= does all of that, and stops the router when perception stops unexpectedly or fails.

The man page is explicit that a service exiting on its own does not propagate through Requires=.¹ An OOM kill is that kind of exit, and it left perception failed. Of the three, only BindsTo= predicts a router stopped 19 ms after perception’s main process died.

That makes the binding the leading hypothesis, not the only one. A router set to StopWhenUnneeded=, which stops a unit once no active unit requires it, could produce the same two lines.¹ So could a script that stops the router when perception fails.

Why it came back

14:32:09.141  systemd          perception.service: Scheduled restart job, restart counter is at 1.
14:32:09.203  systemd          Started MAVLink router.
14:32:09.214  systemd          Started Perception service.
14:32:10.702  mavlink-routerd  Opened UART [1]fc: /dev/ttyTHS1 baud=921600
14:32:10.718  mavlink-routerd  Opened UDP Client [5]mesh: 10.20.0.2:14550

The router started again 62 ms after perception’s restart job was scheduled. Two kinds of setting could explain that. A Wants= or Requires= in perception.service starts the router whenever perception starts, and BindsTo= and PartOf= both carry restarts across.¹

The order of the two “Started” lines says one more thing. With After=perception.service, the router would not start until perception had finished starting.¹ The router reported started 11 ms before perception did, so any binding here lacks the After= the man page suggests pairing with it.¹

RestartMode= explains why a restart still trips a binding. By default, an automatic restart passes through a failed or inactive state.² With RestartMode=direct, added in systemd 254, the service skips that state and dependent units are not notified.²

A started unit is not a working link. mavlink-routerd opened the flight controller’s UART 1.499 s after its unit started, and the mesh endpoint 16 ms later. Range control heard the vehicle 156 ms after that.

Test it before anyone swaps a radio

Start with the configuration systemd actually loaded, not the file you expect. The first command is the report’s own next step:⁵

systemctl show mavlink-router.service -p BindsTo -p PartOf -p Requires
systemctl show perception.service -p BoundBy -p ConsistsOf -p RequiredBy -p Wants
systemctl show mavlink-router.service -p After -p StopWhenUnneeded
systemctl cat mavlink-router.service

The second command reads the same links from the other end. BindsTo= on the router shows up as BoundBy= on perception, and PartOf= as ConsistsOf=.¹ systemctl cat prints the unit file and its drop-ins, so you can see which file adds a binding.⁵

The upstream unit is not the culprit. The one mavlink-router ships names no other unit in its [Unit] section:³

[Unit]
Description=MAVLink Router

[Service]
Type=simple
ExecStart=@BINDIR@/mavlink-routerd
Restart=on-failure

[Install]
WantedBy=multi-user.target

A binding, if there is one, was added downstream, in a unit file or a drop-in. The project’s README recommends its configuration file over editing the systemd unit.⁴

Then reproduce the stop on a bench unit running the same image. The OOM killer ends its victim with SIGKILL, so send the same signal to perception’s main process and watch both units:⁷

# Terminal 1: both units, with microsecond times
journalctl -f -o short-precise -u perception.service -u mavlink-router.service
# Terminal 2: the signal the OOM killer sends, without the memory pressure
kill -s KILL "$(systemctl show -p MainPID --value perception.service)"

If the binding is there, “Stopping MAVLink router...” follows “Main process exited” within milliseconds, as it did in the example. The unit will not report oom-kill this time, and that does not matter: BindsTo= reacts to the stop, not to its cause.¹ Time the gap at the ground receiver while you do it.

If the binding is there

Ask whether the router needs perception at all. Its job is to route MAVLink between endpoints over UART, UDP and TCP.⁴

If perception is the one that needs the router, put the dependency on perception’s side: Wants=mavlink-router.service and After=mavlink-router.service in perception.service. The man page calls Wants= the recommended way to hook one unit’s start-up to another’s.¹

If the binding has to stay, RestartMode=direct is the man page’s tool for this case: dependent units are not notified of the temporary failure.² It needs systemd 254 or later, so check systemctl --version first.⁵

Either way, run systemctl daemon-reload, since systemd keeps its own view of the unit until you do.⁵ Then repeat the bench kill. The fix is proven when the router survives it and the ground receiver never notices.