Fix cold-boot UART2 ownership/configuration race #10

Closed
opened 2026-10-01 15:10:17 +02:00 by hermes · 3 comments
Collaborator

Symptom

Orange Pi uses the non-console GX12 UART on /dev/ttyS2.

After a cold boot, Ground Station logs:

UART connected: /dev/ttyS2

but receives no valid GS frames. The service begins receiving frames only after this manual sequence:

sudo systemctl stop groundstation
sudo stty -F /dev/ttyS2 115200 raw -echo -ixon -ixoff
sudo timeout 5 cat /dev/ttyS2 | hexdump -C
sudo systemctl start groundstation

The raw capture then shows valid traffic and the Ground Station works.

Important facts

  • /dev/ttyS2 is the correct physical endpoint; raw capture proves RX wiring and the GX12 stream.
  • internal/serialport.Open115200 already sets 115200, raw Iflag/Oflag/Lflag, CS8/CREAD/CLOCAL, VMIN=1, VTIME=0.
  • Therefore do not add a blind second termios reset. First identify what changes the TTY after Ground Station configures it.

Leading hypothesis to prove/disprove

A boot-time unit — likely serial-getty@ttyS2.service enabled by Orange Pi UART configuration — opens or reconfigures /dev/ttyS2 after Ground Station. Manual stty after boot then wins the race.

Investigation / acceptance

  1. On a fresh boot before manual stty, capture:
    stty -F /dev/ttyS2 -a
    systemctl status serial-getty@ttyS2.service --no-pager
    journalctl -b -u serial-getty@ttyS2.service -u groundstation --no-pager
    fuser -v /dev/ttyS2
    
  2. Compare stty -a before and after the manual recovery sequence.
  3. Identify the competing owner/configuration change.
  4. Fix ownership declaratively: the Ground Station UART must have exactly one service owner and must work after a cold boot without the manual recovery sequence.
  5. Add a regression/integration check or documented boot validation; do not touch UART0/boot console.

Do not change GX12 framing, WFB/video, or the UART device path while investigating.

## Symptom Orange Pi uses the non-console GX12 UART on `/dev/ttyS2`. After a cold boot, Ground Station logs: ```text UART connected: /dev/ttyS2 ``` but receives no valid `GS` frames. The service begins receiving frames only after this manual sequence: ```sh sudo systemctl stop groundstation sudo stty -F /dev/ttyS2 115200 raw -echo -ixon -ixoff sudo timeout 5 cat /dev/ttyS2 | hexdump -C sudo systemctl start groundstation ``` The raw capture then shows valid traffic and the Ground Station works. ## Important facts - `/dev/ttyS2` is the correct physical endpoint; raw capture proves RX wiring and the GX12 stream. - `internal/serialport.Open115200` already sets 115200, raw `Iflag/Oflag/Lflag`, CS8/CREAD/CLOCAL, VMIN=1, VTIME=0. - Therefore do not add a blind second termios reset. First identify what changes the TTY after Ground Station configures it. ## Leading hypothesis to prove/disprove A boot-time unit — likely `serial-getty@ttyS2.service` enabled by Orange Pi UART configuration — opens or reconfigures `/dev/ttyS2` after Ground Station. Manual `stty` after boot then wins the race. ## Investigation / acceptance 1. On a fresh boot before manual `stty`, capture: ```sh stty -F /dev/ttyS2 -a systemctl status serial-getty@ttyS2.service --no-pager journalctl -b -u serial-getty@ttyS2.service -u groundstation --no-pager fuser -v /dev/ttyS2 ``` 2. Compare `stty -a` before and after the manual recovery sequence. 3. Identify the competing owner/configuration change. 4. Fix ownership declaratively: the Ground Station UART must have exactly one service owner and must work after a cold boot without the manual recovery sequence. 5. Add a regression/integration check or documented boot validation; do not touch UART0/boot console. Do not change GX12 framing, WFB/video, or the UART device path while investigating.
grzesiu was assigned by hermes 2026-10-01 15:10:17 +02:00
grzesiu was unassigned by piotr 2026-10-01 15:11:42 +02:00
Author
Collaborator

Root cause confirmed

Fresh-boot evidence rules out serial-getty and a competing process:

serial-getty@ttyS2.service → inactive
fuser /dev/ttyS2           → only groundstation owns it

But while Ground Station logs UART connected, the actual TTY state is:

speed 9600 baud

The GX12 emits 115200, so no valid frames can be decoded. Manual stty -F /dev/ttyS2 115200 raw ... changes the real baud and immediately restores operation.

The bug is in internal/serialport/serial_linux.go: it assigns termios.Ispeed and termios.Ospeed, but on Linux TCSETS selects the baud rate through the CBAUD bits in termios.Cflag. It leaves the inherited boot-time 9600 setting active.

Required fix

Set the actual Linux baud selection, then read it back and fail Open115200 if it is not B115200. The relevant operation is equivalent to:

termios.Cflag &^= unix.CBAUD
termios.Cflag |= unix.B115200

Keep the existing raw/CS8/CREAD/CLOCAL/VMIN/VTIME configuration. Add a focused helper/unit test for the termios transformation if possible, and an Orange Pi cold-boot acceptance check:

stty -F /dev/ttyS2 -a   # must report 115200 before any manual stty

Do not touch serial-getty, device paths, WFB, or GX12 framing.

## Root cause confirmed Fresh-boot evidence rules out `serial-getty` and a competing process: ```text serial-getty@ttyS2.service → inactive fuser /dev/ttyS2 → only groundstation owns it ``` But while Ground Station logs `UART connected`, the actual TTY state is: ```text speed 9600 baud ``` The GX12 emits 115200, so no valid frames can be decoded. Manual `stty -F /dev/ttyS2 115200 raw ...` changes the real baud and immediately restores operation. The bug is in `internal/serialport/serial_linux.go`: it assigns `termios.Ispeed` and `termios.Ospeed`, but on Linux `TCSETS` selects the baud rate through the `CBAUD` bits in `termios.Cflag`. It leaves the inherited boot-time 9600 setting active. ## Required fix Set the actual Linux baud selection, then read it back and fail `Open115200` if it is not B115200. The relevant operation is equivalent to: ```go termios.Cflag &^= unix.CBAUD termios.Cflag |= unix.B115200 ``` Keep the existing raw/CS8/CREAD/CLOCAL/VMIN/VTIME configuration. Add a focused helper/unit test for the termios transformation if possible, and an Orange Pi cold-boot acceptance check: ```sh stty -F /dev/ttyS2 -a # must report 115200 before any manual stty ``` Do not touch serial-getty, device paths, WFB, or GX12 framing.
grzesiu was assigned by piotr 2026-10-01 15:15:30 +02:00
Author
Collaborator

Assignment boundary correction

Grzesiu does not have access to the Orange Pi or any physical device. His scope is the code fix and automated/unit validation only.

Piotr will perform the hardware/OS acceptance after a reviewable PR is available:

# after cold boot, before manual stty
stty -F /dev/ttyS2 -a
systemctl status groundstation --no-pager

Expected: 115200 baud and valid GS traffic without manual recovery.

## Assignment boundary correction Grzesiu does not have access to the Orange Pi or any physical device. His scope is the code fix and automated/unit validation only. Piotr will perform the hardware/OS acceptance after a reviewable PR is available: ```sh # after cold boot, before manual stty stty -F /dev/ttyS2 -a systemctl status groundstation --no-pager ``` Expected: 115200 baud and valid `GS` traffic without manual recovery.
piotr closed this issue 2026-10-01 15:35:03 +02:00
Collaborator

Zakończone: PR #11 został zmergowany do main jako d3bc369164494438627c9ac82c6ca9421bf08644.

Poprawka ustawia B115200 przez bity CBAUD, odczytuje termios po TCSETS i odrzuca konfigurację, jeśli efektywna prędkość nie wynosi 115200. Nie zmieniono serial-getty, ścieżek urządzeń, UART0, GX12 ani WFB.

Walidacja: testy transformacji i readback termios, git diff --check, skan dodanych linii, gofmt, go vet ./..., go test ./..., buildy Linux/arm64. CI dla dokładnego SHA: success (Build Ground Station / build (push)).

Pozostaje uzgodniona akceptacja sprzętowa Piotra po zimnym starcie: stty -F /dev/ttyS2 -a ma pokazać 115200, a ramki GS mają działać bez ręcznego stty.

Zakończone: PR https://git.piksar.eu/piotr/groundstation/pulls/11 został zmergowany do `main` jako `d3bc369164494438627c9ac82c6ca9421bf08644`. Poprawka ustawia `B115200` przez bity `CBAUD`, odczytuje termios po `TCSETS` i odrzuca konfigurację, jeśli efektywna prędkość nie wynosi 115200. Nie zmieniono serial-getty, ścieżek urządzeń, UART0, GX12 ani WFB. Walidacja: testy transformacji i readback termios, `git diff --check`, skan dodanych linii, `gofmt`, `go vet ./...`, `go test ./...`, buildy Linux/arm64. CI dla dokładnego SHA: **success** (`Build Ground Station / build (push)`). Pozostaje uzgodniona akceptacja sprzętowa Piotra po zimnym starcie: `stty -F /dev/ttyS2 -a` ma pokazać 115200, a ramki `GS` mają działać bez ręcznego `stty`.
Sign in to join this conversation.
No labels
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: piotr/groundstation#10