EMBEDDED OS ENGINEER

Job Description

Our client is looking for a hands-on Embedded OS Engineer to own the Ubuntu-based  
Linux firmware stack on our MK2 drone platform (Qualcomm QRB5165). You will be the  
person who builds the OS, keeps it secure, makes it reliable, and ships it through  
automated CI/CD pipelines.  

This is not a role for someone who has only worked at the application layer. We need  
someone comfortable navigating low-level OS internals, Yocto's BitBake machinery,  
system unit files, and the realities of headless embedded hardware with no display and  
limited recovery options.

**Yocto / BitBake Build System**
Own and maintain the teal-mk2-build repository, including layer configuration,  
local.conf tuning, and image recipes that produce the qti-ubuntu-robotics-image.
Write and maintain .bb recipes and .bbappend overlays; create .patch files (git  
diff--based) against the pinned Qualcomm QRB5165 BSP (LU.UM.3.3.1) so that  
upstream sources are never modified directly.  
• Debug BitBake task failures---understanding the do_fetch ? do_compile ?  
do_rootfs ? do_image pipeline, sstate-cache behavior, and when to use -c  
cleansstate vs. -c rootfs -f.  
• Optimize build performance: reduce full-build times by understanding QEMU  
emulation overhead vs. native ARM64 compilation, tuning  
BB_NUMBER_THREADS and PARALLEL_MAKE, and leveraging shared sstatecache  
across Jenkins agents.  
• Manage package inclusion, exclusion (IMAGE_INSTALL:remove,  
PACKAGE_EXCLUDE, BAD_RECOMMENDATIONS), and inter-recipe  
dependency graphs.

**Rootfs Post-Processing \& OS Hardening**
Implement ROOTFS_POSTPROCESS_COMMAND functions to perform postbuild  
OS customization: package upgrades, apt security patching (Ubuntu ESM),  
service installation, and file system fixups.  
• Remove unnecessary services from production images---including HTTP servers  
(lighttpd), TFTP daemons, and other attack-surface-expanding services identified  
in Blue List / Nessus security assessments.  
• Write bash scripts and systemd service units that execute reliably in a headless,  
rootfs post-install context (no interactive terminal, limited /proc and /dev  
availability).  
• Manage the dpkg / apt ecosystem within the embedded rootfs: pinning packages,  
handling held packages, validating package state, and ensuring apt lock files are  
clean across incremental builds

**Jenkins CI/CD Pipelines**
Maintain and improve Jenkins pipelines that build Yocto firmware images,  
including handling concurrent build isolation, artifact staging, and race conditions  
between parallel jobs.
Diagnose and resolve Jenkins agent performance issues: Java heap tuning,  
Docker overlay2 disk pressure, container lifecycle management, and build  
environment reproducibility.  
• Implement proper artifact copy patterns (stageDir isolation) to prevent race  
conditions when multiple builds run concurrently against shared directories.  
• Manage GitLab repository structure and branch protection for the build repo;  
implement .patch-based change workflows that preserve the integrity of pinned  
upstream BSP branches

**Embedded Linux System Configuration**  
• Configure systemd-networkd for runtime network mode switching (DHCP ? linklocal)  
on headless devices using udev rules, button-press event handlers, and  
LED feedback mechanisms.  
• Write and maintain systemd service units and timers: understanding WantedBy,  
After, Requires, and ExecStart semantics for embedded boot sequences.  
• Implement reliable USB logging and file transfer services (usb-gadget, udevadm)  
for field diagnostics on devices with no screen.  
• Develop and maintain Prometheus node_exporter integrations and processexporter  
configurations for drone fleet health monitoring via Grafana dashboards.

**Security \& Compliance**  
• Remediate Nessus / Blue List security findings affecting the embedded OS: SSH  
hardening (key-only auth, sshd_config tuning), open port reduction, and service  
inventory documentation.  
• Maintain /etc/shadow hygiene, locked account policies, and PAM configurations  
appropriate for production embedded devices.  
• Generate and maintain security compliance artifacts (port inventories, service  
lists, patch status reports) for internal security assessors.
Reliability \& Performance Tuning  
• Profile and diagnose slow boot sequences, runaway processes, and memory  
pressure on ARM64 embedded hardware.  
• Use screen, tmux, and remote shell tooling to manage long-running build and  
deployment sessions on headless servers and devices.  
• Implement OS-level monitoring: log capture services, boot-time diagnostics, and  
watchdog patterns for unattended field deployment.  
• Advise on cloud build infrastructure choices (AWS Graviton / Azure ARM64) to  
eliminate QEMU emulation overhead and achieve 3--5× build time reductions.