ESE 3500  ·  Team 15  ·  Spring 2026

Argus

Augmented reality smart glasses with a transparent OLED HUD, live sensor processing, and real-time activity tracking — always in your line of sight.

Seth Lee  ·  sethlee@seas.upenn.edu Jerry Zhang  ·  jerryjz@seas.upenn.edu Thomas Liu  ·  tholiu@seas.upenn.edu
github.com/upenn-embedded/final-project-s26-t15

Loading 3D model…

🖱 Drag to rotate 🔍 Scroll to zoom ⇧ Right-drag to pan
Argus Frame — Fusion 360 Final Design

Final Report

Argus Thumbnail
Argus Thumbnail

1. Video

2. Images

Demo Day Success
Demo Day Success
Final Assembly
Final Assembly
Visible Projection
Visible Projection
Final CAD
Final CAD
Electrical Systems
Electrical Systems
Firmware and Drivers
Firmware and Drivers
Isometric View
Isometric View
Side view
Side view

3. Results

Optical System

The optical subsystem was highly successful, creating a visible and clear projection of the OLED in the user's view. The text on the OLED was reflected by two mirrors and directed through a convex lens, hitting a beamsplitter that brought half the light into the user's eye. A virtual image was created 256mm in front, overlaid on the rest of the field of view to create the augmented reality effect. The final design used a 25mm diameter convex lens with a 71mm focal point, two 35×15mm first-surface mirrors placed at 45° and parallel to each other, and a 50×30mm 50/50 beamsplitter. Total light path distance was 56mm, producing a virtual image at −265mm with a magnification of 4.73. A key design change — switching from a 32mm to 71mm focal length lens — was required to push the virtual image beyond 250mm, the minimum for a clear human eye focus.

Mechanical System

The enclosure and lid were designed in Fusion 360, with subsystems optimized in Onshape. The enclosure fits the STM32, OLED, IMU, and buttons. Due to soldering difficulty, not all components ended up inside the enclosure during the demo — however, with thinner flexible wires they would fit comfortably, as validated by calipers and CAD assembly. The optical distances were accurately reflected in the CAD, enabling a clear image on the first assembly attempt.

Hardware & Firmware Systems

The hardware in the final product includes an STM32F411CEU6 (STM32 Black Pill) for its small form factor. One of the five SPI peripherals, specifically SPI4, is used to communicate and display information on the DFR0934 OLED, which served as our main output device. The glasses had two main input devices. The first, using the SPI2 peripheral, is the LSM6DS0 IMU that records the step count. The second, using an ADC, is a photoresistor divider that captures the ambient light intensity, which allows the system to automatically adjust OLED brightness. Using the step count, the speed was also approximated (in MPH) and displayed onto the OLED screen. To interface with the smart glasses, four buttons were used to trigger external interrupts via GPIO pins.

To keep the firmware organized, the drivers and code for the SPI peripherals, OLED, ADC, etc. were managed in different files. The main.c file then connects these different files to manage the main GUI. The GUI consisted of five distinct screens that the user could cycle through: the welcome screen, time screen, step-count screen, countdown timer screen, and brightness control screen.

3.1 SRS Results

IDDescriptionValidation Outcome
SRS-01 The STM32 shall read the IMU accelerometer data over I2C or SPI at a minimum rate of 50 Hz to enable accurate step detection. ~ Partial — The LSM6DS0 6-axis IMU's step data was accurately measured over SPI. However, while desoldering/re-soldering the STM32 to fit the tight space of the glasses enclosure, the SPI peripheral for the IMU was broken for the final product.
SRS-02 The step detection algorithm shall detect a walking step within 400ms of it occurring, with no more than 10% error over a 20-step walk test. ~ Partial — The LSM6DS0's step data was accurately updated within 400ms during testing. However, the SPI peripheral for the IMU was broken during final assembly.
SRS-03 The display shall update its shown content within 500ms of a button press (full round-trip: glasses ESP32 → base ESP32 → STM32 → base ESP32 → glasses ESP32 → display). N/A — The ESP32 was not used due to excessive power consumption. Decided by the first sprint review.
SRS-04 The STM32 shall synchronize its RTC with NTP time via the base ESP32 over UART once at startup. ✓ Achieved — Although the ESP32 was not used, the RTC fully worked by calibrating it with the time of the computer that flashed the code during compilation.
SRS-05 The display shall cycle through at least three different modes, such as: time view, step count view, and compass view. ✓ Achieved — The glasses had five different screens it could cycle through. Validated in the demo video.
SRS-06 The STM32 shall receive and process data from the base ESP32 over UART within 100ms of transmission. N/A — The ESP32 is not used.
SRS-07 The STM32 shall compute the compass direction from the magnetometer data and display cardinal direction (N, NE, E, SE, S, SW, W, NW) accurate to within ±22.5 degrees after calibration. N/A — A magnetometer was not used in the final product.
SRS-08 The OLED display will communicate using SPI translated through UART wirelessly between the two ESP32S3 modules to maximize screen data bandwidth. ✓ Achieved — Although the ESP32 was not used, the OLED successfully displayed output via SPI as validated in the demo video.

3.2 HRS Results

IDDescriptionValidation Outcome
HRS-01 The system shall use an IMU with at least 3-axis accelerometer, 3-axis gyroscope, and 3-axis magnetometer capabilities, communicating over I2C, for step-count and compass data. ~ Partial — Instead of a 9-axis IMU, a 6-axis IMU successfully worked using SPI.
HRS-02 The display shall be small enough to mount on glasses frames and support at least two lines of text output. ✓ Achieved — More than two lines of text were supported and the display mounted on the glasses frame as validated in the demo video.
HRS-03 Several push buttons shall be mounted on the glasses frame and register a press within 100ms of actuation to enable mode toggling, parameter changes, and power on/off. ~ Partial — Buttons are responsive per the demo video. However, there is a 300ms hardcoded delay to prevent debouncing.
HRS-04 Two ESP32 modules shall maintain wireless connection with a round-trip latency of less than 500ms within 5m of distance. N/A — ESP32 not used.
HRS-05 The STM32 shall communicate with the base ESP32 over UART at a baud rate of at least 9600 bps. N/A — ESP32 not used.
HRS-06 The glasses shall be powered by a battery capable of powering the glasses for at least 15–30 minutes continuously. ✓ Achieved — STM32 draws ~45mA, OLED ~15mA, IMU ~5mA, photoresistor ~3.3mA. Total max ~70mA. The LP552530 1S LiPo (350mAh) yields 350mAh ÷ 70mA = 5 hours — well above the 30-minute requirement.

4. Conclusion

Argus successfully demonstrated a functional pair of bare-metal smart glasses built around the STM32F411CEU. Overall, this project was successful as the most important features were completed and demonstrated. The optics and OLED screen worked well as shown in the demo video — the virtual image was successfully projected and could be comfortably seen ~10 inches in front of the user's eye. The real-time clock and timer also worked accurately. There were challenges primarily due to the compact form factor: to fit all components inside the enclosure, header pins had to be desoldered, which applied excess heat to some GPIO pins and SPI peripherals. One button and the IMU's SPI peripheral were broken as a result. Another button was soldered onto a working GPIO pin, and the OLED's SPI peripheral remained fully functional. Despite these setbacks, the IMU worked during initial testing, and with another STM32 the step counter would have worked. All other features performed well.

From this experience, we learned that building a prototype involves not only just prototyping, but also integrating the final product. Ultimately, this is where we struggled the most to turn design and MVP into functional product. Working with the large Nucleo board was much easier than a STM32 Black Pill with many pins missing functionality. In the future, we would be more deliberate in choosing and assembling parts. Several of the first IMUs chosen simply could not work due to issues between STM32 and those specific I2C peripherals, and even prototyping would have been more deliberate (mostly not soldering in those jumper pins because removing those broke our board). Additionally, we would have bought at least one more board for redundancy. As a next step, we really wanted to pursue more navigational features. We think that Argus could have great application as a tool to notify you in situations where you should keep your attention ahead of yourself, so integration with maps or a compass with more intensive bio-tracking could turn this into a much more functional product.

References

I. Gbati, Bare-Metal Embedded C Programming: Develop High-Performance Embedded Systems with C for Arm Microcontrollers. Birmingham, UK: Packt Publishing, 2024.


Final Project Proposal

01 Abstract

Argus is a pair of augmented reality glasses that utilizes a HUD and sensor suite to collect and display movement information along with live timers that always keep the user up to date with their activities. Using a STM32 MCU and XIAO ESP32S3 processors, we can wirelessly transmit data from the glasses to our pocket. Featuring a transparent OLED for the HUD, a 9-axis IMU for spatial tracking, and capacitive touch sensors, we combine portability with performance. The ESP32s relay data wirelessly from the glasses to the STM32 in the user's pocket which serves as the compute hub.

02 Motivation

Argus helps bridge the gap between physical activity and real-time data accessibility without sacrificing situational awareness. While commercial AR glasses often prioritize complex media ecosystems and overlays, Argus focuses on lightweight, purpose-focused architecture that offloads heavy processing to a pocket-based STM32 to minimize weight. The sensors provide a distraction-free, hands-free HUD for users. Instead of having to look at a watch or pull out a phone, users can find relevant telemetry in their line of sight. The emphasis on power efficiency and simplicity makes it perfect for tasks like checking direction of travel while running or biking — where looking down could be unsafe.

03 System Block Diagram

System Block Diagram

The refined system diagram (MVP Demo & Final):

Refined System Diagram

Note: this diagram only shows four buttons instead of five since the fifth button on the final product was used to replace a non-functional button due to one of the GPIO pins breaking.

04 Design Sketches

The frame of the glasses is customized through 3D printing and laser cutting to house all hardware components.

Design Sketch
Initial design sketch
Mechanical Layout
Mechanical layout with optics
Hardware Mount Prototype
Hardware mount prototype

05 Software Requirements

IDDescription
SRS-01The STM32 shall read the IMU accelerometer data over I2C at a minimum rate of 50 Hz to enable accurate step detection.
SRS-02The step detection algorithm shall detect a walking step within 400ms of it occurring, with no more than 10% error over a 20-step walk test.
SRS-03The display shall update its shown content within 500ms of a button press (full round-trip: glasses ESP32 → base ESP32 → STM32 → base ESP32 → glasses ESP32 → display).
SRS-04The STM32 shall synchronize its RTC with NTP time via the base ESP32 over UART once at startup.
SRS-05The display shall cycle through at least three different modes: time view, step count view, and compass view.
SRS-06The STM32 shall receive and process data from the base ESP32 over UART within 100ms of transmission.
SRS-07The STM32 shall compute the compass direction from the magnetometer data and display cardinal direction (N, NE, E, SE, S, SW, W, NW) accurate to within ±22.5 degrees after calibration.
SRS-08The OLED display will communicate using SPI translated through UART wirelessly between the two ESP32S3 modules to maximize screen data bandwidth.

06 Hardware Requirements

IDDescription
HRS-01The system shall use an IMU with at least 3-axis accelerometer, 3-axis gyroscope, and 3-axis magnetometer capabilities, communicating over I2C, for step-count and compass data.
HRS-02The display shall be small enough to mount on glasses frames and support at least two lines of text output.
HRS-03Several push buttons shall be mounted on the glasses frame and register a press within 100ms of actuation to enable mode toggling, parameter changes, and power on/off.
HRS-04Two ESP32 modules shall maintain wireless connection with a round-trip latency of less than 500ms within 5m of distance.
HRS-05The STM32 shall communicate with the base ESP32 over UART at a baud rate of at least 9600 bps.
HRS-06The glasses shall be powered by a battery capable of powering the glasses for at least 15–30 minutes continuously.

07 Bill of Materials

ComponentPartJustification
Main ProcessorNUCLEO-F411REMaster controller: handles step-detection (SRS-02), RTC sync (SRS-04), and I2C sensor polling at 50 Hz (SRS-01).
Wireless Bridge2× XIAO ESP32S3Communication relay: facilitates 500ms round-trip display update (SRS-03) and wireless link (HRS-04).
Transparent HUDDFR0934 OLEDVisual interface: small-form-factor mounting (HRS-02) supporting multi-mode display (SRS-05).
Motion Tracking9-DOF IMU (BNO085)3-axis Accel/Gyro/Mag data (HRS-01); enables compass accuracy within 22.5° (SRS-07) and step detection (SRS-02).
Power IsolationBuck ConverterPrevents WiFi-burst brownouts; ensures 15–30 min continuous run-time (HRS-06).
User Input5× Tactile SwitchesSatisfies HRS-03 (registration within 100ms); enables mode toggling (SRS-05).
Power Source1S LiPo BatteryLightweight for glasses-mounting while meeting run-time specs (HRS-06).
Status IndicatorPower LEDHardware-level on-state confirmation for power-on sequences (HRS-03).

08 Sprint Planning

Sprint 1

  • Connect individual components (STM32 ↔ ESP32)
  • Verify UART + WiFi connection
  • Confirm button press registration
  • Prototype glasses frame
  • Synchronize RTC with NTP

Sprint 2

  • STM32 receives IMU data over I2C
  • Display shows text/time
  • Finalize power source for glasses
  • Finalize glasses frame + fitting tests

MVP Demo

  • Steps algorithm + compass working
  • Full integration of ESP32 + display onto frame
  • RTC calibration + sync verified

Final Demo

  • Full end-to-end system functional
  • All requirements met
  • Live demo with someone wearing the glasses

09 Build Log

Sprint Review #1
Hardware decisions & optical design

Switched from BNO085 to LSM6DS0 to avoid the F411 clock-stretching errata — the LSM6DS0 has a built-in step counter and uses SPI, matching the screen bus and simplifying the driver stack.

Calculated object distance constraints from target virtual image distance and magnification. Designed mechanical layout for hardware and optics, with a secondary mirror to reduce mount size.

Optics calculator
d₀ vs. d₁ optics calculator
Prototype mount
Prototype hardware clip mount
April 9, 2026
SPI driver written, IMU driver in progress

Main SPI driver is complete. IMU driver for the LSM6DS0 is in progress using the textbook as reference. Basic GPIO input and input capture via buttons is working on the STM32. Mechanical design nearly finished.

Sprint Review #2
OLED arrived, optics redesigned, IMU validated

The OLED arrived and its driver is in development. Raytracing simulation in Onshape was used to minimize the total mount thickness, and components were measured with calipers and imported into Fusion 360.

The step counter is fully working — there is an intentional delay to avoid over-counting from oscillatory motion. Step counter demo video.

Design options
Design variants analyzed
Onshape raytracing
Onshape optical simulation
Fusion 360 CAD
Fusion 360 component layout
Step counter hardware
Step counter setup

MVP Status at Sprint 2

GoalStatus
IMU Step CounterComplete
On-Screen Timer SystemIncomplete
2–4 Buttons for Screen ControlIncomplete
Optical Waveguide HUDIn Progress
CAD of Glasses / Waveguide HolderIn Progress
Navigation System (Compass)Parts ordered
MVP Demo
OLED functional, full firmware stack

Two independent SPI buses run simultaneously — one for the OLED, one for the IMU. The firmware stack spans main.c, oled.c, spi.c, imu.c, and font.c. IMU interrupts refresh the display on each step event.

Working OLED
Working OLED display
IMU setup
IMU setup
Cardboard chassis
Flexible cardboard chassis prototype
Components in chassis
Components fitted in chassis
Glasses connection
Glasses frame connection slots
Updated CAD with mirror
CAD updated with mirror dimensions

Optical analysis confirmed the virtual image is achievable with the updated lens and beamsplitter dimensions.

Optical analysis
Optical dimension analysis
Ray simulation
Ray simulation — beamsplitter + lens

10 Final Updates

A new lens was ordered due to supply chain delays — 25mm diameter, 71mm focal length. With the constraint that the human eye focuses best at distances greater than 25cm, the object distance was set to 56mm to produce a virtual image beyond 250mm. The enclosure was redesigned to accommodate this, with mirrors placed parallel to maintain constant horizontal light travel distance.

New lens calculations
Revised lens / image distance calc
Mirror placement
Parallel mirror placement design
Printed enclosure
3D-printed enclosure

Assembly of hardware components and projection testing:

Hardware assembly
Hardware assembly
Projection test 1
Projection test
Projection test 2
Projection result