final-project-shorts-and-sparks

final-project-skeleton

* Team Name: Shorts and Sparks
* Team Members: Tim Zhang and James Steeman
* Github Repository URL: https://github.com/upenn-embedded/final-project-shorts-and-sparks
* Github Pages Website URL:  https://upenn-embedded.github.io/final-project-shorts-and-sparks/ (full repo) or https://jamessteeman.github.io/t19ese5190f24/ (final report only repo)
* Description of hardware: Atmega328PB, M2 Macbook Pro

Final Project Report

Don’t forget to make the GitHub pages public website! If you’ve never made a Github pages website before, you can follow this webpage (though, substitute your final project repository for the Github username one in the quickstart guide): https://docs.github.com/en/pages/quickstart

1. Video

Google Drive Video

2. Images

Demo Day picture: Left Center: James, Right Center: Tim

Annotated image of components placed above the ceiling:

Annotated image of turret below the ceiling:

Final Hardware Diagram: Note: Camera damage prevented camera code from implementation in final integration (see below for more infomation)

Final Code Diagram: Note: Camera damage prevented camera code from implementation in final integration (see below for more infomation)

3. Results

The turret features 2 degrees of freedom, one azimuth rotation and one elevation rotation. This combined motion allows it to point anywhere in a half sphere below the ceiling. The azimuth control uses a NEMA 17 motor with a 4:1 internal gear reduction and acceleration control to prevent step skipping and enable greater top speed. This allows the platform itself to easily hit over 3 rotations every second, well beyond our expected maximum of 1 rps. The elevation control uses a high torque servo with a barrel connected to the arm to direct the water jet.

Water is pumped using a 12V gear pump that is both self priming and reversible. A rotary union and tubing is used to transport water across the rotating element. A nozzle attachment enables powerful water jets over 10m.

On board buzzers are programmed to produce an obnoxious fire alarm.

This final solution/result was essentially the same as the proposed; however, due to the damage to the camera’s themopile array, autonomous mode is currently not available for the turret. We are planning to complete this implementation with a replacement camera after the course.

For demonstration purposes, the turret is pre-programmed to known directions of the targets, and is able to consistently hit targets at various distances up to a 2 meters radius, over 4 times the original design requirements. This is shown in the video demo where the turret easily blasts mock “fire” targets.

3.1 Software Requirements Specification (SRS) Results

SRS 01 – Motor control functionality shall be implemented to enable smooth, fast and accurate motion over the complete range of turret motion.

• Achieved.
• Measurements were made by setting a position and rotating the turret to that position. In the process, there was no step skipping from the stepper, and the servo would also reach its position. This was implemented separately for stepper and servo.
• The servo had no issues being smooth and fast, but had accuracy issues as it was not perfectly linear. Characterization data was created where the duty cycles were swept across the entire range of motion and the physical output arm angle was recorded. This resulted in a slightly cubic relationship and allows our calibrated servo to be accurate of around ± 1.5 degrees.
• The stepper faced no accuracy issues as it is inherently designed to “step” at constant intervals. However, they faced issues of step skipping at higher torque. In order to decrease torque to retain step accuracy and smoothness without sacrificing speed, acceleration profiles were used where the stepper will try to linearly accelerate to the desired speed at some fixed acceleration(and deceleration). This method allows for extremely smooth operation while retaining speed. Tests successfully rotated the platform at 300rpm/sec acceleration and 150rpm max speeds, well above our initial goal of an 60rpm/1rps average.

SRS 02 - Thermal imaging data shall be acquired repeatedly as the turret rotates (azimuth).

• Not achieved.
• I2C communication with MLX90640 thermal camera (32x24 thermopile array) established successfully. Calibration parameters were obtained from camera EEPROM, status registers were configured successfully, and both frame constants and updating frame data was read from camera RAM. scan() was written and tested to read in values for the entire frame for each time the turret would stop to take and process a thermal image in scanning mode. align() was written for similar process in the correction term alignment stage of moving the turret to point at a detected thermal hotspot.
• At some point before the final demo, the camera’s thermopile array was damaged (when we first recieved the camera we tested it with an arduino library, which gave very reasonable temperature values and from which we were able to see hand shapes by printing to Serial monitor, later testing with the same library and circuit gave 0˚C reading directly next to pixels reading >300˚C with no in-between values anywhere in the frame). This damage likely occured because we purchased the version without a breakout board to save $20 of budget during our first order. In hindsight, spending the extra money on the breakout board for built-in protections was probably the better choice for our first time working with such a sensitive component.
• In the course of the project, we discovered that this camera requires more resources than the ATmega328PB has available. The first wall was storing the raw data of a frame, as the 768 pixels at 2 bytes of data per pixel would require over 1.5K bytes on the stack, nearly the entirety of the approx 1.7Kb stack on the ATmega’s 2K SRAM. This issues was solved by decreasing the number of pixels read, stored and processed at the same time. This increased available resources by using the same smaller array over and over to store values for data processing, as our proccessing was fairly simple, just checking if temperature exceeded a threshold and storing the cooresponding pixel as the return position error value. The data structure and processing in the align() function was slightly more complex, but the same idea was used to minimize resource usage. Additionally, pointers and pass by reference was used heavily so unnecessary copies of variables and data structures did not eat up stack space, particularly where multiple functions would be called inside on another.
• Similarly, the camera’s EEPROM stored over 800 2 byte values from which many 3 to 8 bit calibration parameters needed to be recovered. In the manufaturer’s API, these values were read and processed on runtime, but this was impossible on our ATmega, as we simply did not have enough space to store these values on runtime. To get around this, all of the EEPROM values were read without being stored while connected to a logic analyser. From the logic analyser software,A .csv file of the session was pulled and parsed with python scripts to obtain the desired values and there corresponding addresses in a more useful structure. Based on the datasheet’s information on the paramters, their addresses, and the math necessary to recover them, a mix of bit manipulation and math in c and manual conversion techniques were used to recover all necessary values. These values were then included as macros or in constant arrays in our MLX90640.c, as we had much more than sufficient flash memory to include the macro values on compile time and put the sturcuted values in read only memory. Having values in arrays and 2D arrays allowed row, column, and pixel specific parameters to be easily accessed by simply knowing which column and row coorespond to the current pixel (say in a nested for loops for sequentially reading and calibrating temperature readings).
• Ultimately, the necessity for going to many significant decimal points in the math for calibration parameters and recovering temperature data was an insurmountable roadblock on our ATmega. The attempt to use int32_t integer math with *1000 /1000 to retain some decimal places was unsuccessful due to overflowing before reaching close to the number of decimal points such that calculations would provide sufficiently accurate temperature results. Simply using float math on our ATmega was neither sufficient nor reasonalbe as the software simulated float math can take thousands of clock cycles to perform float divisions, and there would be many thoughsands of floating point divisions necessary in the recovery of parameters and calibration calculations for each frame of data. This would be far too limiting on the operating capability of the turret.
• Using a ARM cortex M4 on the itsy bitsy M4 was briefly considered, as by reading the datasheet it seemed possible to implement a new I2C driver and reuse the already nearly complete MLX90640 library with much better performance as the clock speed would be 8x faster, I2C FM+ mode would be possible, additional SRAM was available, and an FPU would make float math viable, however this was not completed as the damage to the camera was discovered at about the same time as this idea.

SRS 03 – In seeking/pointing state, the control logic shall account projectile motion so that the water lands on the flame.

• Deemed unnecessary
• Updated pump + nozzle shoots water at a near straight path, this simplifies aiming and calculation, eliminating the need for the distance sensor entirely. When running some preliminary tests, the water jet was able to shoot (accidentally) across the entire length of the of detkin. Further tests showed that at around 4m, the drop due to gravity was only around 15cm. This means that within our designed limits of a 0.5m radius circle directly below, projectile motion is negligible. In fact, for the video demo, targets were shown to be accurately hit at over 2 meters.

SRS 04 – Image processing functionality shall locate pixels above threshold temperature upon each collection of thermal imaging data.

• Not achieved.
• Camera Damage prevented this from being integrated into the final implementation.(See SRS 02 for more information on the damage and work with the thermal camera). Code was written for this functionality, but was not properly tested due to the ATmega328PB limitations and damaged camera.

SRS 05 – The system shall switch correctly between sensing state, seeking/pointing state, and firing state.

• Not achieved.
• Though main() turret control pseudocode was written based on functions from our custom camera and motor libraries, this was not present in our final result due to the damaged camera.

SRS 06 – The system shall follow timing requirements by referencing properly scaled timers/counters.

• Deemed unnecessary
• When first writing this SRS during project proposal, We were not very clear on these things. However, as we complete the project, we realize that this SRS made no sense as this will automatically be completed as long as any pwm/counter is implemented correctly. There is no need to take special effort to achieve this.

3.2 Hardware Requirements Specification (HRS) Results

HRS 01 – Project shall be based on ATmega328P microcontroller

• Achieved
• This is automatically completed as this is the definition of the final project of this class.

HRS 02 – A thermal imaging type sensor shall be used for flame detection. The sensor shall detect min temperature range 0 ~ 300˚C (higher temperatures definitely signifies abnormal heating, potentially flame), a frame rate over 15FPS, and a resolution above 16x16.

• Not achieved.
• Due to the thermal camera malfunctioning, this was not able to be achieved. However, even with a fully functioning camera, certain aspects would not be possible due to unforseen limitations in ATmega328PB communication and calculation capabilities. Specifically, I2C limited to FM mode rather than FM+ (1MHz) supported by the sensor and relatively slow F_CPU and lack of an FPU leading to unreasonably long temperature recovery calculations for the 768 pixels (and necessary calibration calculations).
• With our selection of the MLX 90640, the temperature range is 0~300˚C, with a resolution of 24x32. This achieves our HRH requirements. However, while the camera itself has a refresh rate of up to 64 Hz, the above ATmega328PB limitations make the HRS frame rate impossible. We considered implementing an I2C driver on an Itsy Bitsy M4 (ARM Cortex M4 at 120MHz with an FPU), however, damage to the camera’s thermopile array ultimately prevented this from being viable.

HRS 03 – Servo Motors shall be used for turret positioning. They shall be able to handle the load of the barrel + camera.

• Achieved (besides the fact that our camera is currently broken).
• When originally writing this HRS, cheap 9g servos were the main consideration as they were abundant. However, we later acquired high torque servos dedicated for CR car steering. This provided way more torque than we would need, allowing it to easily handle any load applied to the barrel from the wires/tubing/weight etc.

HRS 04 – pump motor shall be used for shooting water. They shall be able to at least fire jets of water within the specified 0.5m operational radius.

• Achieved.
• Pump independently tested and shoots well beyond HRS specs (see final video). With nozzle attachment, it could easily hit approximately 10 meters with arcing, and within 3 meters virtually 0 projectile motion effects. This was mainly due to finding a powerful pump, and combining it with a small nozzle.

HRS 05 – A Buzzer shall be used for sounding an alarm. It shall be internally driven.

• Achieved.
• 4 internally driven buzzers are triggered to sound at intervals similar to a real fire alarm. This means that they are easily driven by simply applying a voltage across them.

HRS 06 (new) – Turret shall be able to continuously rotate, while preventing any leakage/bad electrical contact.

• Achieved.
• This is an added HRS since the issue was not considered at project proposal. However, this is really important for any turret where continuous azimuth rotation is desired. A slip ring and rotary union was implemented to prevent wires and tubes from twisting and allow continuous rotation.

HRS 07 (new) – The turret shall not leak water through the nozzle when in the off state.

• Achieved.
• This is an added HRS also due to an issue discovered after project proposal. Since the water tank is in the ceiling but the nozzle was beneath it, this would cause a syphoning effect after the initial firing, draining all of the water while the pump is off. This was originally solved using a valve. However this create a lot of complexity both mechanically and requiring another pwm control. Later, it was discovered that the pump was reversible. Hence the current improved solution is to use an H bridge to reverse the flow after each firing, pumping the water back past the point of syphoning.

4. Conclusion

This project provoked a significant amount of learning, as running into issues, troubleshooting, and finding solutions not only cemented topics from this course, but also concepts from prior courses and engineering experiences. We learned about mechanical prototyping, electrical prototyping, using datasheets, component selection, product design, bare metal coding, I2C, image processing, python scripting, servo motors, stepper motors, dealing with the limitations of an MCU, dealing with unnexpected mechanical and electrical issues and failures, using github, and more.

While we would have liked the final result of this project to include a functional thermal camera implementation, the learning process from all of the issues involved with the camera spurred significant improvement in understanding and implementing efficient code for embedded contexts. Similar learning took place through all of the issues we faced through our iterative design and testing processes.

We are certainly proud of coming from essentially zero experience in embedded systems at the beginning of the project to having a well made prototye (with successful non-camera functionality) and a much greater understanding of embedded systems. Despite falling a bit short on our naive intentions for the final outcome of this project, we both feel quite confident in our ability to complete future projects from start to finish, beginning with selecting compatible components and MCUs.

For this project, we are plannning on purchasing a new thermal camera so we can update the turrent with the intended autonomous functionality. Additionally, there is a chance we will take this project idea and update it for our ESE 5160 project next semester.

Final Demo (Writeup)

1. Three class topics you covered

• PWM: Both Servo and Stepper motors used different PWM implementation for precision turret positioning.
• Interrupts: Pin change interrupts on buttons for showing state changes. Timer Output Compare ISR with external clock for step counting and acceleration profile.
• Power Management: Wall supply (AC-12VDC), 2 Buck-Boost for separate 5V logic
• I2C driver: Implemented capable of communicating with 2 byte memory address and data thermal camera. Functions available enable communication with memory addresses and data of arbitrary number of bytes as well as I2C autoincrement (except the camera does not support this) (NOTE: this is not integrated in final demo due to camera issues, but learning from this discussed heavily)
• Image Processing: processing logic implemented in MLX90640.c library file for scanning and alignment modes to find error from current orientation to hotest pixel above threshold (not integrated into demo due to camera issues, but learning from this discussed heavily)
• ADC: attempted as hail mary joystick control for demo as alternative to camera after discovering camera damage - unsuccessful within time constraints.

2. Input device

• Camera (Note: damaged so not incorporated in live demo but work with/ learning from device shown and discussed)
• Buttons for toggling operational states

3. Output Device

• Stepper motor for azimouth control
• Servo motor for elevation control
• Pump for water shoot and preventing siphoning
• Buzzers to sound fire alarm

4. Complexity (what was difficult in your project and on what things did you spend a large amount of time?)

• Everything ranged from fairly to very challenging due to our relative incoming inexperience, unexpected intricacies and design contraints, and our design and implementation philosophy being to go as robustly and thoroughly as possible.

5. Integration (what things broke when you put them together and how did you deal with it?)

• Software
  • Limited hardware resources in memory and computation power spurred the necessity of extreme (and proper) optimization in our approach to memory management and stack usage.
    • python scripting and c preprocessing to store 1700+ bytes of calibration data in flash instead of runtime
    • Pointers and pass by reference
    • Reading frame data chunck by chunck to minimize size of data storage arrays
    • Use of separate .c and .h files for different functionalities
    • etc.
• Hardware
  • Mechanical Design of a turret with continuous rotation (slip ring and rotary union necessary and hard to source and design around)
  • Unexpected interactions between electrical components (voltage spikes damaging other components, shorts etc.)
  • High power consumption ~ 60 Watts
    • Power brick instead of batteries
    • Need to use components rated to hand higher power in power management
• Overall: Budget contraints relative to project ambition (we sourced many components from detkin for “free” and quality, as well as approval for $50 over budget and personal $15 budget, and designing and implementing our own components (H-Bridge)). We may not have known what to do with the budget at the start, but in hindsight, it would have been very helpful for mitigating issues due to unsuccessful or damaged components. General improvement of quality of life.

MVP Demo

Show a system block diagram & explain the hardware implementation. the main actuators are the stepper for azimuth control and the servo for elevation control. slip ring and rotary union are used to ensure 360 degree continuous rotation. pump used to shoot water, connected to an H-Bridge. This H-bridge is crucial since the pump normally experience siphoning, by reversing the flow shortly, this issue is bypassed. buzzers used to sound an alarm

Explain your firmware implementation, including application logic and critical drivers you’ve written. main logic is a loop that scans the area by rotating the turret around 360 degrees, stopping every 30 degrees to scan for a heat signature. Once heat signature is acquired, the motors will work together to aim at target roughly. After this, images are rapidly taken and processed to facilitate precise aiming of the actual target. At this point, the pump would be activated and the flame will be (hopefully) extinguished.

the main drivers include: I2C for camera-atmega communication Servo library that automatically set PWM dutycycle based on desired angle Stepper library that automatically accelerates and decelerate the stepper according to the set target steps, or free spinning continuously at some set speed.

Have you achieved some or all of your Software Requirements Specification (SRS)? Show how you collected data and the outcomes.

SRS 01 – Motor control functionality shall be implemented to enable smooth, fast and accurate motion over the complete range of turret motion.

• Achieved. Servo motors automatically go to the position with a PWM input. Stepper has an upper speed limit and acceleration and deceleration limits. These are all implemented to ensure no skipping steps.

SRS 02 – Thermal imaging data shall be acquired repeatedly as the turret rotates (azimuth).

• In progress. I2C communication established, sequential pixel aquisition and image processing taking more MCU resources and implementation time than initially expected. Sensor math (calibration and convertion to temperature) is in progress and mostly mapped out in skeleton code.

SRS 03 – In seeking/pointing state, the control logic shall account projectile motion so that the water lands on the flame.

• Deemed uneccessary as updated pump + nozzle shoots water at a near straight path, this simplifies aiming and calculation, eleminating the need for the distance sensor entirely.

SRS 04 – Image processing functionality shall locate pixels above threshold temperature upon each collection of thermal imaging data.

• In progress.

SRS 05 – The system shall switch correctly between sensing state, seeking/pointing state, and firing state.

• Planned but requires completion of Image processing for final integration/testing.

SRS 06 – The system shall follow timing requirements by referencing properly scaled timers/counters.

• Achieved/In progress. Successful use of PWM hardware and I2C hardware for timing control thus far. Timer/counter will be used for buzzer, servo PWM, and Stepper stepping.

Have you achieved some or all of your Hardware Requirements Specification (HRS)? Show how you collected data and the outcomes.

HRS 01 – Project shall be based on ATmega328P microcontroller

• Achieved

HRS 02 – A thermal imaging type sensor shall be used for flame detection. The sensor shall detect min temperature range 0 ~ 100C (higher temperatures definitely signifies abnormal heating, potentially flame), A frame rate over 15FPS, and a resolution above 16*16.

• Software Implementation In Progress: resolution spec successful, frame rate spec may be achievabe in tracking mode, but not in searching mode. Estimates made from Logic Analyzer I2C communication rates.

HRS 03 – Servo Motors shall be used for turret positioning. They shall be able to handle the load of the barrel + camera.

• Achieved. Control code implementented and tested with mechanical prototyping under conditions similar to expected, stepper was found to be too weak in prototyping and was swapped out for a larger one. The servo specifically needed characterization (not perfectly linear), this was done by fitting a cubic curve to collected angle and pwm datapoints.

HRS 04 – pump motor shall be used for shooting water. They shall be able to at least fire jets of water within the specified 0.5m operational radius.

• Achieved. Pump independantly tested and shoots well beyond HRS specs (approximately 5 meters +).

HRS 05 – buzzer shall be used for sounding an alarm. It shall be internally driven.

• Achieved. Buzzers aquired and “fire alarm” pattern determined and tested, uses an inernal timer to toggle on/off.

Show off the remaining elements that will make your project whole: mechanical casework, supporting graphical user interface (GUI), web portal, etc.

[Shown during actual demo]

Demo your device.

What is the riskiest part remaining of your project?

Integration, where things just magically break. Currently all of our sub parts can function on its own, but when working together, things may go wrong.

How do you plan to de-risk this?

Test small steps of integration at a time, and also be patient and expect to spend a lot of time debugging.

What questions or help do you need from the teaching team?

moral support and maybe math help on the camera calculations (there is a LOT of math)

Sprint review #2

Current state of project

We tested all electrical components and the mechanical parts that have arrived. We completed third set of orders for power supply, better rotary union, and PU tubes with bonus $50 budget. With our own money we purchased a slew ring as well. Completed a fair ammount of code progress and have begun structural and electrical integration. Prototyping done on OnShape for CAD and using RPL for laser cutting.

Last week’s progress

• Budget increased by $50 for final orders
  • Power supply 12V 5A
  • Improved rotary union
  • PU tubes
  • Slew ring (with personal budget)
• Completed Stepper Bare Metal implementation (speed control)
  • Calculated propper current limitation to prevent overheating (were way overcurrenting initially)
• Recharacterized Servo Motor in Bare Metal (positional control ±2˚)
• Began I2C library
  • Obtained Logic Analyser
  • Discovered VL53L1X hides register information so implementing custom driver is not achievable in the timeframe of the project
    • Will just include for calibration as an ungraded add-on using an arduino circuit + UART in the setup of the Turret main()
  • I2C still used for thermal camera
• First iterations of CAD using OnShape
  • OnShape View Link
• Lasercutting in RPL
  • See CAD section for .dwg files
  • See Images section for lasercut pictures
  • Figured out necessary kerf values for improved gear meshing
• Slew Ring
  • Arrived and tested
  • Integrated internal gearing using press fit
• Slip Ring
  • Connections work for all channels
  • PWM tested accross every channel with no noticable slewing
• Valve with push fit connections does not meet performance requirements
  • Will use a different valve from Detkin controlled by a second servo
• Thermal Camera Functionality tested with Arduino
• Found and Tested improved Water Pump
• Completed Power regulation circuit design
  • Tested on breadboard: functional given lower currents
  • Use barreljacks and power supply rather than batteries as initially planned
    • Constant voltage and current is desired
    • Real world implementation would have access to building water and electrical lines (battery use would not make sense)
• Soldered the button with inbuilt LED to match desired functionality
• Read a bunch of Datasheets
• Burnt and aquired new ATmega328PB xplained minis (yes… multiple)

Next week’s plan

Note: All work expected to be done while in-person together (basically all free time). Specific asyncronous tasks split as sensible (research and datasheet reading).

Code:

• Test I2C library on Logic Analyser
• Code Camera Driver for I2C
• Bare Metal C buzzer implementation (CTC and ISR in main.c file)
• main.c file
  • Integration
  • Controls

Electrical Integration:

• Power circuit on a perf board
  • Appropriate current considerations
  • Screw Terminals for connection to other boards in future

Mechanical Integration:

• Structural integragtion of servo, stepper, slip ring mounts on turret base
  • CAD in OnShape
  • Lasercut at RPL
• Iteratively test lasercut (i.e. kerf)

Component Testing:

• Test arriving components

Sprint review #1

Current state of project

We have sourced all electrical components and most mechanical parts. All sensors and actuators have had functionality tested, aside from the camera that just arrived. We completed second order for a necessary mechanical component (for the water tubing), and have written starting code for working with our servos in bare metal C.

Last week’s progress

Planned progress:

• Stepper motor and driver: speed, torque, and current draw acceptable, potential thermal concerns (to be verified with more testing)
• Time of Flight: distance (20cm to 2m) and accuracy very good (sub 1cm accurate with ambient light). Field of View servicable.
• Power regulators: Sparkfun regulator tested with 5V and 3.3V output, servicable. Still need to test 12V regulator
• CAD progress minimal as dependant on other aspects of mechanical design and component sourcing that was still in progress over this week.
• Bare Metal C code written for controlling Servo motor with 16 bit timer PWM to .1 degree resolution from 0 to 180 degrees. Needs to be turned into a .c and .h file library.
• Bare Metal pseudocode written for Stepper motor. Needs to be implemented and tested in Bare Metal C and put as library
• No progress on I2C currently
• Switching regulator works at 3.3V and linear regulators for 3.3V found in Detkin. Need to test with camera.
• Read a lot of datasheets, particularly for mechanical parts and electrical components while testing.

Unplanned progress:

• Home Depot run for swivel joint unsuccesful (online description misleading)
• Parts Order 2: potentially functional swivel joint with ball valve
• Discovered siphoning issue with planned tubing configuration, solution using ball valve as mentioned in parts order
• Gear ratio and configuration determined for rotating base
• Parts Order 1 arrived
• Potential math optimization for thermal image calibration process: apply reverse calibration to convert threshold value to raw threshold value instead of calibrating all 768 pixels raw data before processing. This may be redundant with internal sensor calculations TBD.
• Determined buzzer pattern similar to a typical fire alarm for maximal annoyingness when fire detected

Next week’s plan

Note: All work expected to be done while in-person together (basically all day on weekends and weekday afternoons when no immediately due assignemnts). Specific asyncronous tasks split as sensible.

• Potential budget request for PU (Polyurethane) tubing to meet ordered swivel joint specs

Code:

• Convert Bare Metal Servo code into library
• Stepper Bare Metal implementation and library
• Begin I2C (camera and ToF)

Mechanical:

• CAD and Lasercut Version1 gears
• (Potential) CAD servo, stepper, slip ring mounts (mechanical connectors)

Component Testing:

• Thermal Camera: data collection
• Slip Ring: electrical connections and current

Sprint review trial

Current state of project

We are currently planning, testing and sourcing components.

Last week’s progress

Read a lot of datasheets and determined component viability.

Scavenged and tested many parts from Detkin:

• MG996R Servo: Elevation control
  • Also tested other servos, this was the highest quality
  • Characterized PWM controls for 0-180 degree range
• NEMA 11 Stepper Motor: Azimuth control
  • Smallest form factor offered, planning to test if this is serviceable
• A4988 Stepper Motor Driver
  • Widely used and meets specs
• 12V submersible pump
  • Tested flow rate, deemed sufficient
• Several potential power regulation boards
• Multiple Time of Flight distance sensor options: for ranging applications
• Internally Driven Buzzers 4kHz 12V
  • Loud and annoying as intended
• Various Switches
• Backup 8x8 Thermal Imaging Camera
• Lazer pointer for testing

Submitted First order form:

• MLX90640 32x24 Thermal Imaging Camera
• Slip Ring
  • rated for 6 channels 10 A
  • with center hole for water tubing

Planned Future orders:

• Swivel union for tubing (May need Home Depot run)
• Kiddy Pool?
  • No flood detkin
• Battery Pack Holder

Planned Mechanical Design

• Lasercutting structural components
• 3D print outer shell

Next week’s plan

Testing Components:

• Stepper motor with motor driver (speed and torque)
• Time of Flight Sensors (distance / accuracy / Field of View)
• Power regulators

First Iteration Turret CAD

• Azimouth and Pitch joints
• Tubing Nozzle: faster water shoot better

Bare Metal Code:

• Servo control as a library (.c and .h files)
• Stepper control as a library (.c and .h files)
• I2C library
  • Begin driver for ToF sensor
  • 8x8 backup camera

Find a 5V to 3.3V linear regulator for camera (scavenging from Detkin as usual).

More datasheet reading (Huzzah!!!)

Final Project Proposal

1. Abstract

This project involves the development of an autonomous sentry turret equipped with thermal imaging sensors to detect and respond to fire outbreaks. The turret uses its thermal sensor to identify heat signatures above a certain threshold, deploying a precise waterjet to suppress the flames. This solution aims to augment existing fire suppression systems such as sprinklers to provide a more rapid and targeted response to fires at the earliest stage in areas prone to fire hazards, preventing unnecessary property damage and injuries.

2. Motivation

Fires can happen at any time, anywhere. Having a system to respond to this uncertain situation is important. Our turret would allow us to target any flame within range of operation.

Response time is key to defeating fires. As the fire grows, it becomes more difficult to extinguish. Hence, our system using thermal imaging offers a nearly instantaneous detection and reaction to flames, increasing the success rate of extinguishing.

Current systems such as sprinklers are a last resort tool, usually triggering after significant damage. Our system would detect and extinguish the first hint of fires, before it causes significant damage, if any.

Fire department response times may greatly, leaving us stranded without means of defeating the flame. Our system however aims to act as an autonomous firefighter to contain/extinguish the fire as soon as possible.

3. Goals

Interpreted as checkpoint goals to guide progress.

Complete an I2C library for collecting data from the thermal imaging sensor.

Turret aims at flame based on thermal camera data

Water jet can be shot far enough (0.5m)

Design a neat mechanical enclosure to conceal the device hardware (Limit exposed wires).

Use perf-boards for final electrical implementations ensure electrical connections.

By the end of the final project, a singular elevated turret should be able to detect and extinguish a small candle flame lit anywhere in a 0.5m radius directly below without any human intervention.

4. System Block Diagram

5. Design Sketches

6. Software Requirements Specification (SRS)

6.1 Overview

Control turret motors to repeatedly collect thermal imaging data until heat source is detected. Initiate alarms and control the turret motors to point the water cannon at the heat source and activate the water pump until the heat source is eliminated, then return to scanning mode.

6.2 Users

People who own / run spaces with elevated fire risk and expensive equipment (e.g. Lab Spaces, Warehouses, etc.)

6.3 Definitions, Abbreviations

Currently N/A, may update during implemention process

6.4 Functionality

SRS 01 – Motor control functionality shall be implemented to enable smooth, fast and accurate motion over the complete range of turret motion.

SRS 02 – Thermal imaging data shall be acquired repeatedly as the turret rotates (azimuth).

SRS 03 – In seeking/pointing state, the control logic shall account projectile motion so that the water lands on the flame.

SRS 04 – Image processing functionality shall locate pixels above threshold temperature upon each collection of thermal imaging data.

SRS 05 – The system shall switch correctly between sensing state, seeking/pointing state, and firing state.

SRS 06 – The system shall follow timing requirements by referencing properly scaled timers/counters.

7. Hardware Requirements Specification (HRS)

7.1 Overview

A 2 axis turret controlled using servo motors that would aim a barrel and thermal camera at a flame. A pump would shoot water through the barrel at the targeted flame, while sounding a buzzer.

7.2 Definitions, Abbreviations

Elevation: the angle of the turret’s pithing movement (up and down)

Azimuth: the angle of the turret’s yawing movement (side to side)

7.3 Functionality

HRS 01 – Project shall be based on ATmega328P microcontroller

HRS 02 – A thermal imaging type sensor shall be used for flame detection. The sensor shall detect min temperature range 0 ~ 100C (higher temperatures definitely signifies abnormal heating, potentially flame), A frame rate over 15FPS, and a resolution above 16*16.

HRS 03 – Servo Motors shall be used for turret positioning. They shall be able to handle the load of the barrel + camera.

HRS 04 – pump motor shall be used for shooting water. They shall be able to at least fire jets of water within the specified 0.5m operational radius.

HRS 05 – buzzer shall be used for sounding an alarm. It shall be internally driven.

8. Components

The most important component is the thermal imaging sensor. This is what allows the flame to be detected and located. All other hardware such as motors, buzzers, pump… have less strict requirements, potentially even available in Detkin. Passive components and MOSFETs will be acquired from Detkin.

Thermal Imaging Sensor (Thermopile or Thermal Camera)

Frontrunner (Thermal Camera): MLX90640

• https://www.digikey.com/en/products/detail/melexis-technologies-nv/MLX90640ESF-BAB-000-TU/8638464

While these thermal sensor types are both more expensive than a simple IR flame sensor, both these options collect many data points across the sensor field of view, allowing the precise location of the flame to be acquired (critical functionality).

The links and information for other components can be seen in the Final Project BOM.xlsx

BOM LINK

9. Final Demo

For the final demonstration, we shall exhibit an instance of the elevated turret detecting and firing at a hot object introduced at an arbitrary time and location (within the design specification of 0.5m radius region at 1m height).

Due to restrictions on open flames for testing, we will be using a hot object to simulate the flames. One possibility is a cup of hot water, as shooting additional water into the cup with decrease the temperature of the water, “dousing” the simulated “flame”.

10. Methodology

To avoid being stuck, we plan to approach this problem in small chunks, where each chunk is unaffected by its connecting chunks. I.e. We would work on just the turret control and get it fully functional before using the sensor to feed it data. This would allow us to break down the whole project into smaller, easier to debug steps. This would also be really good if components arrive at different times, so our overall pace is not affected.

Furthermore, we plan to ensure our implementation is future proofed (as best we can), so that if we ever decide to add features/upgrades, the base implementation would need no/minimal change.

11. Evaluation

Test condition: A singular elevated turret at ~1m high with a small flame lit within a 0.5m radius (1m diameter) circle centered directly below.

Minimum success metric: Flames shall be detected and extinguished within 15 seconds for >=50% of instances. 20 seconds for >=90% of instances.

12. Proposal Presentation

Slides have been added to the Final Project Proposal slide deck in the Google Drive.

References

Fill in your references here as you work on your proposal and final submission. Describe any libraries used here.

• No libraries used. Reference Datasheets folder for component information.

Github Repo Submission Resources

You can remove this section if you don’t need these references.

• ESE5160 Example Repo Submission
• Markdown Guide: Basic Syntax
• Adobe free video to gif converter
• Curated list of example READMEs
• VS Code is heavily recommended to develop code and handle Git commits
  • Code formatting and extension recommendation files come with this repository.
  • Ctrl+Shift+V will render the README.md (maybe not the images though)