Search robots for house fires

Know the house before you walk in.

Robot dogs map a burning house, find the people inside, and tell the crew where to go.

Choose a missionHow it's built

Pick a house. Send in the dogs.

CinderEmber

LiveThree-bed ranch
Through the smoke

Colour sees smoke. Heat sees people.

Hallway seen by the colour camera
The same hallway seen by the thermal camera
ColourHeat
Same hallway, two cameras. Scroll or drag.
Top view
It sweeps the room. A body-warm shape gets flagged.
Thermal frame with a person-sized warm shape on the floorReal OpenCV 5 output
OpenCV traces the warm shape. That outline is what the dog reports.
Team

Every dog maps. The radio shares it.

Each dog's map0.0 s
  • Dog A
  • Dog B
  • Dog C
Lead dog

Simplified model.

2 sbetween radio messages
12 mradio range
1 wallit gets through

How they walk in.

Two dogs, no driver. Each one picks the next room on its own.

00 rooms
Visual SLAM

Two robots, one map.

Robot BRobot A24 ORB matches
BAagree
72%walls agree
2.39°heading error
Results

120 runs. 30 houses.

Every run is scored against the true house, which the robots never get to read. The runs use our headless simulator and the same robot brain, with the radio on and off, and with one robot or two.

580 of 600people found (97%)
96%of rooms entered997 of 1,036
95%of the floor searched, median run
715 of 721confirmations correct (99%)
One full live run in our Unity simulator, shown at 3× speed. The robots found all 5 people with no wrong confirmations, then held in the hallway when the fire closed the only exit. They never drive through flames.
How it's built

OpenCV 5 sees. An agent decides. AWS runs it.

On every dog
01

Sense

Lidar, depth and heat cameras in our Unity simulator.

02

Seewith OpenCV 5

  • RGB-D odometrycv2.Odometry
  • Warm-shape contoursfindContours
  • Thermal colourapplyColorMap
  • Keypoints for SLAMORB
03

Decide

A language-model agent picks the next room or the next look.

04

Act and share

Reflexes back off from heat at once. It can spray water on a small fire. The radio merges maps.

What it sees changes what it does next

On AWS
AWS AmplifyHosts this site
AWS Lambda on GravitonOpenCV 5 vision API, arm64
Amazon ECS on FargateThe robots' agent service, arm64