Vision-based action planning for image-based condition correction

Context

NDA · Robotics / Manufacturing. A visual scene is to be moved step by step into a defined target state through automated actions — with no physical hardware available during development.

Problem

The controller had to derive from images which robot actions reduce the deviation from the target state. Development and testing could not depend on the availability of the robot.

What we built

A vision-based system for controlling automated actions: it analyses the difference between the current image and a reference image and computes a sequence of corrective robot actions that minimises the visual deviation. Developed and tested entirely in simulation.

Key engineering decision

Simulation-first — and image processing for both error quantification and action planning. Because the same image comparison measures how far the scene is from the target and at the same time determines the next action, the solution stays hardware-agnostic: it can be developed without a robot and later integrated flexibly into real systems.

Result

• Rapid development without physical hardware
• Flexible integration in a hardware-agnostic environment
• Image-based state correction as a reusable building block