All Resources

Advancing Camera Control with Edge AI at ICCAR 2025

Sina Ehsani, Sr AI Engineer

I recently had the incredible opportunity to represent Armada AI at the IEEE International Conference on Automation and Robotics (ICCAR) 2025 in the beautiful city of Kyoto, Japan. I was there to present our latest paper, "Camera Control at the Edge with Language Models for Scene Understanding," and the response to our Optimized Prompt-based Unified System (OPUS) was truly energizing. This research was spearheaded by the brilliant work of Alexiy Buynitsky from Purdue University during his internship with us at Armada. It was a thoroughly collaborative effort, and I was proud to develop it alongside Alexiy and my dedicated colleagues, Bhanu Pallakonda and Pragyana Mishra.

The Challenge: Today's robotic cameras need to intelligently understand and react to dynamic real-world environments. However, deploying powerful AI like Large Language Models (LLMs) efficiently on local "edge" hardware, while keeping them user-friendly, has been a significant hurdle.

Armada's Solution: The OPUS framework is our answer. It’s a system we developed that allows LLMs to control Pan-Tilt-Zoom (PTZ) cameras using natural, everyday language. Imagine telling a camera what to look for, and it just understands – that’s what we aimed for and achieved with OPUS.

Figure 1. OPUS makes complex camera control intuitive. As shown, a high-level user command is all it takes for the system to identify the 'Gray Car,' then tilt to the 'License Plate,' and zoom in accordingly, a task that would typically require multiple, precise manual inputs.



Key Advantages of OPUS:

  • Edge-Ready Efficiency: Designed for high performance on local devices, reducing cloud reliance and enabling faster responses.
  • Intuitive Natural Language Control: Users interact with camera systems using everyday language, no complex programming needed.
  • Superior Performance: OPUS significantly outperforms traditional methods, with our fine-tuned models surpassing even much larger ones, such as GPT-4 and Gemini Pro.
  • Reliable and Precise: Ensures commands are accurately executed via a predefined API, minimizing errors.

Figure 2. Presenting OPUS at Kyoto - ICCAR 25 conference.

Read the Full Paper: Dive deeper into the technical details of OPUS by reading our full paper, available on arXiv.