Adaptive Edge-Cloud CV: When Milliseconds Matter

At a 24-hour hackathon, we asked: what if your computer vision model could dynamically choose where to run - on device, edge, or cloud - based on real-time conditions? We built it, and won.

The Vision

Traditional CV pipelines are rigid: - Edge-only: Limited by device compute - Cloud-only: Network latency kills real-time - Hybrid: Manual configuration, not adaptive

We built an intelligent system that makes millisecond decisions about where to process each frame.

Architecture

Camera → Preprocessor → Scheduler → [Device | Edge | Cloud]
                          ↑
                     Latency Monitor
                     Battery Monitor
                     Network Monitor

Key Innovation: Adaptive Scheduling

class AdaptiveScheduler:
    def __init__(self):
        self.device_latency = []
        self.cloud_latency = []
        self.battery_level = 100
        self.network_quality = 'excellent'

    def decide_backend(self, frame, model_complexity):
        """Intelligent decision on where to run inference"""

        # Critical: Always use device for sub-50ms tasks
        if model_complexity < 0.3:
            return 'device'

        # Battery conservation mode
        if self.battery_level < 20:
            return 'cloud'

        # Network-aware decision
        if self.network_quality == 'poor':
            return 'device'  # Don't waste time on network

        # Latency-based decision
        device_latency = np.mean(self.device_latency[-10:])
        cloud_latency = np.mean(self.cloud_latency[-10:])

        if device_latency < cloud_latency * 0.8:
            return 'device'
        else:
            return 'cloud'

Snapdragon AI Integration

Used Qualcomm Hexagon DSP for on-device acceleration:

import snpe  # Snapdragon Neural Processing Engine

def setup_snapdragon_model(model_path):
    """Load model optimized for Hexagon DSP"""
    runtime = snpe.NeuralNetwork(model_path)
    runtime.setRuntimeProcessor(snpe.RuntimeProcessor.DSP)
    return runtime

def run_on_device(frame, runtime):
    """Ultra-fast inference on Hexagon DSP"""
    # Preprocess
    input_tensor = preprocess_frame(frame)

    # Run on DSP (< 15ms for MobileNet)
    output = runtime.execute(input_tensor)

    # Postprocess
    results = decode_output(output)
    return results

Results: 8-15ms on Snapdragon 865 vs 40-60ms on CPU.

Multi-Device Pipeline

Distribute workload across phone, tablet, edge device:

class MultiDeviceOrchestrator:
    def __init__(self, devices):
        self.devices = devices  # [phone, tablet, edge_box]
        self.task_queue = queue.Queue()

    def distribute_workload(self, video_stream):
        """Split video processing across devices"""
        frame_count = 0

        for frame in video_stream:
            # Round-robin distribution
            device = self.devices[frame_count % len(self.devices)]

            # Send frame to device
            self.task_queue.put({
                'device': device,
                'frame': frame,
                'timestamp': time.time()
            })

            frame_count += 1

    def collect_results(self):
        """Aggregate results from all devices"""
        results = []
        while not self.task_queue.empty():
            task = self.task_queue.get()
            result = task['device'].process(task['frame'])
            results.append(result)
        return results

Dynamic Model Selection

Switch between models based on context:

models = {
    'tiny': MobileNetV2(width=0.35),    # 5ms, 65% accuracy
    'small': MobileNetV2(width=0.75),   # 12ms, 75% accuracy
    'medium': EfficientNet-B0,          # 25ms, 82% accuracy
    'large': EfficientNet-B3,           # 80ms, 88% accuracy
}

def select_model(priority, battery, network):
    if priority == 'latency':
        return models['tiny']
    elif priority == 'accuracy':
        return models['large']
    elif battery < 30:
        return models['tiny']  # Save battery
    elif network == 'poor':
        return models['small']  # On-device
    else:
        return models['medium']  # Balanced

Results

Latency Comparison:

Battery Impact:

• Traditional cloud: 18% battery/hour
• Our system: 12% battery/hour
• 33% improvement in battery life

Real-time Performance:

• Processes 30 FPS consistently
• Maintains < 50ms latency 95% of the time
• Gracefully degrades under poor conditions

Hackathon Demo

Live demo detecting objects in 30 FPS video: - Device mode: 8ms latency, 70% accuracy - Cloud mode: 45ms latency, 88% accuracy - Adaptive: Automatically switches based on battery/network - Multi-device: Distributes across 3 devices, processes 90 FPS

Judges were impressed by real-time adaptation visualization.

Technical Challenges

1. Synchronization: Keep multi-device results aligned
2. Model conversion: Optimize for Hexagon DSP
3. Network prediction: Forecast quality before sending
4. Battery profiling: Different models, different drain rates

Impact & Learnings

Why it matters: - AR/VR applications need < 20ms latency - IoT devices have limited compute - Network is unreliable - Battery life is precious

Key insights: 1. Hardware acceleration is crucial: DSP is 3-5x faster than CPU 2. Monitoring enables intelligence: Can't adapt without metrics 3. Fallback is mandatory: System must work offline 4. Trade-offs are application-specific: No one-size-fits-all

This project won because it solved a real problem: making CV practical on real devices in real conditions.