Fix: ElizaOS VisionService Fails to Initialize (ENABLE_OBJECT_DETECTION Boolean Bug)

In the ElizaOS autonomous agent framework (formerly ai16z Eliza), the @elizaos/plugin-vision package empowers autonomous AI agents with image understanding, real-time webcam processing, object detection (YOLO / MobileNet), and human pose tracking.

However, developers configuring multimodal vision agents frequently discover that despite explicitly configuring settings in character.json or .env, the agent’s vision capabilities remain completely dormant:

[VisionService] Initializing VisionService...
[VisionService] Object detection: DISABLED
[VisionService] Pose detection:   DISABLED
[VisionService] Face recognition: ENABLED
[AgentRuntime] Warning: Requested visual inspection action skipped — no active detection pipeline.

If your agent swarms are also facing database deadlocks during parallel task execution, check our diagnosis on Fixing ElizaOS SQLITE_BUSY: Database is Locked in Parallel Agent Swarms.

Technical Forensics: The Defect in @elizaos/plugin-vision

This defect (cataloged in ElizaOS GitHub Issue #31000) stems from inconsistent type coercion in plugins/plugin-vision/src/service.ts.

In ElizaOS, IAgentRuntime.getSetting(key: string) has the TypeScript return signature:

getSetting(key: string): string | boolean | number | null | undefined;

When configuration parameters are loaded from .env, values may be strings. But when loaded from a character configuration file (characters/my-agent.character.json), JSON parsers preserve primitive boolean types (true or false).

Code Discrepancy in VisionService.initConfig()

Examining plugins/plugin-vision/src/service.ts:

// AFFECTED CODE: plugins/plugin-vision/src/service.ts
const config = {
  // BUG: Strict equality against string "true"
  enableObjectDetection:
    runtime.getSetting("ENABLE_OBJECT_DETECTION") === "true" ||
    runtime.getSetting("VISION_ENABLE_OBJECT_DETECTION") === "true",

  enablePoseDetection:
    runtime.getSetting("ENABLE_POSE_DETECTION") === "true" ||
    runtime.getSetting("VISION_ENABLE_POSE_DETECTION") === "true",

  // WORKING: Face recognition uses boolean-aware helper
  enableFaceRecognition: getBooleanSetting(
    runtime,
    "ENABLE_FACE_RECOGNITION",
    "VISION_ENABLE_FACE_RECOGNITION"
  ),
};

When a developer sets:

{
  "settings": {
    "ENABLE_OBJECT_DETECTION": true
  }
}

The runtime returns true (boolean). The comparison evaluates:

$$\text{Boolean}(true) === \text{String}(“true”) \implies \mathbf{false}$$

The config sets enableObjectDetection = false. Consequently, the YOLO/MobileNet weights are never loaded into V8 memory, and any incoming image payloads skip object bounding box analysis.

The Immediate Workaround: String Quoting in Configurations

If you cannot immediately update the npm package, update your character file and .env to enforce literal string values:

In characters/agent.character.json:

{
  "name": "VisionSentinel",
  "settings": {
    "ENABLE_OBJECT_DETECTION": "true",
    "ENABLE_POSE_DETECTION": "true",
    "VISION_ENABLE_OBJECT_DETECTION": "true",
    "VISION_ENABLE_POSE_DETECTION": "true"
  }
}

In .env:

# Explicit string values for ElizaOS vision plugins
ENABLE_OBJECT_DETECTION="true"
ENABLE_POSE_DETECTION="true"

Permanent Solution: Normalizing Boolean Settings Helper

To permanently resolve this across all plugins in your ElizaOS fork or local repository, introduce a resilient boolean coercion utility:

// utils/boolean-settings.ts
import { IAgentRuntime } from '@elizaos/core';

export function parseBooleanSetting(
  runtime: IAgentRuntime,
  ...keys: string[]
): boolean {
  for (const key of keys) {
    const value = runtime.getSetting(key);
    
    // Check if undefined or null
    if (value === undefined || value === null) {
      continue;
    }

    // Direct boolean
    if (typeof value === 'boolean') {
      return value;
    }

    // String normalization
    if (typeof value === 'string') {
      const normalized = value.trim().toLowerCase();
      if (normalized === 'true' || normalized === '1' || normalized === 'yes') {
        return true;
      }
      if (normalized === 'false' || normalized === '0' || normalized === 'no') {
        return false;
      }
    }

    // Number representation
    if (typeof value === 'number') {
      return value === 1;
    }
  }

  return false;
}

Applying the Patch to service.ts

Apply this patch to plugins/plugin-vision/src/service.ts:

--- a/plugins/plugin-vision/src/service.ts
+++ b/plugins/plugin-vision/src/service.ts
@@ -45,12 +45,14 @@ export class VisionService extends Service {
     const config = {
-      enableObjectDetection:
-        runtime.getSetting("ENABLE_OBJECT_DETECTION") === "true" ||
-        runtime.getSetting("VISION_ENABLE_OBJECT_DETECTION") === "true",
-      enablePoseDetection:
-        runtime.getSetting("ENABLE_POSE_DETECTION") === "true" ||
-        runtime.getSetting("VISION_ENABLE_POSE_DETECTION") === "true",
+      enableObjectDetection: parseBooleanSetting(
+        runtime,
+        "ENABLE_OBJECT_DETECTION",
+        "VISION_ENABLE_OBJECT_DETECTION"
+      ),
+      enablePoseDetection: parseBooleanSetting(
+        runtime,
+        "ENABLE_POSE_DETECTION",
+        "VISION_ENABLE_POSE_DETECTION"
+      ),
       enableFaceRecognition: parseBooleanSetting(
         runtime,

Verification Test Script

Run this standalone TypeScript test to verify that your agent runtime correctly instantiates VisionService with both string and boolean configurations:

// test-vision-config.ts
import { parseBooleanSetting } from './utils/boolean-settings';

const mockRuntimeWithBooleans = {
  getSetting: (key: string) => {
    const settings: Record<string, any> = {
      ENABLE_OBJECT_DETECTION: true,       // Native boolean
      ENABLE_POSE_DETECTION: 'true',       // String
      ENABLE_FACE_RECOGNITION: '1',        // Numeric string
      DISABLED_FEATURE: false,
    };
    return settings[key];
  },
} as any;

console.log('--- Vision Settings Coercion Test ---');
console.log('Object Detection (from boolean true):', parseBooleanSetting(mockRuntimeWithBooleans, 'ENABLE_OBJECT_DETECTION'));
console.log('Pose Detection   (from string "true"):', parseBooleanSetting(mockRuntimeWithBooleans, 'ENABLE_POSE_DETECTION'));
console.log('Face Recognition (from string "1"):   ', parseBooleanSetting(mockRuntimeWithBooleans, 'ENABLE_FACE_RECOGNITION'));
console.log('Disabled Feature (from boolean false):', parseBooleanSetting(mockRuntimeWithBooleans, 'DISABLED_FEATURE'));

// Assertions
if (
  parseBooleanSetting(mockRuntimeWithBooleans, 'ENABLE_OBJECT_DETECTION') === true &&
  parseBooleanSetting(mockRuntimeWithBooleans, 'ENABLE_POSE_DETECTION') === true
) {
  console.log('✓ SUCCESS: All vision settings parsed accurately regardless of primitive type.');
} else {
  console.error('✗ FAILURE: Type mismatch detected.');
  process.exit(1);
}

Executing npx tsx test-vision-config.ts confirms that the vision pipeline initializes correctly, ensuring your autonomous agent will process camera feeds and detect scene objects without silent failure.