WebInfer

Node.js Daemon

Run WebInfer inference on desktop (Windows, Linux, macOS) with any browser

What is the Daemon?

The WebInfer daemon is a simple Node.js tool that runs @webinfer/server on your desktop (Windows, Linux, or macOS). It requires Node.js to be installed on your system.

🌐 Works with ALL browsers

The BIG benefit: No browser extension needed! Use WebInfer with Firefox, Safari, Edge, Brave, or any browser. The daemon runs as a local server that any browser can connect to.

Best for:

  • • Using non-Chrome browsers
  • • Local development
  • • Testing without extension
  • • CI/CD pipelines
  • • Quick prototyping

Features:

  • • HTTP and SSE endpoints
  • • Debug mode with detailed logs
  • • Token-based auth
  • • CORS enabled

Quick Start

Run the daemon:
npx webinfer daemon

Starts server at http://localhost:54321

Updating

Get the latest version:
npx webinfer@latest daemon

Using npx, pnpm dlx, or yarn dlx always fetches the latest version automatically.

If you installed globally with npm install -g webinfer, update with npm install -g webinfer@latest

Storage & Preferences

⚠️ Separate Storage

The daemon and browser extension store preferences in different locations. They do not share configuration.

If you switch from the extension to the daemon (or vice versa), you'll need to re-enter your API keys and configure your providers again.

Extension Storage:

Browser's IndexedDB

Daemon Storage:

~/.webinfer/ directory

Configuration

Environment Variables
Variable
Default
Description
PORT54321Server port
HOSTlocalhostServer host
DEBUGfalseEnable debug logs
AUTH_ENABLEDtrueRequire token auth
CLI Options
terminal
# Custom portnpx webinfer daemon --port 3000# Debug mode - see detailed logsnpx webinfer daemon --debug# Disable auth (development only!)npx webinfer daemon --no-auth

💡 Debug Mode for Developers

Use --debug to see detailed logs showing how WebInfer routes and processes your requests:

  • • Provider selection logic and priority order
  • • Request transformation and message formatting
  • • API calls and responses from each provider
  • • Fallback behavior when a provider fails
  • • Storage operations and conversation tracking

Perfect for understanding the full request pipeline and debugging integration issues.

API Endpoints

POST
/api

Execute LLM requests

const response = await fetch('http://localhost:54321/api', {  method: 'POST',  headers: {    'Content-Type': 'application/json',    'Authorization': 'Bearer YOUR_TOKEN'  },  body: JSON.stringify({    prompt: 'Explain TypeScript'  })})const data = await response.json()
POST
/sse

Server-Sent Events for streaming

GET
/health

Health check endpoint

GET
/config/providers

List configured providers

GET
/.well-known/webinfer.json

Discovery manifest for auto-configuration. Returns capabilities, endpoints, and auth requirements.

Sharing Your Daemon

The daemon exposes a discovery endpoint that enables others on your network to connect to it as a provider. Share the URL and they can auto-configure.

GET /.well-known/webinfer.json

Returns a manifest with capabilities, auth requirements, and connection info.

When the daemon starts, it shows:

# Share this daemon with others:

Discovery URL: http://localhost:54321

# Add to another WebInfer client:

CLI: webinfer provider add http://localhost:54321

In-App: Settings → Integrations → Add from URL

⚠️ Network Access

By default, the daemon only listens on localhost. To share over your network, start with --host 0.0.0.0and ensure your firewall allows connections.

Authentication

The daemon generates a token on first startup, saved to ~/.webinfer/daemon.token

terminal
# View your tokencat ~/.webinfer/daemon.token# Use in requestsexport WEBLLM_TOKEN="webinfer-daemon-..."curl -H "Authorization: Bearer $WEBLLM_TOKEN" \  http://localhost:54321/api

Client Integration

The @webinfer/client SDK automatically detects and connects to the daemon:

app.js
import { generateText } from 'webinfer'// Client auto-detects daemon at localhost:54321const result = await generateText({  prompt: 'Hello!'})console.log(result.text)

Python Inference Engine
Optional

For advanced local inference capabilities like audio generation, WebInfer provides a Python-based inference engine. This enables models that require PyTorch/diffusers (like Stable Audio Open) to work with WebInfer.

🔌 Separate & Optional

The Python inference engine runs as a separate process from the Node.js daemon. They communicate over HTTP (localhost). The Node.js daemon works perfectly without Python installed.

Node.js Daemon (port 54321) → HTTP → Python Engine (port 8765)

Stable Audio Open 1.0

Generate up to 47 seconds of high-quality stereo audio at 44.1kHz from text prompts. Perfect for ambient sounds, music, and sound effects.

Requirements:

  • • Python 3.10+
  • • CUDA GPU (recommended) or CPU
  • • ~5GB VRAM for Stable Audio
  • • ~10GB disk space for models

Supported:

  • • NVIDIA CUDA GPUs
  • • Apple Silicon (MPS)
  • • CPU fallback
  • • WAV, MP3, OGG, FLAC output
Installation & Setup

Install the Python inference engine with audio support:

terminal
# Install with audio dependencies (PyTorch, diffusers, etc.)pip install webinfer-inference[audio]# Or install base package onlypip install webinfer-inference
Python Server API
GET
/healthServer status, GPU info, loaded models
GET
/modelsList available models
POST
/audio/generateGenerate audio from prompt
POST
/models/:id/loadPreload a model
POST
/models/:id/unloadUnload model to free memory
OpenAPI docs available at http://127.0.0.1:8765/docs
Environment Variables
Variable
Default
Description
WEBINFER_INFERENCE_HOST127.0.0.1Server host
WEBINFER_INFERENCE_PORT8765Server port
WEBINFER_INFERENCE_DEVICEautocuda, cpu, mps, or auto
WEBINFER_INFERENCE_CACHE_DIR~/.cache/huggingfaceModel cache directory

The legacy WEBLLM_INFERENCE_* names are still read as a fallback for backward compatibility; prefer the WEBINFER_INFERENCE_* names going forward.

Enabling/Disabling Python Inference

The Python inference provider is automatically detected when the Python server is running. To enable or disable it:

To Enable:

  1. Install the Python package: pip install webinfer-inference[audio]
  2. Start the Python server: webinfer-inference serve
  3. The python-inference provider will auto-detect and become available

To Disable:

  1. Simply stop the Python server (Ctrl+C)
  2. Or don't install Python at all—the Node.js daemon works without it
  3. The provider will show as unavailable when the server is not running
Note: No configuration needed in WebInfer settings. The provider availability is determined by whether the Python server is reachable at http://127.0.0.1:8765
Desktop App Packaging (macOS, Windows, Linux)

If you're packaging the daemon into a desktop app (Electron, Tauri, etc.):

✅ Node.js Daemon Only (Recommended)

Package just the Node.js daemon. It works standalone for text/chat with all API providers (OpenAI, Anthropic, Ollama, etc.). No Python required. Keeps package size small (~50MB).

⚠️ With Python Inference

To include Python inference (for audio generation), you have two options:

  • • Bundle Python: Use PyInstaller to create standalone Python executable (~500MB+)
  • • External dependency: Require users to install Python and pip install webinfer-inference

Architecture: Since Node.js daemon and Python engine are separate processes communicating over localhost HTTP, they can be packaged independently. The daemon will auto-detect the Python server when it's running.

Model Plugin System

The Python inference engine uses a plugin-based model system. Models are self-describing and check their own dependencies, so only models that can actually run are exposed.

How it works

  1. Models register themselves using the @register_model decorator
  2. Each model checks its own dependencies via check_dependencies()
  3. The /models endpoint only returns models whose deps are installed
  4. WebInfer providers fetch this list dynamically
Creating a Custom Model Plugin

To add a new model, create a Python file in the models directory:

models/my_model.py
from .base import BaseModel, register_modelfrom typing import ClassVar@register_modelclass MyCustomModel(BaseModel):    """My custom model for audio/text/image generation"""    # Required metadata    id: ClassVar[str] = "my-model-v1"    name: ClassVar[str] = "My Custom Model"    type: ClassVar[str] = "audio"  # or "text", "image"    description: ClassVar[str] = "Description of what this model does"    capabilities: ClassVar[list[str]] = ["audio-generation"]    # Optional metadata    vram_required: ClassVar[str] = "~4GB"    max_duration: ClassVar[float] = 30.0  # for audio    sample_rate: ClassVar[int] = 44100    # for audio    @classmethod    def check_dependencies(cls) -> tuple[bool, list[str]]:        """Check if required packages are installed"""        missing = []        try:            import my_required_package        except ImportError:            missing.append("my_required_package")        return len(missing) == 0, missing    async def load(self, device: str, cache_dir: str | None = None):        """Load model into memory"""        # Your loading logic here        self._loaded = True    async def unload(self):        """Unload model from memory"""        self._model_data = None        self._loaded = False    async def generate(self, prompt: str, **kwargs):        """Generate output from prompt"""        # Your generation logic here        pass
Installing Custom Models

Drop your model file into the models directory:

# Find your webinfer-inference installationpython -c "import webinfer_inference; print(webinfer_inference.__path__[0])"# Copy your model to the models directorycp my_model.py /path/to/webinfer_inference/models/# Edit models/__init__.py to import your modelecho "from .my_model import MyCustomModel" >> /path/to/webinfer_inference/models/__init__.py# Restart the serverwebinfer-inference serve
Bundled Models

These models come pre-bundled with webinfer-inference[audio]:

stable-audio-open-1.0

Generate up to 47s of stereo audio @ 44.1kHz. Requires: torch, diffusers, torchaudio

Audio

Coming soon: More bundled models for image generation (Flux), speech-to-text (Whisper), and text embeddings. Check the GitHub repo for updates.

Model Discovery Flow
┌─────────────────────────────────────────────────────────────┐
│                    Model Discovery                           │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  Python Server Startup                                      │
│    │                                                        │
│    ├── Import models/__init__.py                           │
│    │     └── Imports all model files                       │
│    │         └── @register_model adds to MODEL_REGISTRY    │
│    │                                                        │
│    └── For each model in registry:                         │
│          └── model.check_dependencies()                    │
│              ├── True  → Model available                   │
│              └── False → Model hidden (deps missing)       │
│                                                             │
│  GET /models                                                │
│    └── Returns only models where deps are installed        │
│                                                             │
│  WebInfer Provider                                           │
│    └── Fetches /models → Gets dynamic list                 │
│        └── Routes requests to available models             │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Best Practices

✅ Use with any browser

The daemon works with Firefox, Safari, Edge, Brave, and any browser—not just Chrome. Perfect for cross-browser development and testing.

✅ Keep auth enabled

Only disable auth for testing. Always use token authentication to prevent unauthorized access from other applications on your machine.

✅ Use debug mode when developing

Run with --debug to see detailed logs and understand how your requests are routed and processed.

✅ Remember separate storage

The daemon doesn't share configuration with the browser extension. You'll need to configure providers and API keys separately.