WebInfer

Browser Usage

Use the WebLLM SDK directly in the browser

Quick Start

Install the Client SDK

terminal

npm install @webllm/client

Basic Usage

app.js

import { generateText, isAvailable } from 'webllm'// Check if WebLLM is availableif (await isAvailable()) {  // Make an AI request  const result = await generateText({    prompt: 'Explain quantum computing in simple terms'  })  console.log(result.text)} else {  console.log('WebLLM extension not installed')}

API Methods

generateText()

General text generation

import { generateText } from 'webllm'const result = await generateText({  messages: [    { role: 'system', content: 'You are a helpful assistant.' },    { role: 'user', content: 'Your prompt here' }  ]})console.log(result.text)

streamText()

Stream responses in real-time

import { streamText } from 'webllm'const { textStream } = await streamText({  prompt: 'Write a story'})for await (const chunk of textStream) {  process.stdout.write(chunk)}

isAvailable()

Check if WebLLM is available

import { isAvailable } from 'webllm'if (await isAvailable()) {  // WebLLM is ready to use}

requestPermission()

Request AI permission from user

import { requestPermission } from 'webllm'const status = await requestPermission()if (status === 'granted') {  // User granted permission}

Available Capabilities

WebLLM supports multiple AI capabilities through a unified API:

Core Functions

generateText

Text generation

streamText

Streaming text

generateObject

Structured output

generateImage

Image generation

embed

Text embeddings

embedMany

Batch embeddings

Common Options

• prompt: Text prompt

• messages: Chat messages array

• model: Specific model ID

• provider: Specific provider

• mock: Use mock mode for testing

Error Handling

error-handling.js

import { generateText } from 'webllm'try {  const result = await generateText({    prompt: 'Your prompt'  })  console.log(result.text)} catch (error) {  if (error.name === 'PermissionDenied') {    console.log('User denied permission')  } else if (error.name === 'NoProviderAvailable') {    console.log('No AI provider configured')  } else if (error.name === 'RateLimitExceeded') {    console.log('Rate limit exceeded')  } else {    console.error('Request failed:', error)  }}

CDN Usage (No Build Step)

index.html

<!DOCTYPE html><html><head>  <script type="module">    import { generateText } from 'https://esm.sh/webllm'    const result = await generateText({      prompt: 'Hello AI!'    })    document.body.innerHTML = result.text  </script></head><body></body></html>

Next Steps

API Reference

Complete API documentation

View API Docs →

Examples

Working code examples

View Examples →