Introduction

WebGLLM runs language models in the browser on its own WebGL 2 engine, for devices and browsers without WebGPU. Its API follows OpenAI's Chat Completions.

InstallCopy Anchor Link

npm install webgllm

First chatCopy Anchor Link

import { WebGLLM } from "webgllm"

const webgllm = new WebGLLM()

const completion = await webgllm.chat.completions.create({
  model: "LiquidAI/LFM3-1.7B-GGUF",
  messages: [{ role: "user", content: "What is the capital of France?" }]
})

console.log(completion.choices[0].message.content)

The first call downloads the model into the browser's storage and loads it. The package is in early development; the API may change.