A lightweight, tightly-scoped language-model assistant for Webex Calling and Contact Center — helping engineers explore configuration, troubleshoot flows, and plan migrations. This page includes a working demo and the full build approach: high-level design, low-level design, and reference code.
Click any sample question to see how the assistant responds. This demo runs on pre-written, expert-reviewed answers — no live model calls — so it is free to run and safe to share.
These mirror the kind of queries the assistant is designed to handle.
Planning document. The design and code below describe how this assistant will be built. Full implementation documentation will be published once the tool is developed and lab-validated.
"SLM" — Small Language Model — refers to a deliberately narrow, focused assistant rather than a general chatbot. There are two viable build paths, and this project documents both so the right one can be chosen for the deployment context.
Use a hosted model (e.g. Claude Haiku) behind a secure proxy, grounded with Retrieval-Augmented Generation over Webex documentation. The "small" comes from tight scoping and retrieval, not model size. Fast to stand up, broad coverage, pay-per-use.
Run a genuinely small open model (1–4B parameters) on local hardware, fine-tuned on a Webex corpus with QLoRA. Fully on-premise, no per-query cost, no data leaving the network — at the cost of training effort and infrastructure.
| Factor | A · Hosted API + RAG | B · Local LLM + Fine-tuning |
|---|---|---|
| Setup speed | Fast — days | Slower — weeks |
| Cost model | Pay-per-query (low with Haiku) | Upfront compute, then free inference |
| Data privacy | Queries leave your network | Fully on-premise |
| Answer quality | High — frontier model | Good for the trained domain |
| Hardware | None required | GPU for training; modest box for serving |
| Maintenance | Low — vendor maintains model | Higher — retrain, host, monitor |
| Best fit | Quick demo, broad coverage | Offline, privacy-sensitive deployments |
The HLD describes the major components and how data flows through the system for each approach.
The browser never touches the model directly. An edge proxy holds the API key, enforces limits, and grounds each query with retrieved Webex documentation before calling the model.
Two pipelines: an offline training pipeline that produces a fine-tuned model, and an inference pipeline that serves it locally.
The LLD breaks each component into concrete, implementable detail.
Cloudflare Pages Function. API key stored as an encrypted environment secret. Validates request shape, enforces per-IP daily rate limit via KV, caps max_tokens.
Webex docs chunked (~400 words), embedded with a sentence-transformer, stored in a vector index. At query time the top 3–4 chunks are retrieved and injected as grounding context.
System prompt locks scope to Webex topics, instructs the model to answer only from retrieved context and to recommend verifying against Cisco documentation. Off-topic queries are politely declined.
A 1–4B instruction model — e.g. Phi-3-mini, Llama 3.2 3B, or Qwen 2.5 3B — small enough to fine-tune on a single GPU and serve on a modest box.
Webex documentation converted into instruction/response pairs (JSONL). Cleaned, de-duplicated, and split into train and held-out evaluation sets.
QLoRA — 4-bit quantization plus low-rank adapters — keeps memory low. The trained adapter is merged, converted to GGUF, and served via Ollama or llama.cpp behind a small FastAPI wrapper.
The end-to-end sequence for each path, from raw documentation to a working assistant.
Gather Webex Calling and Contact Center documentation as the knowledge source.
Split docs into passages and generate embeddings.
Store embeddings in a searchable vector store.
Create the function that holds the key and calls the model.
Rate limits, token caps, topic scoping, off-topic refusal.
Connect the front-end to the proxy endpoint.
Validate answers, set a Console spend cap, ship.
Choose a 1–4B instruction model that fits available hardware.
Convert Webex docs into instruction/response pairs.
De-duplicate and separate train vs evaluation sets.
Train low-rank adapters on the quantized base model.
Score answers on the held-out set; iterate if needed.
Merge the adapter and export to GGUF format.
Run via Ollama, wrap with an API, link the UI.
Starter snippets for the core of each approach. These are illustrative — production use requires testing, error handling, and validation.
// API key stays server-side as an encrypted env secret.
const RATE = 5; // questions per IP per day
export async function onRequestPost({ request, env }) {
const ip = request.headers.get('CF-Connecting-IP') || 'anon';
const day = new Date().toISOString().slice(0, 10);
const key = `rl:${ip}:${day}`;
// per-IP daily rate limit via Workers KV
const used = parseInt(await env.RL.get(key) || '0', 10);
if (used >= RATE)
return Response.json({ error: 'Daily demo limit reached.' }, { status: 429 });
const { question, context } = await request.json();
const r = await fetch('https://api.anthropic.com/v1/messages', {
method: 'POST',
headers: {
'content-type': 'application/json',
'x-api-key': env.ANTHROPIC_API_KEY,
'anthropic-version': '2023-06-01'
},
body: JSON.stringify({
model: 'claude-haiku-4-5-20251001',
max_tokens: 350, // hard output cap
system: 'You are a Webex assistant. Answer ONLY Webex ' +
'Calling and Contact Center questions, grounded in ' +
'the supplied context. Decline anything off-topic.',
messages: [{ role: 'user',
content: `Context:\n${context}\n\nQ: ${question}` }]
})
});
const data = await r.json();
await env.RL.put(key, String(used + 1), { expirationTtl: 86400 });
return Response.json({ answer: data.content?.[0]?.text || '' });
}import glob, json
from sentence_transformers import SentenceTransformer
embed = SentenceTransformer('all-MiniLM-L6-v2')
chunks = []
for path in glob.glob('webex_docs/*.md'):
words = open(path).read().split()
# chunk into ~400-word passages
for i in range(0, len(words), 400):
chunks.append(' '.join(words[i:i + 400]))
vectors = embed.encode(chunks).tolist()
json.dump([{'text': c, 'vec': v} for c, v in zip(chunks, vectors)],
open('index.json', 'w')){"instruction": "How do I migrate an Avaya hunt group to Webex Calling?",
"output": "Map it to a Webex Calling Hunt Group or Call Queue ..."}
{"instruction": "What ports does Webex Calling use?",
"output": "SIP signalling, a UDP media range, and HTTPS ..."}from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import LoraConfig, get_peft_model
from datasets import load_dataset
from trl import SFTTrainer
BASE = 'microsoft/Phi-3-mini-4k-instruct' # ~3.8B params
# 4-bit quantization keeps GPU memory low
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type='nf4')
model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb,
device_map='auto')
tok = AutoTokenizer.from_pretrained(BASE)
# low-rank adapters — only these weights are trained
lora = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05,
target_modules=['q_proj', 'v_proj'],
task_type='CAUSAL_LM')
model = get_peft_model(model, lora)
data = load_dataset('json', data_files='webex_train.jsonl')['train']
trainer = SFTTrainer(model=model, tokenizer=tok, train_dataset=data,
max_seq_length=1024)
trainer.train()
model.save_pretrained('webex-slm-adapter')# after merging the adapter and exporting to GGUF
FROM ./webex-slm-merged.gguf
PARAMETER temperature 0.3
PARAMETER num_ctx 4096
SYSTEM """You are the SLM for Webex Assistant. Answer Webex
Calling and Contact Center questions concisely, and recommend
verifying against official Cisco documentation."""# register and run the model locally
ollama create webex-slm -f Modelfile
ollama run webex-slmNext step. Once the assistant is built and lab-validated, this page will expand into full documentation — covering data sourcing, evaluation results, deployment, and operations — and convert into the standard AbhavTech documentation framework.