A domain-specific AI consulting system for Cisco Webex Calling and Contact Center — helping engineers explore configuration, troubleshoot flows, and plan migrations with verifiable, traceable reasoning. This page includes a working demo, the system design, and a component map, with the full source on GitHub.
View source on GitHub ·rajmohan80/wxcc-slm
Click any sample question to see how the assistant responds. This demo runs on pre-written, expert-reviewed answers — no live model calls — so it is free to run and safe to share.
These mirror the kind of queries the assistant is designed to handle.
Planning document. The design and code below describe how this assistant will be built. Full implementation documentation will be published once the tool is developed and lab-validated.
"SLM" — Small Language Model — refers to a deliberately narrow, focused assistant rather than a general chatbot. There are two viable build paths, and this project documents both so the right one can be chosen for the deployment context.
Use a hosted model (e.g. Claude Haiku) behind a secure proxy, grounded with Retrieval-Augmented Generation over Webex documentation. The "small" comes from tight scoping and retrieval, not model size. Fast to stand up, broad coverage, pay-per-use.
Run a genuinely small open model (1–4B parameters) on local hardware, fine-tuned on a Webex corpus with QLoRA. Fully on-premise, no per-query cost, no data leaving the network — at the cost of training effort and infrastructure.
| Factor | A · Hosted API + RAG | B · Local LLM + Fine-tuning |
|---|---|---|
| Setup speed | Fast — days | Slower — weeks |
| Cost model | Pay-per-query (low with Haiku) | Upfront compute, then free inference |
| Data privacy | Queries leave your network | Fully on-premise |
| Answer quality | High — frontier model | Good for the trained domain |
| Hardware | None required | GPU for training; modest box for serving |
| Maintenance | Low — vendor maintains model | Higher — retrain, host, monitor |
| Best fit | Quick demo, broad coverage | Offline, privacy-sensitive deployments |
The HLD describes the major components and how data flows through the system for each approach.
The browser never touches the model directly. An edge proxy holds the API key, enforces limits, and grounds each query with retrieved Webex documentation before calling the model.
Two pipelines: an offline training pipeline that produces a fine-tuned model, and an inference pipeline that serves it locally.
The LLD breaks each component into concrete, implementable detail.
Cloudflare Pages Function. API key stored as an encrypted environment secret. Validates request shape, enforces per-IP daily rate limit via KV, caps max_tokens.
Webex docs chunked (~400 words), embedded with a sentence-transformer, stored in a vector index. At query time the top 3–4 chunks are retrieved and injected as grounding context.
System prompt locks scope to Webex topics, instructs the model to answer only from retrieved context and to recommend verifying against Cisco documentation. Off-topic queries are politely declined.
A 1–4B instruction model — e.g. Phi-3-mini, Llama 3.2 3B, or Qwen 2.5 3B — small enough to fine-tune on a single GPU and serve on a modest box.
Webex documentation converted into instruction/response pairs (JSONL). Cleaned, de-duplicated, and split into train and held-out evaluation sets.
QLoRA — 4-bit quantization plus low-rank adapters — keeps memory low. The trained adapter is merged, converted to GGUF, and served via Ollama or llama.cpp behind a small FastAPI wrapper.
The end-to-end sequence for each path, from raw documentation to a working assistant.
Gather Webex Calling and Contact Center documentation as the knowledge source.
Split docs into passages and generate embeddings.
Store embeddings in a searchable vector store.
Create the function that holds the key and calls the model.
Rate limits, token caps, topic scoping, off-topic refusal.
Connect the front-end to the proxy endpoint.
Validate answers, set a Console spend cap, ship.
Choose a 1–4B instruction model that fits available hardware.
Convert Webex docs into instruction/response pairs.
De-duplicate and separate train vs evaluation sets.
Train low-rank adapters on the quantized base model.
Score answers on the held-out set; iterate if needed.
Merge the adapter and export to GGUF format.
Run via Ollama, wrap with an API, link the UI.
The working build now lives on GitHub. Rather than a single hosted-API demo, it is a domain-specific consulting system: a structured knowledge layer decides what is correct, a RAG pipeline supplies grounded evidence, and the model only classifies, retrieves, and formats. Here is how it is put together and which tools do which job.
Full repository —github.com/rajmohan80/wxcc-slm
Enterprise AI consulting fails when the intelligence lives only in model weights — they are opaque, unauditable, and stale the moment a new Cisco release ships. This build puts the intelligence in four structured workbooks and a provenance-ranked corpus instead. The model classifies intent, retrieves grounded evidence, and formats the answer; the workbooks decide what is correct. Certain queries never reach the generator at all — a stop-condition check runs before any LLM call, so an impossible request (for example, a data-locality region Cisco does not offer) returns a sourced blocker and a valid alternative rather than a hallucination.
Steps 3–5, 7 and 9 are deterministic workbook rules — not model calls. The LLM is invoked only where judgement genuinely helps: intent, scenario, and the architecture draft.
| Layer | Component | Technology | Role | Status |
|---|---|---|---|---|
| Knowledge | Workbooks A–D | Structured rows (1,414) | Requirements, product knowledge, architecture patterns, engineering guardrails | Complete |
| Knowledge | RAG corpus | Qdrant + BGE-M3 | 48+ provenance-tiered docs, 2,633 chunks, Tier-1 ranked above Tier-2 | Ingested |
| Pipeline | Intent flow | LangGraph (9-step) | Classify → detect → check → retrieve → flag → generate → validate | Complete |
| Pipeline | Query engine | Qdrant retrieval | Provenance-ranked scoring with knowledge-date stamping | Complete |
| Pipeline | Agent | LangChain ReAct | Tool-calling agent with session memory and thread isolation | Complete |
| Delivery | REST API | FastAPI | /query and /health, local dev on :8000 | Local |
| Delivery | MCP server | FastMCP | 11 tools for agentic access to the knowledge layer | Local |
| Delivery | Frontend | Streamlit | Interactive demo UI, local dev on :8501 | Local |
| Automation | Freshness loop | n8n (4 workflows) | Weekly source checks keep the knowledge-date stamp honest | Planned |
| Ops | Deployment | GCP Cloud Run | Public demo endpoint, post-validation | Planned |
Demo build. The pipeline runs locally, the knowledge base is complete, and the architecture is production-ready. Cloud Run deployment, a public Streamlit demo, and the n8n automation loop are on the roadmap. Full source, schema references, and the build status board are in the GitHub repository.