Coding AI Agents in TypeScript: Lesson 1 (Sessions 5–6) — Local Providers

Sessions 1–4 gave us a clean CLI + provider abstraction + streaming + deterministic tests.

Series navigation

• Lesson 1 (Sessions 1–4): CLI + Provider + Streaming + Tests: /blog/coding-ai-agents-in-typescript/learning-ai-agents-lesson-1
• Lesson 1 add-on (Sessions 5–6): Local providers (Ollama + LM Studio) — run without an OpenAI subscription (you’re here)

Sessions 5–6 are where it starts feeling real: you run the exact same agent-playground loop against locally hosted models—without rewriting your CLI.

You’ll add (and test) two flavors of “local”:

• Ollama: native API (/api/chat) with newline-delimited JSON (JSONL/NDJSON) streaming
• LM Studio: an OpenAI-compatible local server (so you can reuse the OpenAI provider)

TL;DR

By the end of these sessions you can:

• Switch providers via LLM_PROVIDER (openai, ollama, lmstudio)
• Stream tokens from:
  • OpenAI-style SSE (OpenAI / LM Studio)
  • JSONL streaming (Ollama)
• Keep tests deterministic with local mock servers for both streaming formats

The key mental model: “same contract, different wire format”

Your CLI depends on one thing:

• LLMProvider.streamChat() yields LLMEvents (token, done, error)

The provider decides how to turn HTTP streaming into those events.

flowchart LR
  CLI[CLI UI] -->|LLMEvent stream| CLI
  CLI --> P{Provider}

  P -->|SSE frames| OpenAI[(OpenAI-compatible server)]
  P -->|JSONL lines| Ollama[(Ollama /api/chat)]

Session 5 — Multi-provider streaming (OpenAI + Ollama)

Session 5 is where the abstraction pays off: you add OllamaProvider without breaking the OpenAI code path.

Step 1: Run the Session 5 tests

cd learning-paths/coding-ai-agents/lesson-1/session-5
pnpm install

# If pnpm warns about ignored build scripts (e.g., esbuild), you may need:
# pnpm approve-builds

pnpm test

You should see deterministic tests for:

• OpenAI provider streaming (SSE)
• Ollama provider streaming (JSONL)

Step 2: Understand Ollama’s streaming format (JSONL)

Ollama’s /api/chat streams newline-delimited JSON objects.

Session 5 adds a tiny parser in src/util/jsonl.ts:

export async function* parseJsonLines(body: ReadableStream<Uint8Array>) {
  // ...decode chunks into a buffer...
  // ...split on "\n"...
  // ...JSON.parse each line...
}

This is the JSONL equivalent of “split SSE frames on \n\n.”

Step 3: Implement OllamaProvider

Provider implementation in src/llm/providers/ollama.ts:

• POSTs to /api/chat
• Reads JSONL lines
• Yields normalized LLMEvents

Key excerpt:

for await (const raw of parseJsonLines(res.body)) {
  const chunk = raw as OllamaChatChunk;

  const token = chunk.message?.content;
  if (token) yield { type: "token", token };

  if (chunk.done) {
    yield { type: "done" };
    return;
  }
}

Notice the contract doesn’t change: only the wire parsing does.

Step 4: Run against a real Ollama install (optional)

If you have Ollama running locally:

cd learning-paths/coding-ai-agents/lesson-1/session-5

set LLM_PROVIDER=ollama
set OLLAMA_MODEL=llama3.2

pnpm dev -- run --prompt "Write a haiku about TypeScript streams"

Session 6 — LM Studio (OpenAI-compatible local server)

Session 6 is the cleanest kind of “local provider”: LM Studio can expose an OpenAI-compatible server, so you can reuse the OpenAI provider implementation.

Step 1: Run the Session 6 tests

cd learning-paths/coding-ai-agents/lesson-1/session-6
pnpm install

# If pnpm warns about ignored build scripts (e.g., esbuild), you may need:
# pnpm approve-builds

pnpm test

Step 2: Understand the LM Studio provider design

The LM Studio provider in src/llm/providers/lmstudio.ts is a wrapper around OpenAIProvider:

export class LMStudioProvider implements LLMProvider {
  readonly name = "lmstudio";
  private readonly openAi: OpenAIProvider;

  constructor(opts: LMStudioProviderOptions) {
    this.openAi = new OpenAIProvider({
      apiKey: undefined,
      model: opts.model,
      baseUrl: opts.baseUrl,
      timeoutMs: opts.timeoutMs,
      debug: opts.debug,
    });
  }

  streamChat(request: ChatRequest): AsyncIterable<LLMEvent> {
    return this.openAi.streamChat(request);
  }
}

That’s the “OpenAI-compatible” advantage: you don’t re-implement SSE parsing.

Step 3: Run against a real LM Studio server (optional)

Session 6 suggests:

# Start the LM Studio server
lms server start --port 1234

cd learning-paths/coding-ai-agents/lesson-1/session-6

set LLM_PROVIDER=lmstudio
set LMSTUDIO_MODEL=openai/gpt-oss-20b

pnpm dev -- run --prompt "Write a haiku about local models"

Your CLI stays the same—only the provider selection and model change.

Deterministic testing strategy (why this matters)

Sessions 5–6 keep tests stable by standing up local mock servers:

• OpenAI mock server: emits SSE frames with deterministic tokens
• Ollama mock server: emits JSONL lines with deterministic tokens

This validates your streaming parsing and event normalization without:

• paying for API calls
• hitting rate limits
• depending on network flakiness

Troubleshooting

• Ollama streams but you get no tokens

  • Confirm you’re parsing JSONL lines and reading chunk.message.content.

• LM Studio works but OpenAI key validation fails

  • LM Studio uses apiKey: undefined intentionally; it’s local.
  • Make sure env validation allows missing keys when using local base URLs.

• Tests hang

  • Check AI_TIMEOUT_MS is reasonable.
  • Ensure your mock servers call res.end() after sending completion.

Next steps

Once you can run locally, the next “agent” milestones are about orchestration—not provider plumbing:

• Add a second command: agent-playground chat + transcript save
• Add tool calling (function selection) behind the same provider boundary
• Add structured logs when AI_DEBUG=true
• Add evals: deterministic test prompts + expected behaviors

Further reading

• Ollama API docs: https://docs.ollama.com/api
• LM Studio developer docs: https://lmstudio.ai/docs/developer
• MDN: Server-sent events: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events
• Vitest: https://vitest.dev/

---

Suggested tags: local-llm, ollama, lm-studio, streaming, typescript

Social blurb (optional): Lesson 1 continues: swap your agent loop from cloud calls to local models with Ollama (JSONL streaming) and LM Studio (OpenAI-compatible streaming)—and keep it testable with deterministic mock servers.