TWCC Ollama Proxy

TWCC Ollama Proxy forwards local Ollama API requests to a TWCC (Taiwan Computing Cloud) development-type container (CCS) for LLM inference, exposing an endpoint at localhost:11434 that is API-compatible with standard Ollama. It currently supports the CrystalMind and GemmaPro GGUF models, with a reasoning-tuned GemmaPro-r variant for more complex extraction tasks.

Any tool that lets you point at a custom Ollama URL, such as Open WebUI or Continue.dev, can connect to it directly. Each API call spins up a TWCC CCS container with one GPU, runs the requested inference, and tears the container down afterward, so cost is billed only for the container’s short lifetime (typically one to two minutes per call). Setup requires a Windows machine, Python 3.10+ managed with uv, a TWCC account with CCS access, and an uploaded SSH key; model files are staged once on TWCC’s shared HFS storage.

Status: Active — under regular maintenance.