Ask Ben (powered by Gemma 4)

3 Apr 26

aiweb

Google just released the Gemma 4 model family—their most capable open models yet, and the smallest variants are small enough to run entirely in your browser. The E2B model has 2.3 billion effective parameters (5.1B total), where the “E” stands for “efficient”—it uses a technique called Per-Layer Embeddings that reduces compute at inference time while keeping the model’s full representational capacity.1

So naturally I had to see if I could get it running on this site. The widget below loads the LiteRT web build of Gemma 4 E2B via Google’s MediaPipe LLM Inference API and WebGPU, then primes it with content from this site—bio, CV, research interests, and blog post titles—as its system prompt. You can ask it questions about me and my work, and it’ll do its best to answer from that context.

Since I first put this up, Google has released quantisation-aware-trained builds of Gemma 4 for edge devices. A mixed 2/4/8-bit scheme drops the E2B model’s runtime memory below a gigabyte, small enough to fit on a recent phone. The download is still ~2 GB and it still needs a WebGPU-capable browser, so it’s happiest on a high-end phone and a decent connection. The compact widget below lifts the desktop-only guard anyway: same model and the same in-browser inference, now just willing to have a go on your phone.

Footnotes

  1. Each decoder layer gets its own small embedding table rather than sharing a single large one. This means the total parameter count is higher than the effective count, but the model only activates the embeddings it needs for each layer. Clever trick. ↩

Cite this post
@online{swift2026askBenPoweredByGemma4,
  author = {Ben Swift},
  title = {Ask Ben (powered by Gemma 4)},
  url = {https://benswift.me/blog/2026/04/03/ask-ben-powered-by-gemma-4/},
  year = {2026},
  month = {04},
  note = {AT-URI: at://did:plc:tevykrhi4kibtsipzci76d76/site.standard.document/2026-04-03-ask-ben-powered-by-gemma-4},
}