AI

Typed LLM API Use TypeScript types to enforce a schema on LLM output

Model API Gateway Call over 20+ LLMs from a single API. For a WebGPU powered Llama-2 playground, see thiggle

Structured LLM Completion APIs Structured LLM completion APIs. Generate text with LLMs that conform to regex patterns, context-free grammars, or classification.

@react-llm/headless Easy-to-use headless React Hooks to run LLMs in the browser with WebGPU. All inference happens clientside in the browser. See chat.matt-rickard.com for a live example.

LLaMaTab A Chrome Extension that uses @react-llm/headless to run a LLM entirely in the browser. Load the model once and run on any site.

ReLLM Exact structure out of any LLM by constraining pre-generation logits with regex. ParserLLM is an extension to ReLLM that constrains LLMs to only generating a specific context-free grammar (e.g., JSON).

openlm a drop-in OpenAI-compatible Python library that can call LLMs from any other hosted inference API. OpenLM takes in the same parameters as the openai.Completion class and and outputs a similarly structured result.

llm.ts A TypeScript equivalent of OpenLM that works in any JS environment (browser, node, deno)

ScapeNet osrs-ocr Fine-tuned object recognition and OCR models for the MMORPG Old School RuneScape

Dist. Systems

minikube - Run Kubernetes locally

skaffold - Define, develop, and run multi-deployment applications with Kubernetes

kubeflow - Machine Learning Toolkit for Kubernetes

distroless - Built a package manager for adding APT packages into distroless containers

minikube-kvm - Virtualization backend for minikube to run KVM machines on Linux

virgo - Graph-based Declarative Configuration Language

mockerfile - Docker buildable alternative BuildKit frontend in YAML

docker-merge - Merge Docker images like git branches

matt-rickard.com

10-04 The Benefit of a Personal Notation

10-03 Pitfalls of Database Migrations

10-02 Generative Interfaces

10-01 Lessons From Debugging

09-30 Compression / Learning Duality

09-29 Is AI a Platform Shift?

09-28 Passkeys, Crypto, and Signing AI Content

09-27 Is Data Still a Moat?

09-26 Multi-Modal AI is a UX Problem

09-25 Observer-Expectancy at Scale