Agent Runtime Engineer
Connects inference, tool use, memory, evals, world state and interface design into controllable AI systems.
AGI RESEARCH
PERFORMANCE-COST BALANCE
PERFORMANCE-COST BALANCE
AI RESEARCHER
HUMAN-MACHINE INTERACTION
RESEARCH MEETS UTILITY
RESEARCH MEETS UTILITY
Sheng-Kai Huang AI HMI portfolio
Prefer solving the core problem first.
Hi, I am Sheng-Kai Huang, a physics graduate. I like building across code, mechanics, electronics, visuals, motion, and UI. Working with AI helps me turn rough ideas into usable systems faster. My long-term direction is AGI and Thinking Machines, with human-machine interaction as the place where people and AI can actually work well together.
Open full resumeWORKS
Generative Models & MLX
Speech Model Optimization
LLM & Harness
Embodied AI
HMI
0. HUMAN
1. MACHINE
2. FEEDBACK
Give AI a friendly environment and interface for interaction.
Beyond HMI, I want to build interfaces designed for AI itself: tool surfaces where AI can call modules quickly, inspect state, and coordinate its own actions. This will be the next stage of my experiments.
AI Skill Tree
Positioning: AI systems engineer for the 2026 agent stack. I work across inference runtime, MLX porting, agent orchestration, memory, evaluation, world-model state, and HMI so research models become fast, inspectable product systems.
CUDA ↔ MLX / Metal Optimization
Perform cross-framework GPU kernel optimization including PyTorch/CUDA to MLX ports, tensor layout, state_dict surgery, custom op implementation, Metal backend, parity validation and profiling.
Inference Optimization
Design and optimize LLM/agent inference pipelines with KV/prefix cache, chunked prefill, quantization, batching, warm services, and TTFT/decode benchmarks across local and cloud.
Memory, RAG, Evals
Task contracts, verifier loops, accepted / rejected memory, semantic recall, route tests, safety gates and replayable traces.
Robot Latent Planning & Action
Develop latent planning systems for robots using visual state, 3DGS reconstruction, pose and motion modeling, afterstate prediction, diffusion repair, and compact scoring for embodied control.
Human + Agent Interface
CLI, Desktop, Browser, VSMONSTER, 2D / 3D FOCUS and shader controls for observable state, interruption, review and recovery.