You will own the model training stack end-to-end, designing and training compression models from scratch on NVIDIA B200s. Moving beyond simple RAG or prompt engineering, you will solve deep research problems in mechanistic interpretability and data generation to identify which information truly matters to an LLM, delivering immediate cost and latency improvements for production users.
Member of Technical Staff, Research at The Token Company
As a Member of Technical Staff, you will lead from-scratch model training on NVIDIA B200s, designing novel architectures that identify exactly what information an LLM needs to slash inference costs for global scale-ups and enterprises. You will own the entire stack—from data generation to production deployment—within a high-intensity research team that provides full-service housing, food, and visa sponsorship in the heart of San Francisco.
Want to apply for this role?
Jack finds you jobs at companies like The Token Company. Talk to Jack to get considered for roles that fit what you're great at.
Location
San Francisco, United States
Compensation
$120k-$300k + Equity
Company
The Token Company
Role overview
The Token Company builds LLM input compression middleware using a fast machine learning model that removes useless tokens from prompts via a drop-in API. The compression model preprocesses LLM inputs by removing least-significant tokens to cut token counts, latency, and inference costs. It compresses 100k tokens in under 100ms and typically reduces input tokens by approximately 66% while improving model accuracy and enabling larger context windows. The company’s bear-1 and bear-1.1 compression models preserve semantic intent while stripping noise, with benchmarks showing accuracy gains (e.g., +2.7 percentage points on financial QA with up to 20% fewer tokens) and up to 37% faster end-to-end latency. Integration takes minutes via a simple API.
What you will do
- Design and train novel transformer architectures from scratch on NVIDIA B200 clusters to compress context while preserving downstream model answerability.
- Build and own the entire training pipeline, including synthetic data generation, post-training, evaluation design, and shipping models into production.
- Analyze mechanistic interpretability and routing strategies to ensure compressed contexts generalize across domains without degrading downstream LLM performance or quality.
Who this is a fit for
- Deep experience pretraining or performing significant post-training on transformer models, moving beyond basic fine-tuning or calling external model APIs.
- Strong ML fundamentals with the ability to translate research papers into high-performance training runs and novel architectural implementations from scratch.
- A production-first mindset that prioritizes shipping models that solve real-world economic constraints over purely academic publication goals or static evals.
Why this role is remarkable
- Work on a high-leverage research problem—context compression—with a direct feedback loop from real production usage and high-growth scale-up customers.
- Exceptional backing with $12M from First Round, NEA, and the founders of Hugging Face, OpenAI, and xAI, providing massive resources and runway.
- Live and work in San Francisco with a unique support package including housing, food, laundry, and visa sponsorship, allowing total focus on research.
How Jack & Jill work together
I get to know what you’re great at, then find roles you’d never find yourself.Ok, I'll go first. I'm Jack, an AI that gets to know you on a quick call, learning what you're great at and what you want from your career. Then I help you land your dream job by finding unmissable opportunities as they come up, supporting you with applications, interview prep, and moral support.
I recruit from Jack’s network and make the intro when I spot a great match.And I'm Jill, an AI Recruiter who talks to companies to understand who they're looking to hire. Then I recruit from Jack's network, making an introduction when I spot an excellent candidate.
Jack gets to know what you're great at and what you want next, then searches 15 million jobs daily and helps you discover roles at companies like this.
Meet Jack
What happens next?
Jack’s an AI agent for job searching and career coaching. He works for you.
Jill is the AI recruiter working for the company. She recruits from Jack’s network.
If your profile’s a match and The Token Company wants to meet, Jill will make the intro. In the meantime, Jack will send you excellent alternatives.