Applied AI/ML Engineer
Remote (United States)
Job Details
Location: United States
Workplace: Remote
Employment Type: Full-time
Experience: 3+ years delivering ML/AI systems to production
Core Areas: Large Language Models (LLMs), LLM Inference Optimization, Reinforcement Learning, Model Post-Training, Python & PyTorch, GPU Computing, Distributed Training, ML Model Evaluation
Compensation: Proposed annual salary range of $175,000–$250,000 USD, plus equity allocation
About the Role
An Applied AI/ML Engineer is needed to develop, deploy, and optimize AI-powered products across a distributed GPU inference infrastructure. This hands-on engineering position covers the complete machine learning delivery lifecycle, from model selection and prototyping to production inference, evaluation, reinforcement learning, and post-training. The focus is on building reliable AI applications while improving model performance, inference throughput, latency, and operating costs.
The role involves serving large language models using vLLM and SGLang, implementing reinforcement-learning pipelines with Megatron-LM and Prime Intellect frameworks, and optimizing GPU workloads for production environments. The engineer will collaborate with infrastructure specialists on GPU scheduling and fleet utilization and with product stakeholders on development priorities. Success requires independent technical execution, practical experimentation, and the ability to deliver production-ready solutions in ambiguous environments.
What You'll Do
- Own the end-to-end development and delivery of AI-powered features and products, including model selection, prototyping, production deployment, evaluation, and continuous improvement.
- Deploy, operate, and optimize large language model inference using vLLM and SGLang across distributed GPU infrastructure.
- Improve inference throughput, response latency, and cost per token through continuous batching, KV-cache management, quantization, speculative decoding, and multi-model routing.
- Design, implement, and maintain reinforcement-learning and model post-training pipelines using slime, including Megatron-LM and SGLang integrations.
- Develop and operate post-training workflows with Prime Intellect tooling, including prime-rl, the Environments Hub, and verifiers.
- Implement reward functions and verifiers, coordinate training rollouts, manage model weight synchronization, and maintain stability throughout long-running training workloads.
- Build model evaluation harnesses, benchmarks, and testing workflows that assess model quality, inference performance, throughput, and computational costs.
- Use evaluation results, customer feedback, and internal feedback to improve AI model performance and guide technical iterations.
- Collaborate with infrastructure engineers to improve GPU scheduling, resource allocation, and overall fleet utilization.
- Partner with product stakeholders to evaluate technical opportunities, establish development priorities, and deliver practical AI capabilities.
- Independently move AI engineering initiatives from initial concepts to working production systems while navigating changing requirements and technical ambiguity.
Qualifications
Required Experience
- At least 3 years of professional experience developing, deploying, and maintaining machine learning or artificial intelligence systems in production environments.
- Hands-on experience deploying and serving large language model inference using vLLM, SGLang, or TensorRT-LLM.
- Experience with reinforcement learning or model post-training methods such as Group Relative Policy Optimization (GRPO), Proximal Policy Optimization (PPO), Direct Preference Optimization (DPO), or Supervised Fine-Tuning (SFT). Strong adjacent technical experience combined with a clear commitment to developing expertise in these areas is also acceptable.
Required Skills
- Strong Python programming skills and practical proficiency with PyTorch for machine learning development.
- Working knowledge of GPU execution, including batching, GPU memory utilization, and fundamental CUDA concepts.
- Ability to develop, deploy, evaluate, and optimize machine learning systems for production use.
- Understanding of LLM inference serving and the performance considerations involved in operating GPU-based AI workloads.
- Ability to work independently, make technical decisions under uncertainty, and execute efficiently in environments with evolving requirements.
Preferred Qualifications
- Direct experience with slime, prime-rl, the verifiers library, or Megatron-LM for reinforcement learning and model post-training.
- Experience with distributed model training techniques, including Fully Sharded Data Parallel (FSDP), Tensor Parallelism (TP), Pipeline Parallelism (PP), or Data Parallelism (DP).
- Knowledge of model quantization techniques such as FP8 and INT8, prefill/decode (P/D) disaggregation, or speculative decoding.
- Experience with verifiable inference systems or large-scale distributed computing architectures.
- Experience with Kubernetes and container-based application deployment.
- Familiarity with GPU fleet orchestration and workload scheduling tools such as Ray, SkyPilot, or Slurm.
Additional Application Requirements
- A public GitHub profile must be included with the application.
- The GitHub profile must demonstrate at least one year of activity or contribution history.
- Applications without a public GitHub profile or with insufficient GitHub activity history will not be considered.
Benefits
- Medical, dental, and vision insurance for employees based in the United States, with region-adjusted benefits for employees in other locations.
- Flexible paid time off (PTO).
- Professional development opportunities and a budget for conference-related travel.
- Remote-first working environment with regular team off-site gatherings.
Looking for more opportunities?
View All Jobs