Senior AI Engineer, Agentic Evaluation & V&V

Remote (United States)

About the Role

This opportunity is for a Senior AI Engineer focused on building, deploying, and optimizing production machine learning and AI services. The role is hands-on and centers on creating reliable, scalable, and high-performance AI subsystems that turn high-level product and business requirements into production-ready software.

This position owns specific model pipelines and is responsible for making them efficient, testable, maintainable, observable, and reliable in real-world production environments.

Status: Full-Time

Compensation: $159,500 – $182,000 per year

What You’ll Do

• Build, deploy, and optimize machine learning services that support a large-scale technology platform.
• Develop robust ML subsystems and translate high-level requirements into production-ready code.
• Write high-quality production software for data ingestion, feature extraction, and model inference.
• Optimize code for latency, throughput, and resource efficiency.
• Implement CI/CD pipelines, automated testing, and comprehensive logging and monitoring for deployed models.
• Ensure model issues can be detected quickly through strong observability, monitoring, and alerting practices.
• Build and maintain data pipelines required for model training and inference.
• Ensure data quality and consistency at the component level.
• Create reusable software modules, utilities, and internal tools that improve development speed for the broader team.
• Promote clean code practices and test-driven development.
• Translate business requirements into technical specifications and execute them with precision.
• Break complex technical work into clear, deliverable units.
• Monitor daily production model performance and debug incidents when issues occur.
• Run routine model retraining workflows to address data drift.
• Partner with engineering team members and product managers to estimate effort, identify technical risks, and deliver features on schedule.

Qualifications

• 5+ years of professional software development experience, including system design, large-scale services, and production-grade infrastructure.
• 3+ years of hands-on experience in machine learning engineering or applied AI, including deploying and maintaining models in production.
• Technical subject matter expertise in 3+ general software development areas such as server-side systems, databases, security, or related infrastructure, including machine learning infrastructure.
• Demonstrated ability to deliver significant, measurable real-world impact through applied machine learning.
• Proven ability to design and write modular, performant, maintainable, and easy-to-read software that solves complex business problems.
• Proficiency in Python, TensorFlow or PyTorch, and scikit-learn.
• Strong background in MLOps and data infrastructure, including tools such as Airflow, Spark, feature stores, MLflow, and data versioning systems.
• Proven ability to deploy and maintain machine learning models in production using CI/CD, monitoring, and alerting.
• Familiarity with cloud machine learning environments such as AWS, Google Cloud Platform, or Microsoft Azure.
• Experience with containerization technologies such as Kubernetes and Docker.
• Experience building or fine-tuning large language models or generative models for structured business processes.
• Experience with retrieval-augmented pipelines or feedback-driven model retraining.
• Excellent technical communication skills and a product-focused mindset.
• Ability to drive technical initiatives from concept through delivery.

Preferred Experience

• Background in healthcare software operations or financial automation.
• Contributions to open-source machine learning infrastructure projects.
• Published research or conference papers in machine learning, natural language processing, or applied AI.
• Experience leading AI reliability and observability initiatives.
• Experience designing monitoring frameworks, drift detection systems, and alerting systems for multiple production models.

Remote Work Eligibility

This remote role is currently available only to candidates residing in the following U.S. states and districts: AL, AZ, CA, CO, DC, FL, GA, HI, IL, IN, KS, MA, MD, MI, MN, MO, MT, NC, NJ, NM, NV, NY, OH, OK, OR, RI, TN, TX, UT, VA, WA, WI, and WV. Candidates residing in other U.S. states are not eligible at this time.

Additional Information

Compensation may vary based on experience, qualifications, role requirements, and geographic pay zone. Eligible employees may also receive variable pay and a benefits package.