Sr. Data Engineer

Remote (United States)

Compensation

Estimated hourly pay:$72.12 - $91.35 per hour

Employment Type

Full-Time

About the Role

This opportunity is for a Senior Data Engineer to help design, build, and scale high-throughput data infrastructure for a large production platform. The role focuses on streaming pipelines, large-scale data ingestion, data modeling, data quality, and AI-assisted engineering workflows.

This position is suited for a senior engineer who is highly comfortable using AI tools as part of daily development work and who can connect technical architecture decisions to product outcomes, customer value, and long-term business needs.

What You’ll Do

• Build, scale, and operate high-throughput streaming pipelines that process large volumes of data from hundreds of integrations.
• Design and maintain pipelines using technologies such as Airflow, Spark Streaming, Kafka, and Iceberg.
• Ensure pipeline reliability, performance, and data correctness across large-scale production systems.
• Develop and maintain high-quality production datasets that power core product experiences.
• Focus on data modeling, transformation logic, freshness, accuracy, and cost efficiency.
• Implement and improve data quality checks, monitoring, alerting, and observability practices.
• Detect data issues early and reduce downstream impact through stronger reliability controls.
• Contribute to AI-driven tooling that helps triage, debug, and resolve data quality issues.
• Use AI to improve data operations, increase team efficiency, and reduce manual intervention.
• Shape platform architecture and contribute to AI-powered product delivery.
• Help raise the engineering standard for a small, AI-augmented technical team.

Qualifications

• 6+ years of professional experience in data engineering or software engineering.
• Strong experience with distributed data processing and streaming systems, including Spark, PySpark, and Kafka.
• Proficiency in Python, with Pydantic experience preferred.
• Familiarity with Node.js or TypeScript is a plus.
• Experience building and maintaining data pipelines on AWS using tools such as Airflow, Spark Streaming, and Iceberg.
• Solid understanding of data modeling and large-scale dataset management.
• Familiarity with event-driven systems and ingestion patterns using Kafka, SQS, or similar technologies.
• Experience implementing data quality checks, monitoring workflows, and debugging data issues.
• Interest in applying AI, machine learning, or automation to improve data workflows.
• Proven track record leading high-impact initiatives from concept through production in a SaaS environment.
• Expert-level understanding of software design principles.
• Experience with multi-tenant platform architectures.
• Ability to use AI tools as a core part of the engineering workflow.
• Strong systems thinking and ability to connect technical decisions to customer outcomes and business value.
• Clear communication skills and the ability to explain complex tradeoffs to product, design, and executive stakeholders.
• Ability to work effectively in fast-changing environments with ambiguity, speed, and high ownership.

Tech Stack

• Backend: Python, PySpark, Pydantic, Node.js, TypeScript
• Data: Iceberg, PostgreSQL
• Infrastructure: AWS, Kubernetes, Airflow, Spark Streaming
• Messaging: Kafka, SQS