Job description
Data Engineer
Washington, DC (Hybrid)
About the Role:
We are looking for a talented Data Engineer to join our growing AI team. As a Data Engineer, you will design and build the data infrastructure and pipelines that power our AI/ML capabilities. Your work will ensure that our data scientists and ML engineers have clean, reliable, and scalable data to train, evaluate, and deploy models. You will be at the center of enabling our platform’s AI capabilities by ensuring robust data systems are in place to support experimentation, production workflows, and ongoing analytics.
Key Responsibilities:
- Design, build, and maintain scalable ETL/ELT pipelines for structured and unstructured data.
- Develop data architectures that support large-scale training, inference, and analytics workflows.
- Ensure data quality, governance, and lineage across multiple sources and systems.
- Partner with data scientists and ML engineers to deliver high-quality datasets for model development.
- Optimize data workflows for performance, scalability, and reliability on cloud platforms (AWS, GCP, Azure).
- Leverage modern data engineering tools (e.g., Spark, Databricks, Airflow, Kafka, dbt) to support pipelines and workflows.
- Implement monitoring, alerting, and observability for data pipelines to ensure robustness.
- Work across teams to ensure data systems align with platform and business goals.
Qualifications:
- 5+ years of experience as a Data Engineer or in a similar role focused on large-scale data systems.
- Strong programming skills in Python, SQL, and familiarity with Java/Scala a plus.
- Hands-on experience with big data frameworks (e.g., Spark, Flink, Hadoop) and workflow orchestration (Airflow, Prefect, Dagster).
- Proven experience with cloud-based data platforms (AWS, GCP, Azure) and data lake/warehouse technologies (Snowflake, BigQuery, Redshift, Delta Lake).
- Strong understanding of data modeling, ETL/ELT processes, and distributed data systems.
- Experience with streaming data systems (Kafka, Kinesis, Pub/Sub) preferred.
- Knowledge of data governance, security, and compliance best practices.
- Strong analytical and problem-solving skills, with a focus on building maintainable, scalable systems.
- Excellent collaboration skills and ability to work across engineering, product, and AI teams
