Job description
About Stripe
Stripe is a financial infrastructure platform for businesses. Millions of companies—from the world’s largest enterprises to the most ambitious startups—use Stripe to accept payments, grow their revenue, and accelerate new business opportunities. Our mission is to increase the GDP of the internet, and we have a staggering amount of work ahead. That means you have an unprecedented opportunity to put the global economy within everyone’s reach while doing the most important work of your career.
About the team
The Data Movement team is responsible for building and operating a very large scale read-only ecosystem extending the primary online database in Stripe, providing much needed complements to the primary DB in terms of read and query capabilities at massive scale.
We build innovative solutions for real-time ingestion of terabytes of data and billions of events into our data warehouse for analysis and business-specific processing. Our team operates some of Stripe’s highest volume data pipelines, processing over 1 GB/second for many business critical use cases are successful for Stripe, including:
- user-facing reporting products (Sigma),
- user-facing dashboards,
- Risk and Fraud Detection,
- Machine Learning,
- Data Scientists,
- Payments (pay-in and payout),
- Ledger,
- Regulatory Reporting,
- Financial Data Reconciliation and
- the list goes on…
There are about 10,000 downstream jobs built by hundreds of internal teams based on this read-only online ecosystem that the Data Movement team owns and operates.
We adopt a combination of open source technologies and in-house built software to ensure high scalability, reliability, and near-realtime freshness of data. Key example technologies used include: Flink, Spark (and structured streaming), Iceberg (and associated table compaction), Trino, Airflow, Kafka, CDC, and Kubernettes, etc. And we operate these at a non-trivial scale, across multiple regions.
It’s a high stake and high visibility role, especially as a Staff Engineer, you will be one of the most senior members in the org. Not only will you provide strong technical directions for the team, we also look up to you as the culture north star for the organization. We really value strong engineers who have a team-first mentality, a strong bias for action, and a solution-focused mindset.
What you’ll do
As a Staff Software Engineer on our team, you will do the following:
- Design, build, and maintain innovative next-generation or first-generation versions of solutions and products, with an emphasis on reliability, scalability, and efficiency.
- Define and design opinionated architectural strategies, policies, and abstractions that build a long term sustainable solution for internal Stripes, that will in turn enhance the experience of millions of Stripe users.
- Ensure operational excellence and enable a highly available & reliable Data Movement platform across streaming and batch workloads, delivering a world-class data and query ecosystem.
- Collaborate nimbly with high-visibility teams and their stakeholders to support their key initiatives - navigating various types of organizational complexity with the support of your own leadership team.
- Connect your work with improvements in the usability and reliability of Open Source Software (OSS) like Apache Airflow, Iceberg, Spark and contribute back to the OSS community.
Who you are
We’re looking for someone who:
Minimum requirements
- Has very strong operational experience with large-scale, high-availability data pipelines from design, to execution, and safe change management. Expertise in Flink, Spark (especially structured streaming), Airflow, Python, Java, SQL, Trino, and API design is a plus.
- Has experience developing, maintaining, and debugging distributed systems built with open source tools
- Has experience building infrastructure-as-a-product with a strong focus on users needs
- Has strong collaboration and communication skills, and can comfortably interact with both technical and non-technical participants.
- Has the curiosity to continuously learn about new technologies and business processes.
- Is energized by delivering effective, user-first solutions through creative problem-solving and collaboration.
Preferred qualifications
Has experience writing production-level code in expertise in Scala, Spark, Flink, Iceberg, Airflow, Python, Java, and SQL is a plus.
Experience packaging and deploying code into cloud-based environments (AWS, GCP, Azure) with tools including Bazel, Docker Containers, etc
Has experience designing APIs or building developer platforms
Has experience optimizing the end to end performance of distributed systems
Has experience with scaling distributed systems in a rapidly moving environment
Has experience working with data pipelines
Genuine enjoyment of innovation and a deep interest in understanding how things work
