Job Description
The stack: Node.js/Express, TypeScript, React, Postgres, Clickhouse, MongoDB, Heroku
The product: We’re building agentic software for Finance and Operations teams to automate high-volume workflows across systems—from AP and AR to payroll, inventory, and reconciliations. See www.woodrow.ai
The way we work:
- This is a remote job—work anywhere you want, in any timezone on earth (or outside it).
- We are an asynchronous team—no scheduled meetings. We rely on each team member to pick their own schedule and communicate clearly in writing and recorded videos.
- We operate completely autonomously—no one will tell you what to do, we’ll tell you the goals and you’ll figure out how to work towards them with the rest of the team.
The role:
We’re looking for our first Quality & Reliability Engineer who thinks about quality as a system, not a phase at the end of development.
You’ll own quality and reliability across the stack—from test infrastructure to production monitoring to LLM evals. This is a founding role: you’ll assess what we need, prioritize ruthlessly, and build the foundations that let us ship fast without breaking things.
This is not a traditional QA role. You may review some pull requests and help write e2e tests, but you will ALSO own the monitoring side and build guardrails that scale—all with the goal of preventing and catching issues before customers do.
This role combines elements of software engineering, traditional QA, reliability engineering, and devops.
What you’ll work on:
CI & Test Health
- Make sure tests consistently run fast and green means green
- Maintain staging and test environments that are useful
- Help review code with a focus on reliability, edge cases, and failure handling
- Build and maintain integration and end-to-end test suites
- Close gaps in test coverage by writing tests yourself
AI Quality & Evals
- Monitor AI workflow correctness and audit individual workflows to catch regressions
- Design and maintain eval suites for LLM features
- Build a repeatable process for validating prompt changes before they ship
Observability & Production Health
Design and implement monitoring, alerting, and dashboards so we understand system health at a glance
Own observability for our data pipelines external integrations, and AI workflows
Make sure we know about failures before customers tell us
You’re an excellent engineer—you can solve hard problems, you’re AI Native, and ship quickly
Experience with observability tooling (Datadog, Sentry, or similar)
Strong testing background (unit, integration, e2e; automation frameworks) both writing, as well as architecting
Familiarity with LLM-based features and eval approaches (or willingness to learn quickly)
Great communicator via writing and recorded video
Comfortable working in ambiguous, fast-changing environments
Excited about working on a globally-distributed, diverse team where autonomy, accountability, and ownership matter as much as your skillset
Unlimited vacation
Completely flexible work schedule—work literally anytime (and anywhere) you want
Benefits appropriate to your location (health/dental/vision in the USA)
Company-wide retreats multiple times per year
Market-rate compensation including benefits—we’re scrappy, but not scrappy on compensation
