Job description
The world of AI and GenAI is evolving rapidly, and we’re on a mission to build a powerful AI platform that drives innovation across all PandaDoc products. Our goal is to develop scalable, impactful AI solutions that empower small and medium-sized businesses (SMBs) to thrive.
This is a unique opportunity to design, evaluate, and optimize AI models, datasets, and pipelines—helping us turn cutting-edge AI into practical, scalable solutions for tens of thousands of businesses.
What makes this role special?
- AI-First Products – Develop models and systems that enable AI-first experiences, enhancing the value we provide to SMBs.
- High Impact, High Ownership: You’ll have the freedom to build and lead, working cross-functionally with engineers, designers, and product managers and bring the best practices from the industry.
- Real-World AI Challenges: You’ll tackle the complexities of productizing AI and Gen-AI applications—from scalability and observability to security and fine-tuning.
- Innovation at Scale: Your work will enable fast, iterative experimentation for customer-facing AI solutions, helping SMBs grow, compete, and thrive.
In this role, you will:
- Model Evaluation & Testing – Create evaluation frameworks to assess LLMs and GenAI applications, measuring recall, effectiveness, and accuracy.
- Dataset Creation & Data Preparation – Develop high-quality datasets for model training, fine-tuning, and inference.
- Prompt Engineering & Optimization – Establish evaluation pipelines for prompt performance, ensuring reliability and efficiency.
- Model Fine-Tuning – Optimize LLMs for specific business use cases to improve their effectiveness.
- LLM Deployment & Guardrails – Deploy LLMs in production and implement systems to maintain model constraints and safe usage.
- Multi-Model Support – Work with various LLMs (e.g., GPT-4, Claude, fine-tuned models) for efficient routing and task handling.
- AI-Driven Automation – Contribute to AI-powered automation systems that enhance customer workflows and streamline routine tasks.
- Collaboration Across Teams – Work closely with PMs, designers, and engineers to define AI requirements and bring solutions to life.
About you:
We’re looking to complement our current AI/Data Science team by hiring a second Senior Data Scientist with a more specialized focus on working with large volumes of documents, particularly in the context of:
- Metadata extraction – experience building pipelines that extract structured information from documents.
- Retrieval-Augmented Generation (RAG) – designing and optimizing RAG systems, including chunking strategies, retrieval performance, and integration with LLMs.
- Knowledge graphs – understanding of how to represent and query document relationships to support better retrieval and reasoning.
- Semantic and conversational search – familiarity with search systems, especially Elasticsearch or similar tools, and how to build intelligent search experiences, including conversational interfaces.
On top of:
- Python Expertise – 5+ years of experience in Python using in DS.
- GenAI Experience – Hands-on experience with LLMs, RAG, knowledge graphs and fine-tuning.
- LLM Evaluation & Optimization – Proven ability to measure and improve model performance.
- Dataset & Data Preparation Skills – Expertise in building and curating datasets for AI training.
- Production Deployment – Experience deploying LLMs and implementing guardrails for responsible AI usage.
- Trend Awareness – Stays updated on the latest AI research, tools, and frameworks, identifying ways to apply them effectively.
- Customer-Centric Mindset – Passion for building AI solutions that provide real value to end users.
- Fast-Paced & Independent – Thrives in a high-speed environment with autonomy to solve problems.
- Product-Driven Approach – Focuses on building AI solutions that drive tangible business impact.
Why PandaDoc?
At PandaDoc, we’re committed to helping SMBs succeed by providing AI-powered tools that simplify their workflows and drive efficiency. With over 60,000 businesses trusting our platform, we’re now investing in AI to take our impact even further.
You’ll be part of a collaborative, high-energy team that values creativity, autonomy, and real-world impact. If you’re excited about solving complex AI challenges and building tools that matter, let’s talk!
