Job description
As a Principal Data Engineer, you will lead the technical vision, architecture, and delivery of their next-generation GRC data platform. You’ll work at the intersection of AI and cloud-native architecture for building custom data solutions that addresses user critical problems and help us scale intelligent, secure, and resilient data systems across multiple global regions.
You will play a key role in enabling AI-driven insights, automating compliance workflows, and ensuring our platform remains robust and audit-ready for regulated industries.
Data Architecture and Engineering Leadership:
- Responsible for identifying, analysing, and conducting the client’s business requirements gap analysis and processes through document analysis, interviews, workshops, and workflow analysis.
- Guide and support the Data Engineer in the team for executing on projects involving technology upgrade and implementing optimizations wherever appropriateDefine data architecture for scalability, performance and cost-efficiency
- Evaluate and implement new technologies that facilitates or simplifies the end-to-end data lifecycle targeted for reporting and AI use cases
- Documenting the information gathered during the data requirements workshop and preparing the solution architecture documents.
- Partnering with internal stakeholders that includes heads of product, engineering and customer success to identify and streamline priorities and execute it with the team for delivering data processes and products.
- Act as a technical authority providing guidance on data modelling, schema design, metadata management and governance
- Mentor data engineers on the team and help grow the technical capabilities of the data team
- Promote best practices in the ELT processes and the accompanying CI/CD and DevOps for data
- Developing and maintaining SQL queries using various tools, like DBT, Azure Synapse, SQL server, etc. in line with business and customer requirements.
- Developing and maintaining reports in various business intelligence (BI) platforms, including Yellowfin, in line with business and customer requirements.
- Using project management methodologies, principles and techniques, developing project plans and estimate the required effort (cost) and resources
- Working with the broader engineering and product team and other stakeholders on troubleshooting, investigating, and remediating reporting bugs and issues.
- Analysing data and actively coordinating with stakeholders to evaluate the inefficiencies of business process and to ensure that they are accurately updated to eliminate any potential issues.
- Implementing enterprise data warehouse infrastructure and managing the data pipelines feeding into analytics and advanced AI powered analytics subsystems AI/ML Enablement
AI/ML Enablement:
- Design ML-ready data systems that feed into the 6Clicks AI system, including feature stores, training pipelines and inference-serving infrastructure.
- Collaborate with the AI engineering, core engineering and product teams to operationalize models that facilitate automation in the GRC processes
- Partner with product, AI/ML, and platform teams to deliver data solutions that directly power client-facing features
- Set technical direction and best practices for the data engineering functions that drive AI/ML functionalities
- Evaluate emerging technologies and lead critical platform upgrades or migrations that will accelerate AI/ML feature development
Must-have Skills / Qualification:
- At least 8 years of relevant work experience.
- Prior experience in a principal, staff or lead engineer role within high-growth SaaS or enterprise tech environments
- Strong understanding of data security, compliance, and multi-region cloud deployments
- Proven experience in designing and developing ETL pipelines using cloud-based technologies.
- Experience integrating data from multiple sources, including SQL databases, Excel spreadsheets, and APIs.
- Excellent communication and collaboration skills, with the ability to effectively interact with stakeholders at all levels of the organization.
- Detail-oriented with strong analytical and problem-solving skills.
- Self-motivated and able to work independently as well as part of a team. Ability to relate to people, understand their needs and align it with proposed solutions.
Advantageous Skills:
- Azure & AWS cloud experience
- ML/AI experience
- Preferred candidate with master’s in data analytics or related field of study.
- Previous experience in GRC or GRC technologies is a plus.
