Lead Data Engineer

Job description

Company Overview

Founded in 2010, we’ve been recognized as a “Best Places to Work” and have offices in the US (Boulder), UK (London) and India (Chennai). However, we are a remote-first company with employees across the globe! Today, we are a leading B2B marketing provider that offers two distinct solutions:

Integrate

Lead management & data governance SaaS platform for marketing operations and demand marketers. The Integrate platform makes every lead clean, compliant, and actionable, freeing enterprise B2B marketers from bad data and operational headaches so they can focus on what matters: generating revenue.

Pipeline360

Media solutions that combine three powerful demand generation tools: targeted display, content syndication, and a comprehensive marketplace model. Pipeline360 ensures that marketers achieve 100% compliant and marketable leads by effectively engaging with audiences much earlier in the buying cycle, connecting with buyers at every stage of the process, and optimizing programs to drive performance.

Our Mission

• Integrate: exists to make your lead data marketable so you can drive pipeline.
• Pipeline360: exists to make the unpredictable predictable.

Why us?

We are an organization of integrity, talent, passion, and vision with a long track record of growth, customer success, and a commitment to driving leading innovation and delivering world-class customer experience.

The Role:

Integrate’s data is treated as a critical corporate asset and is seen as a competitive advantage in our business. As a Lead Data Engineer you will be working in one of the world’s largest cloud-based data lakes. You should be skilled in the architecture of data warehouse solutions for the Enterprise using multiple platforms (EMR, RDBMS, Columnar, Cloud, Snowflake). You should have extensive experience in the design, creation, management, and business use of extremely large datasets. You should have excellent business and communication skills to be able to work with business owners to develop and define key business questions, and to build data sets that answer those questions. Above all you should be passionate about working with huge data sets and someone who loves to bring datasets together to answer business questions and drive change.

Responsibilities:

• Design and develop workflows, programs, and ETL to support data ingestion, curation, and provisioning of fragmented data for Data Analytics, Product Analytics and AI.
• Work closely with Data Scientists, Software Engineers, Product Managers, Product Analysts and other key stakeholders to gather and define requirements for Integrate’s data needs.
• Use Scala, SQL Snowflake, and BI tools to deliver data to customers.
• Understand MongoDB/PostgreSQL and transactional data workflows.
• Design data models and build data architecture that enables reporting, analytics, advanced AI/ML and Generative AI solutions.
• Develop an understanding of the data and build business acumen.
• Develop and maintain Datawarehouse and Datamart in the cloud using Snowflake.
• Create reporting dashboards for internal and client stakeholders.
• Understand the business use cases and customer value behind large sets of data and develop meaningful analytic solutions.

Basic Qualifications:

• Advanced degree in Statistics, Computer Science or related technical/scientific field.
• 9+ years experience in a Data Engineer development role.
• Advanced knowledge of SQL, Python, and data processing workflow.
• Nice to have Spark/Scala, MLFlow, and AWS experience.
• Strong experience and advanced technical skills writing APIs.
• Extensive knowledge of Data Warehousing, ETL and BI architectures, concepts, and frameworks. And also strong in metadata definition, data migration and integration with emphasis on both high end OLTP and business Intelligence solutions.
• Develop complex Stored procedure and queries to provide to the application along with reporting solutions too.
• Optimize slow-running queries and optimize query performance.
• Create optimized queries and data migration scripts
• Leadership skillsets to mentor and train junior team members and stakeholders.
• Capable of creating long-term and short-term data architecture vision and tactical roadmap to achieve the data architecture vision beginning from the current state
• Strong data management abilities (i.e., understanding data reconciliations).
• Capable of facilitating data discovery sessions involving business subject matter experts.
• Strong communication/partnership skills to gain the trust of stakeholders.
• Knowledge of professional software engineering practices & best practices for the full software development lifecycle, including coding standards, code reviews, source control management, build processes, testing, and operations.

Preferred Qualifications:

• Industry experience as a Data Engineer or related specialty (e.g., Software Engineer, Business Intelligence Engineer, Data Scientist) with a track record of manipulating, processing, and extracting value from large datasets.
•  Experience building data products incrementally and integrating and managing datasets from multiple sources.
•  Query performance tuning skills using Unix profiling tools and SQL
•  Experience leading large-scale data warehousing and analytics projects, including using AWS technologies – Snowflake, Redshift, S3, EC2, Data-pipeline and other big data technologies.

Integrate in the News:

• Best Tech Startups in Arizona (2018-2021)
• Integrate Acquires Akkroo
• Integrate Acquires ListenLoop
• Why Four MarTech CEO’s Bet Big on Integrate