Job description
Who We Are
At Streamline, we are experts in Enterprise Mobility, Product Engineering, and IT Transformation. We help organizations navigate the constantly evolving landscape of IT. Our sole focus is ensuring that our client’s organization is armed with the strategies, products, and solutions that are transformative to their business. Streamline works closely with our clients, takes pride in developing genuine relationships, and embraces open communication and collaboration. Our team is comprised of world-class strategists, architects, engineers, and developers.
Overview
This is a fully remote position. We are seeking a highly experienced and motivated Senior Data Engineer to design, develop, and optimize our data infrastructure and pipelines. The ideal candidate will have extensive experience across the Microsoft Azure data ecosystem and advanced proficiency in key data processing and analysis technologies. You will be instrumental in transforming raw data into actionable insights for the business.
Key Responsibilities
- Design and Development: Design, build, and maintain robust, scalable, and efficient ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks (using Python).
- Data Modeling & Storage: Develop and optimize data models in various storage solutions, including Azure Data Explorer (ADX) and No-SQL databases.
- Performance Optimization: Tune and improve the performance of data pipelines and data storage solutions for optimal data retrieval and processing speeds.
- Collaboration: Work closely with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions that meet business needs.
- Data Visualization Support: Ensure data is accurately structured and prepared for visualization tools, primarily Power BI.
- Quality & Governance: Implement best practices for data quality, security, and governance across all data platforms.
Qualifications and Experience
Bachelor’s or Master’s degree in Computer Science, Engineering, or a related quantitative field.
Minimum 10+ years of professional, hands-on experience across the following technologies:
Python: Expert-level proficiency in Python for data manipulation, scripting, and pipeline development.
Azure: Deep knowledge and experience with the Azure cloud platform and its core data services.
Azure Data Factory (ADF): Extensive experience in building and managing complex, orchestrated data pipelines.
Azure Data Explorer (ADX): Proven experience with ingesting, querying, and optimizing data within ADX (Kusto).
No-SQL: Strong understanding and practical experience with No-SQL databases (e.g., Cosmos DB, MongoDB, Cassandra).
Databricks: Hands-on experience with Databricks environments, Spark, and developing scalable data processing solutions.
Power BI: Experience preparing data specifically for, and a strong understanding of, how data is consumed by Power BI reports and dashboards.
Experience with stream processing technologies (e.g., Azure Event Hubs, Kafka).
Azure certification (e.g., Azure Data Engineer Associate - DP-203).
Technical Skills
- Solid understanding of modern data warehousing concepts, including Kimball and Inmon methodologies.
- Experience with version control systems (e.g., Git).
- Familiarity with CI/CD processes for data infrastructure deployment.
What We Offer
- A challenging and rewarding role in a dynamic and international environment.
- Opportunity to be part of a growing company with a strong commitment to innovation and excellence.
- A supportive and collaborative team culture that values personal growth and development.
- Competitive compensation and benefits package.
