Job description
Description
The Sympower data team is dedicated to delivering forecasting services, business intelligence (BI) dashboards, and self-service data capabilities. We work in close collaboration with other Engineering Teams to seamlessly integrate data within the broader Sympower software ecosystem.
As a Senior Data Engineer at Sympower, your primary focus will be on the technical integration of data. This entails the data infrastructure and integration pipelines of our Data Platform. While the foundational integration pipelines have been established, we are now seeking a skilled individual to expand and enhance this component. You will be responsible for building new and enhancing existing data integration pipelines, optimising the data infrastructure, and ensuring seamless integration (both input and output) within our software systems.
What is in it for you
We are committed to creating an inclusive, values-based culture where everyone feels that they belong and has the opportunity to do meaningful work. We offer a market competitive compensation package, including but not limited to:
30 Days Paid Holiday Leave
1 Day Paid Wellness Leave
1 Day Paid Birthday Leave
Paid Maternity and Partner Leave
Pawternity Leave
Mental Health and Wellbeing Support
Remote Office Budget
Internet Allowance
Development Plan & Budget
Stock Appreciation Rights
2 Days Paid Volunteer Leave
Learn about all of our benefits on our careers page.
Requirements
What you will do
During your daily job, you will:
Design, develop, and maintain scalable data infrastructure and ETL pipelines to support self-service data solutions, ensuring data quality, reliability, and availability for end-users.
Collaborate with analytics engineers to understand data requirements and optimise data structures, enabling efficient querying and data transformations for analytics and reporting.
Automate data quality checks and data observability tools to monitor pipeline health and data accuracy, providing insights and alerts on data anomalies or pipeline failures.
Troubleshoot and resolve data pipeline issues quickly, providing support to analytics engineers and end-users to ensure minimal disruption to data workflows.
Integrate data from various internal and external sources from APIs, Apache Pulsar, (no)SQL databases.
Develop batch and streaming data pipelines with the full Databricks stack, with a strong focus on software engineering best practices.
Drive data architecture and coherent data modelling across our data platform.
You will work with the other roles in the data team (Machine Learning Engineers and Analytics Engineers) to deliver end-to-end data products to the other Sympower teams, the Sympower platform and our customers.
What you need
Strong software-engineering mindset, approach and skills (Python).
Experience with git, CI/CD, testing frameworks, DevOps.
Experience with building and improving data infrastructure, familiar with the Lake House concept and data cloud infrastructure.
Experience with data integration and processing: Big data storage, processing, ELT (batch/streaming) development and optimisation in Pyspark or other frameworks.
Knowledge of data modelling techniques, e.g. Kimball, Inmon, etc.
Advocate of data architecture standards: high-level infrastructure/pipeline design, data flow diagrams, data models.
Experience and affinity with mentoring and coaching other team members, helping them to grow.
Experience driving a project from start to end, including planning, task breakdown, discussions with stakeholders and working together with product management.
Challenging and steering the team’s technical and product roadmap.
Fluent in English and other languages are nice to have.
Nice to have:
- Experience and affinity with our current data team’s tech stack: Python, Databricks, Spark, Delta Lake, Pulsar, ****Postgres, AWS
Who we are
Sympower is accelerating the global transition towards ‘net-zero’ by helping to build smarter, cleaner renewable energy systems. Using our proprietary software platform, we help balance the supply and demand of electricity across international energy networks.
We help businesses, grid operators, asset owners and other energy stakeholders around the world reduce their carbon emissions, integrate more distributed renewable energy resources, and generate new revenue streams by participating in demand-side response services.
Learn more about us in this video.
In 2022, Sympower became a certified B Corp, which shows the company is meeting high standards of verified performance, accountability, and transparency across 5 impact areas: governance, workers, community, environment, and customers.
Sympower is an equal opportunity employer. We encourage a diverse workforce and are committed to creating an inclusive environment for all team members.
Your personal data will be processed in accordance with our Privacy Policy.
MESSAGE TO RECRUITMENT AGENCIES: support for filling this position is not required, so proposals for recruitment services will not be reviewed or responded to.
