Summary
The job is for a data engineer who will develop datasets for various purposes in a multi-cloud environment using Data Bricks and Apache Spark. They should have strong T-SQL skills, familiarity with CI/CD methods, significant experience in Python development, proficiency in PySpark development, and a current Databricks Certification. The company offers education assistance, remote work allowance, internet reimbursement, bench pay, profit share, healthcare reimbursement, 401k with a 4% match, short-term disability, long-term disability, life insurance, and student loan repayment up to $2400 annually.
Requirements
- Strong T-SQL Skills with experience in Snowflake
- Familiarity with CI/CD methods
- Significant experience in Python development
- Highly proficient at PySpark development
- Solid grasp of database engineering and design principles
- Current Databricks Certification
Responsibilities
- Develop datasets that enable data visualization, machine learning and data driven decision making across client product line
- Work with structured and unstructured datasets and automate data pipelines to ingest, analyze, validate, normalize and clean data in a hybrid multi-cloud environment
- Build data ingestion pipelines from multiple data sources using Data Bricks and Apache Spark
- Partner with analysts, solution architects, modelers and developers to build data product and data pipelines
Preferred Qualifications
Experience in building business stakeholder relationships at all levels, leadership communications and managing project delivery from design to launch
Benefits
- Education Assistance/Student Loan Repayment - up to $2400 annually
- Remote work allowance
- Internet reimbursement
- Bench pay
- Profit share
- Healthcare reimbursement
- 401k - 4% match
- Short-Term disability
- Long-Term disability
- Life Insurance
