Job description
Job Title: AWS Glue Developer
Location: Remote
Employment Type: [Full-Time/Contract]
Experience Level: 5 to 12 years
Job Summary:
We are seeking a skilled and experienced AWS Glue Developer to join our dynamic team. The ideal candidate will have a strong background in data integration, ETL (Extract, Transform, Load) processes, and cloud-based data solutions. As an AWS Glue Developer, you will be responsible for designing, developing, and maintaining data pipelines using AWS Glue, ensuring efficient data processing and integration across various data sources. You will work closely with data engineers, data scientists, and other stakeholders to deliver scalable and reliable data solutions.
Key Responsibilities:
Design and Development:
Design, develop, and maintain ETL pipelines using AWS Glue to extract, transform, and load data from various sources (e.g., S3, RDS, Redshift, etc.).
Implement data integration solutions to support business requirements and data analytics needs.
Optimize and tune AWS Glue jobs for performance and cost-efficiency.
Data Processing:
Develop and maintain data processing workflows using AWS Glue, PySpark, and other relevant technologies.
Handle large-scale data processing tasks, ensuring data quality and integrity.
Implement data transformation logic using Python, Scala, or SQL.
Data Cataloging:
Manage and maintain the AWS Glue Data Catalog, including metadata management and schema definitions.
Ensure data is properly cataloged and accessible for querying and analysis.
Collaboration:
Work closely with data engineers, data scientists, and business analysts to understand data requirements and deliver solutions.
Collaborate with cross-functional teams to integrate data pipelines with other systems and applications.
Monitoring and Troubleshooting:
Monitor AWS Glue jobs and data pipelines for performance, errors, and data quality issues.
Troubleshoot and resolve issues related to data processing and integration.
Documentation:
Document data pipelines, workflows, and processes for future reference and knowledge sharing.
Maintain up-to-date documentation on data architecture, ETL processes, and AWS Glue configurations.
Security and Compliance:
Ensure data security and compliance with company policies and industry regulations.
Implement data encryption, access controls, and other security measures as needed.
Qualifications:
Education:
Bachelor’s degree in Computer Science, Information Technology, or a related field.
Experience:
Proven experience as an AWS Glue Developer or in a similar role.
Strong experience with AWS services, particularly AWS Glue, S3, Redshift, RDS, Lambda, and IAM.
Hands-on experience with ETL processes, data integration, and data pipeline development.
Proficiency in programming languages such as Python, Scala, or Java.
Experience with big data technologies like Apache Spark, Hadoop, or similar.
Familiarity with SQL and database management systems (e.g., MySQL, PostgreSQL, etc.).
Experience with data warehousing solutions (e.g., Amazon Redshift, Snowflake, etc.) is a plus.
Skills:
Strong understanding of data modeling, data architecture, and data governance.
Excellent problem-solving skills and attention to detail.
Ability to work independently and as part of a team.
Strong communication and collaboration skills.
Certifications (Preferred):
AWS Certified Data Analytics – Specialty.
AWS Certified Solutions Architect – Associate/Professional.
Other relevant AWS certifications.
Preferred Qualifications:
Experience with CI/CD pipelines and version control systems (e.g., Git, Jenkins, etc.).
Knowledge of data visualization tools (e.g., Tableau, Power BI, etc.).
Familiarity with machine learning and data science workflows.
Experience with containerization technologies (e.g., Docker, Kubernetes).
What We Offer:
Competitive salary and benefits package.
Opportunities for professional growth and development.
A collaborative and innovative work environment.
Flexible working hours and remote work options.
Access to cutting-edge technologies and tools.
