Job description
In Competera, we are building a place where optimal pricing decisions can be made easily. We believe that AI technologies will soon drive all challenging decisions and are capable of helping humans be better. We are now looking for a Senior Software Engineer (Scraping) to take ownership over Data Delivery engineering growth and direction. Our stack: Programming Languages: Python.
Tech Stack: PostgreSQL, asyncio, FastAPI, Apache Kafka, ClickHouse, BigQuery, GCS, Scrapy, Selenium, Pyppeteer.
Infrastructure: Kubernetes, Helm, ArgoCD, Grafana, Prometheus, Sentry, OpsGenie.
You could be a perfect match for the position if
You want to:
- Design and implement scalable and efficient services for a web scraping system using Python and relevant libraries.
- Lead the development team in overcoming anti-scraping measures such as CAPTCHAs, rate limiting, and dynamic content loading.
- Mentor and guide junior engineers in the team, fostering their growth and development.
- Automate existing scraper development processes using tools and frameworks to improve efficiency and maintainability.
- Monitor system performance and proactively identify and address any issues.
- Design and implement microservices architecture for modularity and scalability.
- Apply Domain-Driven Design (DDD) principles to ensure clean and maintainable code.
- Stay up-to-date with the latest advancements and best practices in web scraping.
You have:
- 5+ years of experience as a Software Engineer with a strong track record in web scraping.
- Proven expertise in Python and relevant libraries like Scrapy, Selenium, Pyppeteer, Playwright.
- In-depth understanding of web protocols and how modern web works (SSL, proxies, cookies, headers, browsers).
- Experience with anti-scraping detection and bypass techniques.
- Experience with Kafka, Redis, and microservices architecture.
- Familiarity with Domain-Driven Design (DDD) principles.
- Eng - interm +
Pleasant extras:
- Experience with cloud platforms like AWS or GCP.
- Experience with data quality and cleaning techniques.
- Knowledge of machine learning and data analysis techniques.
Soft skills:
- Ability to lead and mentor junior engineers in overcoming technical challenges.
- Proactive approach to identifying and solving problems, seeking out new opportunities.
- Ability to work independently and as part of a team in a fast-paced environment.
- Ability to learn new technologies and adapt to changing web scraping landscapes.
You’re gonna love it, and here’s why:
- Rich innovative software stack, freedom to choose the best suitable technologies.
- Remote-first ideology: freedom to operate from the home office or any suitable coworking.
- Flexible working hours (we start from 8 to 11 am) and no time tracking systems on.
- Regular performance and compensation reviews.
- Recurrent 1-1s and measurable OKRs.
- In-depth onboarding with a clear success track.
- Competera covers 70% of your training/course fee.
- 20 vacation days, 15 days off, and up to one week of paid Christmas holidays.
- 20 business days of sick leave.
- Partial medical insurance coverage.
- We reimburse the cost of coworking.
Drive innovations with us. Be a Competerian.
