ABOUT THE ROLE
For one of our partners we are looking for a Web Scraping Expert. A savvy specialist of crawling systems, that is capable of ensuring a constant flow of "big data" from sources in all corners of the world.
SOME MORE INTERESTING PROJECT FACTS
You'd be joining an international company (5 continents, 22 countries, 150+ customers, 5000+ users) on a mission of transforming big data into crucial insights. They employ a large scale data acquisition process and develop a platform of enriched data to detect and prevent fraud and financial crime.
Your main task would be to ensure an uninterrupted flow of data from the various sources they are crawling. You would be working closely with the data team: advising them and identifying the issues they run into. Together you'd be building tools that tackle existing impediments and develop techniques and software solution that increase the data yield and quality.
More specifically, they currently run a Java-based crawling system that uses various strategies for crawling the web. Your job would be to understand this system, understand its limitations and come up with solutions that will improve the success rate of the crawlers.
You must therefore have demonstrable experience with crawling systems and frameworks.
It would be great if you could tick off these boxes as well:
- Professional experience working with crawling systems/frameworks (Scrapy, Selenium, headless browsers, Puppeteer etc)
- Good understanding of how the web works (HTTP, SPAs, etc.)
- Solid knowledge and production experience with Python or Java
- BSc or MSc in computer science, data science, engineering or similar discipline
- Fluent in Docker
- Experience writing async code
- Experience with distributed crawling systems
The teams are located in Helsinki, Oslo and Cluj-Napoca. They would prefer that the Web Scraping Expert was based in either Helsinki or Cluj. A good cultural fit is the most important attribute for this role. Their virtual workplace enables a team member to employ a self-motivated, disciplined and highly responsive approach in achieving team success.