Python Developer (Web Scraping)
Scrapinghub is looking for software engineers to join our Professional Services team to work on web crawler development with Scrapy, our flagship open source project.
Are you interested in building web crawlers harnessing the Scrapinghub platform, which powers crawls of over 3 billion pages a month? Do you like working in a company with a strong open source foundation? Scrapinghub helps companies, ranging from Fortune 500 enterprises to up and coming early stage startups, turn web content into useful data with a cloud-based web crawling framework, off-the-shelf datasets, and turn-key web scraping services.
Join us in making the world a better place for web crawler developers with our team of top talented engineers working remotely from more than 30 countries.
RESPONSIBILITIES
- Design, develop and maintain Scrapy web crawlers
- Leverage the Scrapinghub platform and our open source projects to perform distributed information extraction, retrieval and data processing
- Identify and resolve performance and scalability issues with distributed crawling at scale
- Help identify, debug and fix problems with open source projects, including Scrapy
Skills & Requirements
Scrapinghub's platform and Professional Services offerings have been growing tremendously over the past couple of years but there are a lot of big projects waiting in the pipeline, and in this role you would be a key part of that process. Here's what we're looking for:
- 2+ years of software development experience
- Solid Python knowledge
- Familiarity with Linux/UNIX, HTTP, HTML, Javascript and Networking.
- Good communication in written English, Availability to work full time
- Scrapy experience is a big plus.
- Familiarity with techniques and tools for crawling, extracting and processing data (e.g. Scrapy, NLTK, pandas, scikit-learn, mapreduce, nosql, etc)
- Good spoken English
WHAT YOU GET
- Freedom to work from wherever you want.
- A chance to work with smart and self-motivated peers.
- The opportunity to go to conferences and meet with the rest of the team.
About Scrapinghub
Scrapinghub is a startup with the goal of providing the best web scraping technology.
We currently provide services for running Scrapy web crawlers, storing and searching crawled data, visualizing the crawl process, automatic information extraction (based on supervised learning) and a proxy network for routing requests. We also develop open source libraries for web crawling and information extraction.
Our clients are from a diverse range of industries, they're usually technical and build very interesting products with the data and services we provide.
This is an opportunity to join at an early stage where you can have a huge impact on the success of the company.
Desired Skills
Contact Info
- Company Website: https://scrapinghub.com/
