Workaline

Scrapinghub

Python Developer - Web Crawling

Scrapinghub published a month ago

N/A

Mid-Level, Senior, Contract

No office location

Location Availability

BETA
m 2015 05 12 traveling tips for remote workers rel nofollow wherever you want a . have the opportunity to go to conferences and meet
pportunity to go to conferences and meet with the team from across the globe . get the chance to work with cutting edge open source t

Python Developer - Web Crawling

Scrapinghub | No office location
Remote

About this job

Job type: Contract
Experience level: Mid-Level, Senior
Role: Data Scientist
Industry: Big Data, Data Science
Company size: 51-200 people
Company type: Private

Technologies

Job description

About the Job:

Scrapinghub is looking for software engineers to join our Professional Services team to work on web crawler development with Scrapy, our flagship open source project.

Are you interested in building web crawlers harnessing the Scrapinghub platform, which powers crawls of over 3 billion pages a month?

Do you like working in a company with a strong open source foundation?

Scrapinghub helps companies, ranging from Fortune 500 enterprises to up and coming early stage startups, turn web content into useful data with a cloud-based web crawling platform, off-the-shelf datasets, and turn-key web scraping services.

Job Responsibilities:

  • Design, develop and maintain Scrapy web crawlers
  • Leverage the Scrapinghub platform and our open source projects to perform distributed information extraction, retrieval and data processing
  • Identify and resolve performance and scalability issues with distributed crawling at scale
  • Help identify, debug and fix problems with open source projects, including Scrapy

Scrapinghub’s platform and Professional Services offerings have been growing tremendously over the past couple of years but there are a lot of big projects waiting in the pipeline, and in this role you would be a key part of that process. Here’s what we’re looking for:



About you:

  • 2+ years of software development experience.
  • Solid Python knowledge.
  • Familiarity with Linux/UNIX, HTTP, HTML, Javascript and Networking.
  • Good communication in written English.
  • Availability to work full time.

Bonus points for:

  • Scrapy experience is a big plus.
  • Familiarity with techniques and tools for crawling, extracting and processing data (e.g. Scrapy, NLTK, pandas, scikit-learn, mapreduce, nosql, etc).
  • Good spoken English.

Hiring Process:

Stage 1: Technical trial project

Stage 2: Interview with HR Representative 

Stage 3: Technical Interview

Life at Scrapinghub

About Scrapinghub

Scrapinghub is a fast growing and diverse technology business turning web content into useful data with a cloud-based web crawling platform, off-the-shelf datasets, and turn-key web scraping services.

We’re a globally distributed team of 170 Shubbers working from over 30 countries who are passionate about scraping, web crawling, and data science.

As a new Shubber, you will:

Become part of a self-motivated, progressive, multi-cultural team.

Autonomy to make the role your own, supported by great people

Join a team with huge opportunity to make a difference

Have the freedom to work from wherever you want.

Have the opportunity to go to conferences and meet with the team from across the globe.

Get the chance to work with cutting-edge open source technologies and tools.

Benefits

  • Flexible working hours
  • Remote working
  • Paid time off
  • Paid open source work
  • Global team meet ups
  • Learning & Development Opportunities

Joel Test

Source control
One-step build
Daily builds
Bug database
Bugs fixed before writing new code
Up-to-date schedule
Specs
Quiet working conditions
Best tools that money can buy
Testers
Code screening
Hallway usability testing
Learn more about Scrapinghub
Python Developer - Web Crawling at Scrapinghub