Solution Architect - Web Crawling

Scrapinghub published 5 months ago


Mid-Level, Senior, Full-time

No office location

Location Availability

m 2015 05 12 traveling tips for remote workers rel nofollow wherever you want a . have the opportunity to go to conferences and meet
pportunity to go to conferences and meet with the team from across the globe . get the chance to work with cutting edge open source t

Solution Architect - Web Crawling

Scrapinghub | No office location

About this job

Job type: Full-time
Experience level: Mid-Level, Senior
Role: Data Scientist
Industry: Big Data, Data Science
Company size: 51-200 people
Company type: Private


Job description

About the Job:

We are looking for a motivated Solution Architect to further accelerate the rapid growth Scrapinghub has seen in recent years.

As a Solution Architect, you will be in charge of the requirement analysis phase of our Professional Services projects. You will gather the customer requirements, analyze and provide technical solutions to our customers and our internal teams for development and delivery. You will be pivotal in translating customer and technical project requirements across various internal and external teams.

Job Responsibilities:

  • Meet with customers for requirement collection and provide technical consultation and advice.
  • Deconstruct raw business / customer requirements and identifying the actual business problems the customer is trying to solve.
  • Translate customer requirements into a viable business solution, execute the necessary; conduct deep technical investigation, propose the best approach and solution inclusive of effort, cost estimates, and occasionally project timelines which fit the customer’s business goals and expectations.
  • Build Proof of Concepts, to showcase technical expertise in data acquisition projects, assess feasibility, refine scope, and achieve project collaboration and engagement
  • Mentor developers to build a strong team of Solution Architects.
  • Ensure you are up-to-date on Scrapinghub products and service offerings which will enable you to provide an architectural solution that includes the latest capabilities and provide value to our customers
  • Create, write and present customer proposals and ensure it completely captures the technical requirements for the solution proposed
  • Take ownership of learning upcoming product and service capabilities by requesting training sessions from the concerned team.
  • Work with subject matter experts to provide better solutions.

About you:

  • You have solid knowledge and understanding of aspects of web crawling using Python and Scrapy.
  • You have strong client and customer facing experience - minimum of 3 years experience in a similar position within a technical environment
  • You have excellent communication skills, both written and verbal, and are comfortable in document writing, both technical and non technical
  • You possess excellent presentation skills and experience in articulating technical and business solutions to both technical and non technical audiences.
  • You have the ability to articulate and convey technical solutions effectively and clearly to internal teams and stakeholders.
  • You have the zeal to understand customer business requirements really well by asking the right questions.
  • You have the confidence to assert and justify the technical feasibility of a requirement, to make decisions, negotiate scope, and assess risk.
  • You have solid experience of mentoring teams preferably software developers.
  • You demonstrate proven knowledge and experience of estimation, planning, and project management.
  • You are well organized, self-motivated, and communicative -- an essential skill in a remote team.

Bonus points for:

  • You have experience in web crawling in other tools.
  • A proven track record of working with a remote team
  • Knowledge of and interest in web crawling, data mining and information retrieval

Life at Scrapinghub

About Scrapinghub

Scrapinghub is a fast growing and diverse technology business turning web content into useful data with a cloud-based web crawling platform, off-the-shelf datasets, and turn-key web scraping services.

We’re a globally distributed team of 170 Shubbers working from over 30 countries who are passionate about scraping, web crawling, and data science.

As a new Shubber, you will:

Become part of a self-motivated, progressive, multi-cultural team.

Autonomy to make the role your own, supported by great people

Join a team with huge opportunity to make a difference

Have the freedom and flexibility to work from wherever you want.

Have the opportunity to go to conferences and meet with the team from across the globe.

Get the chance to work with cutting-edge open source technologies and tools.


  • Flexible working hours
  • Remote working
  • Paid time off
  • Paid open source work
  • Global team meet ups
  • Learning & Development Opportunities

Joel Test

Source control
One-step build
Daily builds
Bug database
Bugs fixed before writing new code
Up-to-date schedule
Quiet working conditions
Best tools that money can buy
Code screening
Hallway usability testing
Learn more about Scrapinghub
Solution Architect - Web Crawling at Scrapinghub