Course Introduction
  • Introduction: Course overview and requirements
Scraping static web pages
  • What is a static web page?
  • Concept: how to scrape static web pages
  • Jsoup: the jQuery for Java
  • Example: scraping the top 10 Google search results
Scraping dynamic / AJAX web pages
  • What is a dynamic / AJAX web page?
  • Concept: how to scrape dynamic / AJAX web pages
  • Making HTTP Requests with Unirest
  • Example: scraping results from peoplefinders.com
  • Bonus: Proxycrawl to scrape dynamic pages
  • Dealing with WebSockets
Exporting your data
  • Export your data as CSV
  • Export your data as JSON
Going undercover
  • How to stay undetected
  • Bonus: Overview of Proxycrawl
  • Bonus: Example using Proxycrawl for peoplefinders.com
Conclusion
  • Conclusion and what's next
  • Bonus: Mind Map of the course content