Build a web scraper to collect data
Write Ruby scripts to parse Wikipedia and Airbnb.
Use technology to extract the data you need to automate manual tasks.
Topics covered
- Web scraping
- Ruby scripts
- HTML
- Parsing data files
- CSV databases
- Data analysis
Audience
- Beginner level
- Entrepreneurs
- Data enthusiasts
Time
- Video content:
2 hours - Average completion time:
1 week
In this course, you'll learn how to access a webpage's HTML and identify proper selectors for parsing through data. Write a script in the Ruby programming language to scrape Wikipedia and Airbnb and export the information into a database.
Videos
Intro to Web Scraping (2:51)
What is web scraping and how can it be used to gather and analyze data on any website? See examples of how web scraping is used to build apps.
Find Data Within HTML (6:01)
Learn to read HTML and find the appropriate tags to grab the information you need.
Create and Run Script (5:17)
Write your first script in Ruby to store HTML from Wikipedia in a variable. Install and use Ruby gems to access information from the web.
Scrape Wikipedia Tables (7:41)
Match HTML selectors in Ruby to scrape the information you need. Use Ruby methods to display data in text format.
Scrape Airbnb Listings (10:19)
Write a script to scrape different categories of information for each listing including title, price, room type, and location.
Write Data to CSV File (10:08)
Store listing information into array variables and use a loop to write the data contained in the variables into a CSV file.
Parse Data (5:55)
Split listing details into multiple subcategories by parsing the data. Clean up the data into a standardized format.
Scrape Multiple Pages (13:42)
Learn to scrape every page of listings on Airbnb by running a loop. Change the URL structure to scrape different pages.
Check Data Quality (7:19)
Conduct a data quality check in our CSV file to identify errors. Fix formatting errors by manipulating the way data is displayed.
Analyze Data (2:19)
Manipulate the data to draw insights into pricing on Airbnb. Develop an understanding of the power of scraping to automate manual tasks and gather large quantities of information on the web.
What you'll be building
A Ruby script to scrape and parse data from Wikipedia and Airbnb. Store Airbnb listings data in a CSV database to draw insights on pricing for the Brooklyn, NYC listings.
What are you waiting for?
“I also bought other tutorials on Rails earlier and found yours to be project-oriented and straight to the point. You don't waste so much time coding but instead show the way how things can be created as the final outcome, which I really admire, especially that your explanations are easy to understand. ” - Michael J.
"The scope/angle of this course seems perfectly tailored to what I'm looking for. From the looks of it so far, your course is similar in scope to Hartl's book, but takes a far more practical, hands-on approach - also it helps a great deal to see it in action vs. trying to follow along with just a book." - Chris S.
"Studying a new programming language can be daunting given the number of tools and concepts that need interact with each other. This course tackles that issue by providing a practical approach to learning ruby on rails: you get to build your own functional website" - Emmanuel T.