Build an apartment rental site like Craigslist
Learn to use an API to scrape Craigslist apartment rental data.
Build a beautiful web app to display live listings with filters.
Topics covered
- Data scraping
- Ruby on Rails
- JSON
- APIs
- Databases
- Data cleaning
- Script automation
- Web development
Audience
- Intermediate Level
- Recommended prerequisite:
Web scraper course Yelpdemo
Time
- Video content:
4 hours - Average completion time:
3 week
In this course, you'll learn to gather data through an API and use that data to build a custom Ruby on Rails web app. You'll write a script in the Ruby programming language to collect and process the data you receive, then save the information to a database, and build a front-end interface for users to interact with it. By the end of the course, you'll have learned to automate the data collection process and have an app that's live on Heroku.
Chapters
-
Course Introduction (5:04)
Understand more about APIs and why they can be so powerful. See a demonstration of the final app you'll be building.
-
Using the Reference API (11:46)
Register to use the 3taps API and use its Reference API to explore the 3taps dataset.
-
Retrieving Posting Data (9:47)
Learn about the anchor system that 3taps uses to track data requests through its Polling API. Use the Polling API to retrieve our first set of real data.
-
Refining our API Request (6:46)
Add additional parameters to your data request to filter the data that you want to see.
-
Convert our Request to a Script (10:41)
Write a custom script in the Ruby programming language to submit a request to the 3taps API and view the results in the command line.
-
Create a Rake Task (6:18)
Create a new Ruby on Rails web app and learn about rake tasks. Build a custom rake task to make API requests.
-
Save Posts to Database (11:08)
Modify your custom rake task to save the retrieved data into a SQLite database in your app.
-
Clean Up Data Formats (10:49)
Add the Bootstrap framework to style elements on the page. Learn about different data types and use Rails helpers to format the display of the information.
-
Paginate the Index Page (5:20)
Use the 'will_paginate' and 'will_paginate-bootstrap' gems to divide content into multiple pages.
-
Scrape Apartment Details (12:00)
Revisit your original data request and extract additional details to save to your database. Account for possibly absent or incomplete data.
-
Display Details (3:22)
Incorporate additional data fields into the display of your web app.
-
Collect and Store Images (9:41)
Collect image URLs from your API request and store them in a separate, linked database. Display images that are hosted on an external URL.
-
Look Up Neighborhoods (6:41)
Use the 3taps Reference API to write a rake task to scrape location data into a new database.
-
Display Neighborhoods (4:35)
Revise your existing rake task to use a reference table to process data prior to saving it into a database.
-
Build an Advanced Filter (10:16)
Create a search form with multiple fields and save the search parameters in the URL. Use those to filter the results displayed on the page.
-
Improve Filter Display (6:38)
Convert text inputs in the search filter to more intuitive checkboxes and dropdown menus.
-
Structure Filter Layout (12:17)
Use Bootstrap's grid system to organize the input fields of the search filter. Revise the input field labels and add placeholder text to improve the overall look.
-
Design Homepage (12:29)
Add a homepage containing a jumbotron with a high-resolution image. Add CSS styling to give your homepage a more professional look.
-
Add Thumbnail Images (13:12)
Add a thumbnail image preview to the search results page. Use CSS techniques to resize images stored on external URLs and include a default image for post's without any uploaded images.
-
Structure Show Page (11:04)
Use the Bootstrap grid system to design the layout of the page with a post's details. Use 'if statements' to suppress the display of missing data fields.
-
Style Show Page: (7:31)
AUse CSS techniques to scale images proportionally to fit inside their columns. Apply background colors, borders, margins, and other properties.
-
Refresh Our Data (8:22)
Set up your rake task to retrieve all the latest postings when run.
-
Deploy on Heroku (6:05)
Modify your Gemfile and set up your app to save with Git, store code online with GitHub, and launch live on Heroku.
-
Automate Rake Tasks (5:51)
Write a new rake task to clean out stale data. Learn to automate both the collection and removal of data.
-
Conclusion: (1:38)
Review your progress, reflect on the concepts that have been covered, and consider some next steps.
What you'll be building
An apartment rental site for Brooklyn, NYC populated with the most recent listings scraped from Craigslist. Allow users to filter for relevant information like square footage, laundry, pets, and parking.
Launch demo
Add more awesome features
After the completeing the foundational course, head on over to our video library and learn how to implement additional features like social media integration, internationalization, and more. Take your web app to the next level.
View features library
What are you waiting for?
“I also bought other tutorials on Rails earlier and found yours to be project-oriented and straight to the point. You don't waste so much time coding but instead show the way how things can be created as the final outcome, which I really admire, especially that your explanations are easy to understand. ” - Michael J.
"The scope/angle of this course seems perfectly tailored to what I'm looking for. From the looks of it so far, your course is similar in scope to Hartl's book, but takes a far more practical, hands-on approach - also it helps a great deal to see it in action vs. trying to follow along with just a book." - Chris S.
"Studying a new programming language can be daunting given the number of tools and concepts that need interact with each other. This course tackles that issue by providing a practical approach to learning ruby on rails: you get to build your own functional website" - Emmanuel T.