14.3. Plan 2: Get a soup from a URL¶

14.3.1. Plan 2: Example¶

The first step in web scraping is getting information from a webpage. To use the BeautifulSoup web scraping library, we have to put the webpage into something called a soup.

Here is the code for getting a soup from the Cottage Inn location page.

Goal: Get a soup from one webpage
# Load libraries for web scraping
from bs4 import BeautifulSoup
import requests
# Get a soup from a URL
url = 'https://cottageinn.com/pick-a-location/'
r = requests.get(url)
soup = BeautifulSoup(r.content, 'html.parser')

14.3.2. Plan 2: When to use this plan¶

Use this plan when you want to scrape one webpage.

14.3.3. Plan 2: How to use this plan¶

Replace the URL with the URL of the website you want to scrape.

A URL is a web address, like you see in your web browser. It should be complete (starting with http:// or https://). In this plan, a URL should be surrounded by quotes (' ').

Copying a URL from the Cottage Inn location page

14.3.4. Plan 2: Exercises¶

Q-1: If you wanted to get a soup from the MDen homepage instead of the Cottage Inn location page, which part(s) of the code below would you change? Click on those part(s) of the code.Check out the example of this plan above to identify the area that should be changed.

# Load libraries for web scraping
from bs4 import BeautifulSoup
import requests

# Get a soup from a URL
url = 'https://cottageinn.com/pick-a-location/'
r = requests.get(url)
soup = BeautifulSoup(r.content, 'html.parser'):

Before you keep reading...

Runestone Academy can only continue if we get support from individuals like you. As a student you are well aware of the high cost of textbooks. Our mission is to provide great books to you for free, but we ask that you consider a $10 donation, more if you can or less if $10 is a burden.

Support Runestone Academy Today

Note

Click here to go back to the Cottage Inn example

You have attempted 1 of 3 activities on this page