14.13. Code explaining activity¶

Look at the code below, and try to determine what it does.

14.13.1. Relevant tags¶

Here’s the relevant tag from https://www.si.umich.edu/people/barbara-ericson:

# Load libraries for web scraping
from bs4 import BeautifulSoup
import requests
# Get a soup from a URL
url = 'https://www.si.umich.edu/people/barbara-ericson'
r = requests.get(url)
soup = BeautifulSoup(r.content, 'html.parser')

# Get all tags of a certain type from the soup
tags = soup.find_all('a', class_='item-teaser--heading-link')
# Collect info from the tags
collect_info = []
for tag in tags:
    # Get info from tag
    info = tag.get('href')
    collect_info.append(info)

# Get a soup from multiple URLs
base_url = 'https://www.si.umich.edu/'
endings = collect_info
for ending in endings:
     url = base_url + ending
     r = requests.get(url)
     soup = BeautifulSoup(r.content, 'html.parser')

     # Get all tags of a certain type from the soup
     tags = soup.find_all('p')
     # Collect info from the tags
     collect_info = []
     for tag in tags:
         # Get info from tag
         info = tag.text
         collect_info.append(info)

         # Print the info
         print(collect_info)

Q-1: Write down your best guess of what the code does.

You can run the code below and see what happens.

#Get the webpage
# Load libraries for web scraping
from bs4 import BeautifulSoup
import requests
# Get a soup from a URL
url = 'https://www.si.umich.edu/people/barbara-ericson'
r = requests.get(url)
soup = BeautifulSoup(r.content, 'html.parser')

#Extract info from the webpage
# Get all tags of a certain type from the soup
tags = soup.find_all('a', class_='item-teaser--heading-link')
# Collect info from the tags
collect_info = []
for tag in tags:
  # Get link from tag
  info = tag.get('href')
  collect_info.append(info)

#Do something with the info
# Get a soup from multiple URLs
base_url = 'https://www.si.umich.edu/'
endings = collect_info
for ending in endings:
    url = base_url + ending
    r = requests.get(url)
    soup = BeautifulSoup(r.content, 'html.parser')

# Get all tags of a certain type from the soup
    tags = soup.find_all('p')
    # Collect info from the tags
    collect_info = []
    for tag in tags:
        # Get text from tag
        info = tag.text
        collect_info.append(info)

# Print the info
    print(collect_info)

Before you keep reading...

Runestone Academy can only continue if we get support from individuals like you. As a student you are well aware of the high cost of textbooks. Our mission is to provide great books to you for free, but we ask that you consider a $10 donation, more if you can or less if $10 is a burden.

Support Runestone Academy Today

1. Very, very low mental effort
2. Very low mental effort
3. Low mental effort
4. Rather low mental effort
5. Neither low nor high mental effort
6. Rather high mental effort
7. High mental effort
8. Very high mental effort
9. Very, very high mental effort

You have attempted 1 of 5 activities on this page