Before you keep reading...

Runestone Academy can only continue if we get support from individuals like you. As a student you are well aware of the high cost of textbooks. Our mission is to provide great books to you for free, but we ask that you consider a $10 donation, more if you can or less if $10 is a burden.

Support Runestone Academy Today

Before you keep reading...

Making great stuff takes time and $$. If you appreciate the book you are reading now and want to keep quality materials free for other students please consider a donation to Runestone Academy. We ask that you consider a $10 donation, but if you can give more thats great, if $10 is too much for your budget we would be happy with whatever you can afford as a show of support.

Support Runestone Academy Today

14.13. Code explaining activity

Look at the code below, and try to determine what it does.

14.13.1. Relevant tags

Here’s the relevant tag from https://www.si.umich.edu/people/barbara-ericson:

# Load libraries for web scraping
from bs4 import BeautifulSoup
import requests
# Get a soup from a URL
url = 'https://www.si.umich.edu/people/barbara-ericson'
r = requests.get(url)
soup = BeautifulSoup(r.content, 'html.parser')

# Get all tags of a certain type from the soup
tags = soup.find_all('a', class_='item-teaser--heading-link')
# Collect info from the tags
collect_info = []
for tag in tags:
    # Get info from tag
    info = tag.get('href')
    collect_info.append(info)

# Get a soup from multiple URLs
base_url = 'https://www.si.umich.edu/'
endings = collect_info
for ending in endings:
     url = base_url + ending
     r = requests.get(url)
     soup = BeautifulSoup(r.content, 'html.parser')

     # Get all tags of a certain type from the soup
     tags = soup.find_all('p')
     # Collect info from the tags
     collect_info = []
     for tag in tags:
         # Get info from tag
         info = tag.text
         collect_info.append(info)

         # Print the info
         print(collect_info)

Q-1: Write down your best guess of what the code does.

Links to plans

Plan 2: Get a soup from a URL

Plan 3: Get a soup from multiple URLs

Plan 4: Get info from a single tag

Plan 5: Get info from all tags of a certain type

Plan 9: Print info

You can run the code below and see what happens.

#Get the webpage # Load libraries for web scraping from bs4 import BeautifulSoup import requests # Get a soup from a URL url = 'https://www.si.umich.edu/people/barbara-ericson' r = requests.get(url) soup = BeautifulSoup(r.content, 'html.parser') #Extract info from the webpage # Get all tags of a certain type from the soup tags = soup.find_all('a', class_='item-teaser--heading-link') # Collect info from the tags collect_info = [] for tag in tags: # Get link from tag info = tag.get('href') collect_info.append(info) #Do something with the info # Get a soup from multiple URLs base_url = 'https://www.si.umich.edu/' endings = collect_info for ending in endings: url = base_url + ending r = requests.get(url) soup = BeautifulSoup(r.content, 'html.parser') # Get all tags of a certain type from the soup tags = soup.find_all('p') # Collect info from the tags collect_info = [] for tag in tags: # Get text from tag info = tag.text collect_info.append(info) # Print the info print(collect_info)

In solving the preceding problem I invested:
• 1. Very, very low mental effort
• 2. Very low mental effort
• 3. Low mental effort
• 4. Rather low mental effort
• 5. Neither low nor high mental effort
• 6. Rather high mental effort
• 7. High mental effort
• 8. Very high mental effort
• 9. Very, very high mental effort

You have attempted of activities on this page