Before you keep reading...

Runestone Academy can only continue if we get support from individuals like you. As a student you are well aware of the high cost of textbooks. Our mission is to provide great books to you for free, but we ask that you consider a $10 donation, more if you can or less if $10 is a burden.

Support Runestone Academy Today

14.11. Code writing activity part 3

On this page, you will complete the final activity to write code that:

Scrapes all the comments on the Rate My Professor page for Prof. Ericson and Prof. Oney and prints them

Here is the link to Prof. Ericson's Rate My Professor page.

Here is the link to Prof. Oney's Rate My Professor page.

You can see that both the pages have the same layout.

The comments all have the same tag name, which is 'div' tag with class='Comments__StyledComments-dzzyvm-0 dvnRbr'. Here’s what it looks like when you inspect Prof. Ericson’s page:

Links to plans

Plan 2: Get a soup from a URL

Plan 3: Get a soup from multiple URLs

Plan 4: Get info from a single tag

Plan 5: Get info from all tags of a certain type

Plan 9: Print info

Here is the code that you assembled from the plans. Now that you’ve assembled the correct plans, fill in the blanks to complete the code.

#Get the webpage # Load libraries for web scraping from bs4 import BeautifulSoup import requests # Get a soup from multiple URLs base_url = '___URL_goes_here___' endings = ['_endings_', '_go_', '_here_'] for ending in endings: url = base_url + ending r = requests.get(url) soup = BeautifulSoup(r.content, 'html.parser') # Get all tags of a certain type from the soup tags = soup.find_all(____tag_description_goes_here___) # Collect info from the tags collect_info = [] for tag in tags: # Get info from tag info = tag.______ collect_info.append(info) #Do something with the info # Print the info print(______)

In solving the preceding problem I invested:
• 1. Very, very low mental effort
• 2. Very low mental effort
• 3. Low mental effort
• 4. Rather low mental effort
• 5. Neither low nor high mental effort
• 6. Rather high mental effort
• 7. High mental effort
• 8. Very high mental effort
• 9. Very, very high mental effort

Before you keep reading...

Making great stuff takes time and $$. If you appreciate the book you are reading now and want to keep quality materials free for other students please consider a donation to Runestone Academy. We ask that you consider a $10 donation, but if you can give more thats great, if $10 is too much for your budget we would be happy with whatever you can afford as a show of support.

Support Runestone Academy Today

You have attempted 1 of 4 activities on this page