Jeff Bailey
Written by Jeff Bailey

Learning resources, opinions, and facts about technology.

How Do I Scrape Data From a Web Page?

Reading Time: 1.9 minutes
Listen to this post

How do I scrape data from a web page? I recently found that Facebook REALLY doesn't want you to leave their platform. They don't allow you to send a message to all your friends, so you need to get crafty to make an exit.

How Do I Do It?

If you're using Google Chrome, it's straightforward. There are many ways to scrape a web page, but I'm explaining a quick and dirty solution. If you don't use Facebook, welcome to the club! Try the same method on any web page where you need obfuscated data on a web page.

  1. Navigate to: https://www.facebook.com/<Your Facebook ID>/friends
    1. Alternatively, open Facebook and click the Friends tab
  2. Scroll slowly until every friend is listed on the page.
  3. Open the vertical ... menu at the top right of Google Chrome
  4. Choose More Tools
  5. Choose Developer Tools

Once this is complete, you should have a developer console open.

Paste this code into your console.

const result = $x('/html/body/div[1]/div/div/div/div/div/div/div/div/div/div/div/div/div/div/div/div/div/div/div/div/div/div/div/a/@href')
result.map(x => {console.log(x); })

This should log all of your Facebook friend profile page URLs. Now you can save the log, and use the data to populate a CSV file and import it into LoopMessage or another tool that automates sending messages.

6. Right-click the result line and click Save as...

That's it, you've scraped a web page, and now you can make your Facebook exit!

I wish you luck getting out of Failbook land, you're going to need it!

By the way, Fuck Facebook.

References

Export data from the Chrome browser console

Web Scraping using Xpath and Chrome Extension

LoopMessage

awesome-web-scraping