Web scraping with Google Apps Script ~ Checking if a page contains specific text
Tadashi Shigeoka · Sat, April 22, 2017
I wrote scraping-like code in Google Apps Script to check if a page contains specific text, so I’ll introduce it.
Code Processing Flow
- Get response with UrlFetchApp.fetch
- Get HTTP response content as encoded string with response.getContentText
- Check if string contains 'Example' using String.prototype.indexOf()
Google Apps Script Sample Code
function myFunction() {
var contentText = fetchContentText("http://example.com");
if (contentText.indexOf('Example') !== -1) {
Logger.log('◯');
} else {
Logger.log('☓');
}
}
function fetchContentText(url){
var opt = {"contentType":"text/html","method":"get"};
var response = UrlFetchApp.fetch(url, opt);
var contentText = response.getContentText();
return contentText;
}
Basic scraping could be achieved with just a little code.
Based on this code, I think writing code to append judgment results to Google Sheets next would broaden the scope of use.
Reference Information
- Class UrlFetchApp | Apps Script | Google Developers
- Class HTTPResponse | Apps Script | Google Developers
- Class Logger | Apps Script | Google Developers
- String.prototype.indexOf() - JavaScript | MDN
That’s all from the Gemba.