Discussion on Local Google Results Scraper - PHP Class

thenextsoft

thenextsoft supports this item

Supported

19 comments found.

Great work! :-)

Thanks. :)

Great work GLWS :)

Thank you :)

this is sick awesome! thank you – gonna pick this up tomorrow . my only question does it have proxy support?

Thanks! It does have proxy support, you set the ip:port and optionally user:pass. It’s the setProxy() method on https://baileyherbert.com/google-scraper/docs/classes/googlesearch.html

What can you do with this?

Hi, this is for developers. If you know PHP you can use this to implement SERP testing in local regions.

Modern Google scrapers show results somewhat localized to the IP address of the server. This class overrides that and lets you set the location.

Hi – had a couple of issues with this code:

1. I got a 302 redirect (to com.au) when I ran SERP_test or Scrape_Results. Had to manually set setDomain to “com.au” to get it to work. Other domains work if set but leaving at the default (com) results in a 302 (I am based in sunny Australia.

2. Unless I am missing something, the scrape conditions have changed – I needed to use:

$results = $this->getTextBetween(”\”resultStats\”>”, ”<”, $html)

Instead of what was there previously to get the number of pages returned..

Can you check the scrape conditions still work fine for you ?

Thanks, Steve

Hey Steve, sorry for the troubles. The “num_results” is still working fine for me with the default configuration (tested .com and .com.au) so that’s pretty weird. I’ll try to track this bug down.

As for the 302 redirect, thanks for reporting that to me. Seems to be only google.com doing it. I will work on a patch and will have it uploaded within the next day.

Update is available now, should fix everything. Please try it and let me know.

brondo

brondo Purchased

Parse error: syntax error, unexpected ‘[’ in /home/geilo/public_html/scrope/examples/Google.php on line 13

It requires PHP 5.4+. Or you can go in the Google.php file and change the shorthand arrays [] to array(), like on line 13.

Is it possbiel to have longitute and latitude ok business and also export it ot CSV or exel ?

Unfortunately Google does not support lat/long for web search locations. You’d need to use an API to convert the lat/long into a Google-supported location.

As for export, PHP has functions to make CSV files.

Proxy Usage in mandatory. whats the difference between using proxy and without ?

Proxies are for when you need a high number of requests in a short time. Without proxies, you can only send 1 request to Google per minute – any faster and they’ll block you. Adding proxies lets you send more requests per minute.

It doesn’t appear to return Places data anymore. It is empty for every query and location I have tried.

It also is only giving results from my location, even when I set the location before calling results.

Thanks for letting me know. I will look into this when I get home. Google may have changed something due to their latest design changes they’re rolling out.

They did. I checked it out and you need to change the lines when you’re parsing google places to:

http://pastebin.com/fRBNdf8x

Thanks and sorry for the lack of response, this was fixed in an update.

carami

carami Purchased

Great work techiehut, but seems something missing. when I look the results on google they gives 10 per page and the scripts returns 9 in each page, maybe you note that?

Hi and very sorry for the delays. I’ve submitted an update with this issue fixed, it may take a few hours for CodeCanyon to approve it.

Great work the bug is now fixed. Can u do same work with yahoo?

I do plan on making scrapers for other search engines but it won’t be soon, I’m occupied with some other projects for now. :)

fergany

fergany Purchased

how much queries can i make per second using 1 proxy i want to calculate how much proxies i need to make a large volume of queries

Hi, from my testing I was able to make 1 query per 60 seconds per IP address without getting blocked.

nesttle

nesttle Purchased

Hello dear, Can we retrieve local’s phone too? Google usually includes this info like tel://123456/ Thanks in advance :)

Sorry, after looking into this, it is not possible at this time. The location-enabled page that this class scrapes for data does not include phone numbers in the map listings. Maybe in a future update I will add a way to collect the phone numbers.

help please PHP Parse error: syntax error, unexpected ‘[’ /Google.php on line 13

try change array[] to array() but still no working

302 means you’re sending requests too fast to Google and they temporarily blocked you. You should catch the exception.

awesome! thank you! how many seconds need between searches for non error???

I recommend at least 60 seconds between searches, as that’s what I use for my own projects.

Hello, I have 2 questions. 1. Can i have a try before I buy this? 2. Can your API directly return the html content for me to use? Thank you.

Awesome work, thanks. Quick question: If doing multiple pages (nexts) do you suggest waiting between?

Thanks. I don’t know exactly what Google’s internal rules are for when to block bots, but in my experience waiting 1 minute between requests has prevented me from getting blocked.

GabiXO

GabiXO Purchased

Hey! I am getting the following errors: “Notice: Undefined offset: 0 in D:\xampp\htdocs\google-scrap\class\Google.php on line 361” “Notice: Undefined offset: 0 in D:\xampp\htdocs\google-scrap\class\Google.php on line 362”

Have any idea why? Thanks!

Hey, send me the errors to bailey@sitedrive.io real quick along with your PHP version. I’ll do some investigating in the meantime.

Are you going to Captcha breaker (like Death By Captcha) into it?

Sorry, but no. This is a standalone class for scraping Google results, with no third party features.

shadsjeff

shadsjeff Purchased

Great Work!! Thanks

Well, what would you like to scrape? I have a Bing results scraper class mainly for personal use, but it isn’t localized like this one.

shadsjeff

shadsjeff Purchased

I have to scrape file hosting websites like zippyshare, mediafire etc, But I cant find a way to bypass google captcha, Is there a way to do it with Google or Bing?

And also what is the use of localization?

I don’t have any experience with bypassing captchas, sorry! I know there’s some APIs out there for it though.

Localization in this class lets you set the (actual physical) location for the query. It then does some calculations to get Google to show results as if you were actually at the chosen location. This is useful for testing ranks or results in different parts of the world.

by
by
by
by
by
by