UniqWorks

UniqWorks supports this item

Supported

40 comments found.

Hi, is it possible to scrape details from a website with login and password, i do have the login details. Thanks in advance

Hi there!

Thank you for using our plugin! Unfortunatly that’s not possible at the moment.

Warning: curl_setopt() [function.curl-setopt]: CURLOPT_FOLLOWLOCATION cannot be activated when safe_mode is enabled or an open_basedir is set in /home/maxora/gayone.org/wp-content/plugins/wp-web-grabber/wp-web-grabber.php on line 272

Warning: curl_setopt() [function.curl-setopt]: CURLOPT_FOLLOWLOCATION cannot be activated when safe_mode is enabled or an open_basedir is set in /home/maxora/gayone.org/wp-content/plugins/wp-web-grabber/wp-web-grabber.php on line 272

i i’m not a php programmer, i don’t know how to do. i need your help just to hide that error ( with css ) or remove.

OK, try this:

Go to the plugin folder and open the file “wp-web-grabber.php” in your text editor. Then add the following code at line 11:

error_reporting(E_ALL ^ E_WARNING);

Try that and let me know if you still get the error.

thanks a lot, its working.

hginli

hginli Purchased

Hello

i scraping a site it html have live link

i can Disable it live link at my site? i want change it to dead link

thank you very much

Hi there!

The easiest way to do that is using CSS to change the appearance of links, so they look like normal text. Try the following settings:

CSS Class: dead-link

Custom CSS: .dead-link a {text-decoration:none; color:black; cursor:default; border:0}

Note that this might be bad for SEO. A safer solution is using JavaScript to completely replace the links.

Can you scrape linked listing? If they page im crawling has links to other pages with full descriptions.

Hi,

This plugin scrapes only the HTML in the given webpage. It does not follow links on the page to scrape them. If you need to scrape the HTML of the links, you’ll have to do it separately.

suge1w

suge1w Purchased

Hi

How do I change / set the crawling speed? I want to change to once a week.

Hi!

To set the crawl time to once a week enter 10080 in the “cache time” field.

suge1w

suge1w Purchased

Thanks for the reply! Where is the “cache time” field?

suge1w

suge1w Purchased

Nevermind, found the field!

How is this compatible with WooCommerce? I ask because I want to scrape product profiles from a site and display them on mine. I would love it if the products scraped would populate WooCommerce product profiles.

You can use this plugin in places that support shortcodes. This means that you can use it in WooCommerce product description but not in the price or weight field.

Hi before I purchased, can I ask whether the hyperlinked text that i scrape from the news site can be linked to the news site once user clicked?

And can I add the source together with the scraped news title?

Hi, you can’t change the hyperlink using this plugin. You’ll have to use JavaScript code for that. But the plugin can automatically convert relative links to absolute links for you.

WebGrabber uses shortcodes to insert the scraped content into your page. That means you can add any source/link after or before the scraped content in the text editor.

Hi i would like to scrape the news title with outbound link to the external news site. Can you help to customise it? How much does it cost?

Yes I can help you :) Please send me more detail about the customizations you need to my email address: info.uniqworks@gmail.com

How would this scrape directory lists on several pages in same format, like business’s name, phone, addres, email, etc? Who the hell knows what an css selector is and even what code to use? Not eveyone here is a coder!

Hello,

i want to build a form using gravity forms, in that form there will be a field for entering a website.

the form would have other fields too, can i use your plugin to hook into the gravity forms fields and populate them dynamically based on the output from the parsed content from the plugin?

please if there’s any way to explain more, please tell me how to contact your directly :)

Hi,

You can contact me through the contact form in my profile. You can use WebGrabber anywhere that the WordPress text editor is accessible. If the gravity forms has a field that supports text editing tools, then you can use WebGrabber to grab data and display it in that field.

I hope that answers your question. :)

Great plugin, 5 stars! One question, how can you make links within scraped content open in new tab?

Thanks! :) You can rate the plugin here: http://codecanyon.net/downloads

I’ll add an option for that in the next update. For now, you can use JavaScript to make the links open in new tab. Follow these steps:

1- install the embed-code plugin:

https://en-gb.wordpress.org/plugins/embed-code/

2- Add the following code in the Footer Code field:

<script>
var linkList = document.querySelectorAll('.css-selector a');
for(var i in linkList){
    linkList[i].setAttribute('target', '_blank');
}
</script>
3- replace ”.css-selector” in the code with your own CSS selector.

I will try this later today, thank you for getting back to me so promptly.

Is there any way to limit the amount of scraped content displayed? I would like to display only the last five records of a specific css selector instead of all records

You could use the nth-child selector. For example, this code retrieves the 3rd and 4th paragraphs of a page:

p:nth-child(3), p:nth-child(4)

Is this plugin compatible with WP Job Manager? https://wpjobmanager.com/

Unfortunately no :(

It works where ever the editor is? For example would it work on: http://www.wpallimport.com/

Yes it works in most places that the default WordPress text editor is accessible. I haven’t tested it on wpallimport though.

Hi , Can i use your plugin in custom field values? The custom fields will be inturn embeded into the templates which shows the ouput.Possible?

Yes, the plugin works in custom fields as well.

Hi.

I would like to pull recursive (recurring) content from TripAdvisor which is the reviews. There are many reviews on the website so I want to be able to pull the content and display it like comments. Is this possible?

Thanks!

Hi there!

You can scrape one group of comments at a time. If there are multiple list of comments, you need to create a shortcode for each list.

Hi

works fine at all, but there are some stores the plugin didn’t function with them, any idea ?

Hi Sanayar!

Some sites use JavaScript to dynamically load content. There are also sites that block scrapers/robots to prevent them from accessing their content. Unfortunately, WebGrabber cannot scrape these sites.

Hi, does it stores images locally? If yes, can them be fetched with different name? Thanks.

Hi,

No, images are not stored locally. They remain on the site and WebGrabber links to them.

Hi, can scrap a price comparison website, like one product have multiple affiliate price’s?

Yes, you can scrape most price comparison websites, but not all of them. Check the live demo to test the website that you want to scrape.

Hi. I like your plugin. I need it only for scraping prices from online stores and I want to create price comparison page of some product.

So, on my page there will be table for one product with list of offers (prices) from several online stores. But I need to know what will happen when for example one of scraping price of product from some store disappear? I mean that product in some store stops selling, or store delete page with such product. In such case on my comparison price page I will see empty price for such product?

I need very much to know of every case, when some scraping price will be empty. Is there any function in your plugin that will let me know for example by email, that scraping prices are empty? If not, please, can you add such function?

Thank you, David

Hi David,

I implemented those features and sent the plugin to your email address on August 3 (davidki@*m.cz) Have you checked your email?

davidki11

davidki11 Purchased

Thank you. I had to overlook that email. Yesterday I deleted all emails for the last half year (Lots of spam emails). Can you send me it once again, please? And will you add this options also into your plugin for every users? I think that it will be useful.

Thank you, David

No problem :)

I just sent you the plugin again. Check your email. Yes, I will add them to the plugin in the next update so everyone can use them.

Hi,

I tested the demo and like it.

I have some questions :

[1] One of my target sites to be scrapped has the price data in the js variable, not in html.

i.e.

<script> var vueSupport = { title: “xxx”, price : “1000” }; <script>

Is it possible if we modify the plugin to get that value (i.e. using RegEx maybe) ?

[2] For the caching, how do you store the cache? Do you have a new table for it? Is the primary key of the cache built from the url of the product?

[3] Is it possible to print the latest update date/time of that scrapped data?

Thanks

Hi there!

Thanks for checking out our plugin.

[1] You can scrape the entire script if it is inline, but you cannot choose what part of the script to keep because the plugin can only parse html elements.

[2] The data is cached in the WordPress db and the key for it is made of the url and all selected options.

[3] Unfortunately no.

Hi,

Thanks for your reply.

For [1], as I’m a developer, I’d like to know that it’ll be possible if I modify it to use RegEx ?

Does the lib you’re using support RegEx selector ? (If not, do you have any guide for this solution)

Thanks

Yes, if your PHP skill is good enough, you can modify the plugin to use regex.

I use simple-html-dom and phpQuery for HTML parsing, and neither supports regex. You can replace them with another library, but I don’t have a guide for that :)

Could this get java data from sites like investing.com or yahoo finance of their forex in real time? Example investing.com has java tabs of currencies that gets updated in real time at https://www.investing.com/technical/technical-studies. lmk

Yes, I just tested it and it worked perfectly. Note, however, that the data gets updated only when you refresh your page.

by
by
by
by
by
by