Code

Discussion on Scraper - Automatic Content Crawler Plugin for WordPress

Discussion on Scraper - Automatic Content Crawler Plugin for WordPress

By
Cart 2,443 sales

wpBots supports this item

Supported

This author's response time can be up to 1 business day.

1123 comments found.

Hello, no webp support? Not displaying pictures for all websites i try.

Hi, can you please let us know the exact source url from where you are trying to scrape so that we can check it?

Hello, I opened a ticket in the support center 4 days ago requesting a refund, but I am yet to receive a reply. I am unable to use the plugin. It is too complex for me and not meeting my requirements. Can someone please assist me? Thanks.

Hi, we did reply to it – https://app.screencast.com/tulctbDNWZ2Ut
Please kindly check your junk mailbox as well. Thanks!

Hi, I found some bugs on your plugin. 1) If you schedule a task and select “reset index after completed a task” it does that but not right after finished it, it does it after complete the next run.

2) Sometimes the schedules stays on the “In Queue” status for ever. So I have to run them manually.

Example of this: Task ID : d49eb983dad7f31b61fb50373bbac423 Task ID : 2f2c0e60e2d5a407837bc1cfa4434b43

Hi, regarding 2) I have two websites with ur script, same cloud server and configurations. One works fine but the other has this issue. Is there anything else that may causing this problem? Look at this task: Task ID : d49eb983dad7f31b61fb50373bbac423

regards

Hi, can you please reach out to us via support@wpbots.net so that we can look further into the issue for you? Thanks & Regards!

I Just sent you an email

Hi

Can I paste a list of url’s in and scrape the home page (title, meta description, logo image)???

I have a list of 500 websites.

Thanks

Rob

Hi, you can scrape bulk URLs at a time, but the URLs cannot be from different websites – https://app.screencast.com/vpWBvstsX4AhK

OK,thanks. sounds like that could be anew feature in the next release?

What is the purpose of the input that says “enter command” below the website preview? I am trying to remove a specific element that can’t be removed with the remove tool and I was thinking that the command could help me. Do you have any guide or info about it?

I tried with javascript but it didn’t do anything (code that I executed on chrome console and worked)

Hi, the ‘enter command’ is not for removing elements, it’s for checking xpaths. Can you please let me know the exact source url from where you are trying to scrape, and also let me know which element you are trying to remove, so that I can look into it. Regards!

Can we use this plugin to scrape the following page and display it on Wordpress page instead of post.

https://in.tradingview.com/markets/stocks-usa/market-movers-gainers/

There is a list of top gainers in a table. Can we scrape n display it with same style?

Also if the mobile users view it on mobile browser, will the content be response?

Hi, I checked it and unfortunately, could not scrape from the given URL (seems like it has bot prevention in place).

Hi! I am scraping a website, it takes all the posts of that web but when that website has new posts, the scraper doesn’t identify the new ones. I click run and it says that there are not new post. If I reset it, it works but takes all the post (even the scrapped ones)

A couple things: 1) All my posts goes directly to draft (I configured it on that way). 2) I always review the post on draft and after that I change the titles.

So, does the plugin review the titles from my wordpress database or the plugins has its own database where it checks the post titles of scraped pages (to know if it was already posted or not)?

How can I avoid the system to update posts that I personally changed the title manually? Can I edit the source code?

You’ll need to disable/uncheck the option “Check posts by title and update after changes” in the scraper task – https://app.screencast.com/dL6Rtyd201ZP1

Is there a way to scrape and import data into fields with serialized data?

Hi, currently, such feature is not there.

Pre-sale question:

Sorry, but I didn’t quite get this from the plugins description or demo.

I’m willing to create a cron job in WordPress that will scrape product names and prices from websites on regular basis.

And can the scraped data be saved in CSV or XML format with this plugin, so that it can be used (with other plugin) for price comparison?

Yes, you can scrape from given URL, on scheduled time automatically and save the data in CSV format. However, this part is not covered in our demo link https://scraper.site/visual-editor/?purchase_code=demo-account
Can you please reach out to us at support@wpbots.net so that I can provide you with more info regarding this? Thanks!

Hi, I recently purchased this software and have a few questions.

1) How can I add custom text and with links to the end of each post content field?

2) For the post category field, I need to scrape multiple categories from a single post. The target website has any number of categories per post and not a set number.

3) Are new categories created if one does not exist?

2) For the post category field, I need to scrape multiple categories from a single post. The target website has any number of categories per post and not a set number.
>> Currently, scraping and assigning multiple categories is not possible. Only one category can be assigned.

3) Are new categories created if one does not exist?
>> Yes.

Thanks for the response.

I have a follow up question to 1) which was answered by the documentation.

It seems that when using the ‘clip’ feature (by number of characters or by words) and also adding text after the content shortcode, the additional text is also clipped. I have tested this on the website.

Ideally the content should be clipped and THEN additional content added after the content shortcode.

Could you advice if I am perhaps using it wrong or if there is a way around this.

Thanks.

It’ll be clipped altogether (even the text you have added).

Plugin not working, License showing invalid, Need help urgently.

Hi, we have replied to your support ticket regarding this. Let’s connect there. Thanks!

Hello, is it possible to change the content using an AI integration? can you develop something like that?

Yes, I could scrape from the given URL – https://app.screencast.com/9iTjhrPvYx20L

on a single post works, but I think on multiple posts it doesnt

Can you please reach out to us at support@wpbots.net so that we can look further into this for you? Thanks!

Scraper Pro is our headless browser which enables AJAX functionality for dynamic sites. We will soon be adding premium proxy service in Scraper Pro (stay tuned). You can cancel it anytime. Why is this service paid?

So when to read the “Why is this service paid?” its nonsense because you do not offer the possibility to use OUR OWN browserless services

I can spin one in seconds on docker so please add a setting to use our own

Of course if the real reason to make us pay extra is “The server farm is expensive and difficult to maintain”

If its just so you can get more money from us just say it or offer the use our own server alternative.

Beside this thank you for the great plugin its really easy to use!

Hi, we appreciate your valuable feedback. I’ll check with our team if this (using own browserless services) is something that we can implement sooner or later. Thank you!

hello,with that plug in is possible to receive news from different sites into my website? using image,title and some of the text of the original post?i want to have too a link to ther original site.please answer me thanks

You can check the scraping part using our demo link – https://scraper.site/visual-editor/?purchase_code=demo-account
But importing to the site part, you cannot check there.

so where can i check if really works for my needs?

The main part is the scraping part which you can check from the demo link above, and if the preview lists out the post items correctly then there shall be issue importing them to your site.
Also, can you please let us know the exact source url from where you are trying to scrape so that we can check it?

Hi. Ive tested your demo and it seems it will work just fine for my needs. Can this automate posting ? [ so for example, i wanna scrape various web pages and if there are new posts there i would like them automatically post on my WP page ] in certain category ?

Hi, yes, you can schedule the task to run automatically on specific interval and hence automatically import the posts.

Hello, we are very interested in your plugin. We would like to know if we can get technical support from you. Naturally, for a fee, to help us set up your plugin on our site. We have specific sites that we are interested in. We would like to know if you can make us a ready-made setup (sample) so that we can download articles from the necessary sites. Please let us know if we can somehow write you a personal Message. For details. Thank You!

Hi, yes, we do provide technical support to setup plugin on your site. If you have any specific requirement then you can reach out to us at support@wpbots.net
Thanks & Regards!

Can we import variable woocommerce products?

Hi, currently you cannot import variable woocommerce products. It only supports simple products.

Hello, please, I need your help urgently.

Please, can you answer my ticket: 165918

Thanks!

Hi, thanks for contacting us. We have replied to your support ticket and resolved the issue too. Let’s connect there. Thanks!

Hi, I want to know if the featured image is also saved on my server. Thank you!

Hi, thanks for contacting us. We have replied to your support ticket and resolved the issue too. Let’s connect there. Thanks!

Hi! Pre sale question here, can i update a stock from 1 product from amazon for example (or similar market place) only update the stock to my woocommerce each our? Thank you!

Hi, thanks for contacting us. Can you please let me know exact source URL from where you want to scrape and update the item stock? Also send us the screenshot showing the stock that you want to update. Regards!

by
by
by
by
by
by

Tell us what you think!

We'd like to ask you a few questions to help improve CodeCanyon.

Sure, take me to the survey