Discussion on Scrapes - Automatic web content crawler and auto post plugin for WordPress

Octolooks

Octolooks supports this item

Supported

This author's response time can be up to 1 business day.

396 comments found.

Hello,

just wanted to ask if the Scrapes plugin is fully compatible with Woocommerce. I’ve done some scraping from a car sales website and I wanted to get some individual details like transmission, colour, make, model and want these to be filterable in front-end so my process would be to Scrape those info directly into product attributes. the plugin is able to get the basic fields like title, prices, contents, featured image but does not have the ability to scrape directly into product attributes? is there a tutorial available for this kind of process?

Cheers!

Hello rockstar21788,

As it written in our related F.A.Q article https://codecanyon.net/item/scrapes-web-scraper-plugin-for-wordpress/18918857/faqs/28554 grouped, variable and downloadable product types and product attributes are not fully supported currently. But it’s one of the topic that we are working on.

Thanks for your interest, Regards.

webgigant

webgigant Purchased

How can I parse the contents of the entire section of the site, if it navigates in three levels. The first level is an alphabetical list (http://shot.qip.ru/00NqQk-4SwgAVQdD/) The second level is a list of necessary articles with page navigation (http://shot.qip.ru/00NqQk-1SwgAVQdE/) The third is the article I need.

Can I do a job for one parser, or do I need to do a new job for each letter?

Hello webgigant,

If there is a listing page exists that lists all the links including A to Z, you can set this page as a source url, if not as you said you need to set a new job for each one by duplicating the tasks.

Best regards.

Hi Ocolooks, Great Plugin.

Is is possible that we can make encode product url.

Thanks

Hello kronit,

Thank you for your kind comment, you can encode url in post content area by using our template section and write some javascript code.

But we wonder your use case, can you tell us a little bit more why you want to encode the source url?

Also you can share us the url and your use case, both from comments or via private message from http://codecanyon.net/user/octolooks#contact

Best regards.

Private message send. Waiting for your reply.

Regards

Replied

tininho

tininho Purchased

This comment is currently being reviewed.

Hello tininho,

We have investigated your website, the products are loaded via javascript therefore as we described in our F.A.Q. section the content is shown empty as the screenshot you have shared us. You can test websites by disabling javascript on your browser too. For chrome we suggest quick javascript switcher extension it is very handy.

https://codecanyon.net/item/scrapes-web-scraper-plugin-for-wordpress/18918857/faqs/28549

By the way please do not share your purchase code public. So we have flagged your comment.

Best regards.

tininho

tininho Purchased

Oh sorry, I did not realise that the support request went to comments. My bad.

No problem, Envato’s support menu is kind of deceptive sometimes, we are aware of the issue :)

Happy scrapes!

Hello , PreSale Questions for this fantastic plugin :
1- Is it possible to set the post featured image without downloading on my server like WP Automatic Plugin which make it possible through Nelio External Featured Image Plugin
2- Is it possible to grap youtube videos from specific channel ?
Thanks in Advance

Hi GalaxyMarket,

1)Yes you can, it is just a custom field _nelioefi_url and you can set it via our plugin 2)You can grab those videos but embed the code, our plugin does not download the video content in your media gallery if this is your case.

If you have further questions don’t hesitate to ask us.

Kind regards.

How I can fetch the hyperlink of an image ? if possible then can I use regexp to exclude all text and grab only the ID ?

At the end of post options click on add new custom field, then you can use find and replace rules: http://prntscr.com/ep7gua For example:
Find: [^\d]
Replace: leave empty

This way all non digit characters will be removed.

Amazing.. now I feel the Power :D ..

one last thing.. in Ali Express example, can I grab the product colors and post them as options with images ?

We’re glad to hear that :)

Unfortunately we do not fully support options in WooCommerce as we described in our f.a.q section : https://codecanyon.net/item/scrapes-web-scraper-plugin-for-wordpress/18918857/faqs/28554

Kind regards.

When i scrap a page, it pulls through like 20 posts, however gets stuck in the waiting or preparing loop afterwards and when i have unlimited selected it does not pull through unlimited only 20 or so?

Replied your pm. Thank you.

replied

JD732

JD732 Purchased

i am using scrapes on a youtube, following the steps in your youtube account and my featured doest not automatically load. and when i try to select i, it comes out black like nothing is there. is there something i can do or i am doing wrong.

Hi JD732,

You can this xpath for featured image part:
//meta[@property="og:image"]/@content

Kind regards.

Hello,

I wanted to know when will Product attributes be supported in scrapes and also, every time I scrape a product image gallery the images are duplicated and displayed in the product gallery, why is scrape plugin doing that?

Thank you for your support

Hello gtellez1234,

There is no absolute date about, we are looking a technical way to implement it. About the gallery images, if you can send us a temporary login information and the task ID from http://codecanyon.net/user/octolooks#contact we can check for details.

Best regards.

webgigant

webgigant Purchased

I have a problem. If the “Total posts” field is set to “Unlim”, everything works without problems and parses the entire list of articles.

If you put a restriction – for example, 5 articles, then the article is not pars.

Where is the problem?

I do not have problems to do parsing without limits.

But I need to parse several articles gradually.

We couldn’t understand why you need this kind of scraping but we have to ask that if we understand your need correctly.

You want to scrape like 1-3 articles in one run, and in the second run (2 hours later) you want 3-6 and so on? If this is your case there is no option in our plugin to scrape like this.

Thank you

I am running a multi-site wordpress install – and i want my MAIN site to update with post from local sites that uses the same theme, plugins etc—Would this also fetch the images on the page that uses a gallery plugin like Justified Image Grid?

Unfortunately not, it is not suitable for scraping as we described in our f.a.q section: https://codecanyon.net/item/scrapes-web-scraper-plugin-for-wordpress/18918857/faqs/28549

Our best.

Can you or anyone recommend a gallery that does not use Javascript, and will be able to be “scaped”?

Actually we are not expert at WordPress gallery plugins. But target websites have very different structures, some galleries also use javascript to populate images but the sources are in the html but not displayed etc.

In that case Scrapes can get the images because they exist in the html.

Thank you.

Scrapes always ask license purchase code. The plugin always stuck on a certain loop and ask license suddenly. I Install it on WordPress multisite.

Hello vatih,

Can you share us your purchase code and a temp admin via http://codecanyon.net/user/octolooks#contact ? So we can check it

Thank you.

I have a delayed publication plug-in. When working with it, the scrapper begins to parse the article several times. – http://shot.qip.ru/00NqQk-2SwgAVQfA/ How this escape?

Hello webgigant,

As far as we remember you have chosen title for uniqueness right? We check the posts with the title excep trash status. May be the other plugin is making a copy of new posts? When it is time it deletes that post etc. we can not be sure about that.

In order to understand that, if you have SQL access (like phpmyadmin) you can run this query and check if that post is created via our plugin or not:

select meta_value from wp_postmeta where meta_key = ‘_scrape_task_id’ and post_id = YOUR_POST_ID;

You can change YOUR_POST_ID field with the two post ids you want to check. If empty result is returned then it means it is not created by our plugin.

Hope this helps.

Kind regards.

hello i purchase alrdy, i want to ask if i use on domain1.com . can i change that domain , if i want to use another domain ?

Hello richardnatanael10,

For cookie usage you can watch our Facebook tutorial video on our product page. Since login systems differ a lot we can not just say use this or that etc.

On template calculate function if you want to take 15% of something you can use it like this calc([scrape_value] * 0.15)

Kind regards.

it is still running if i close my wordpress ?

Yes scrapes tasks are always running in the background.

Hi, I just bought your plugin. I just need the title to appear on the date. But in its pluign appears date and time. How do I show up just like this: 29/03/2017 I’m from Brazil

We guess this is your website? Can you send us the target website? Thank you.

Hi marcioperuchi,

We’ve looked that detail page but there is no explicit date area at the content as we see. DIA 29 – QUARTA-FEIRA we have translated this text from google translate , it means day 29 wednesday, but there is no month or year information. Sadly our date selector can not grab the information

If you see any other date related field, you can share us the screenshot so we can write xpath or regex rules to grab the date.

Kind regards.

Hello,

I need to scape a website where each post have files attached to it (files can be image, pdf, doc …) Is it possible to do it with your plugin ? Thank you

by
by
by
by
by
by