Code

Discussion on Crawlomatic Multisite Scraper Post Generator Plugin for WordPress

CodeRevolution

CodeRevolution supports this item

Supported

This author's response time can be up to 1 business day.

120 comments found.

nwa1982

nwa1982 Purchased

Hi, Im trying to het a website restored with the waybackmachine. do you know hoe i van do that. I have the webdownloader from inspyder that i use to complete rip a site. Bit when i try IT on the waybackmachine alle the links have the URL from the archive.org in IT. Do you know some methodsbtod to get rode of Them?

Hello,

First of all, thank you for your purchase.

Please check this video tutorial, where I exemplify what you wish to accomplish: https://www.youtube.com/watch?v=8ckPB0c27GI

Regards, Szabi – CodeRevolution.

Welcome

I have a problem with Arabic When you place a shortcut %crawlomatic_0_1 %

Converting to crawlomatic_0_1% %

Hello,

In Twitomatic, this was added in update 1.9.8.1.

Please update the plugin from WordPress’s ‘Plugins’ menu (‘Check for updates’ -> ‘Update now’ for the plugin), after registering your purchase code in the plugin.

Regards.

I can not get the update because I buy Mega WordPress Bundle

You can also get the update. Please register the purchase code from the Mega Bundle in the plugin (you can register the Mega Bundle purchase code in each plugin), and you will get automatic updates for each.

Please let me know if it worked for you.

Regards.

Hey great plugin but I have no idea how to use it. Can setup for me 10 pages to craw from. I can for the help extra. Let me know. Thanks

Hello,

First of all, thank you for your purchase.

Please check the plugin’s tutorial video here: https://www.youtube.com/watch?v=7BYDa72zY7c&list=PLEiGTaa0iBIgcqNzVBaoTCS4ws47vNMuQ

Regards, Szabi – CodeRevolution.

Yes I watch the Video but still have problems to extract content. I need to extract 10 pages and I’m willing to pay for your extra support.

Please send me temp admin login to my email address kisded@yahoo.com

Regards.

can I get all the photos on a webpage article using crawlomatic? for now Im just getting one photo and a-bunch of squares with x’s in the middle.

Hello,

Please try the following config in importing rule settings:

‘Content Query Type’ -> Class ‘Content Query String’ -> entry-content-wrap

Regards.

sample page not all photos appeared on page

https://izodnews.com/2019/02/03/best-of-super-bowl-xxxvi/

Hello,

Images do not appear on this page because they are loaded dynamically, using JavaScript, after page load. The plugin cannot see directly this code, only if you install on your server the phantomjs entension and set the plugin to use it.

More details here: https://www.youtube.com/watch?v=hnEPlQSeAZE

Regards.

Hi, I would like to crawl multiple seed urls in a csv excel file, how can I do that? Thank you for replying at this email address : g.herr@webetsolutions.com

Regards,

Hello,

I sent you an email, please check.

Regards, Szabi – CodeRevolution.

If I buy a regular license to try the plugin in one website, then I can update to the extend to use it to all my websites, paying the price difference between the two or after do I have to buy the license extended to the total price?

I’m interested that the plugin translate the title and the slug too, when I import with the content translation. does this do it?

Hello,

The plugin can crawl a ‘start page’ from a website, and gather all URLs from there, and import them. If this does not match your website list, that one by one importing is the only option.

Regarding German translation issue, can you send me, please, the demo URL where you had this issue, so I can check it?

Regards.

Hello,

Can you reproduce the issue, please, in the demo you sent me? I see that there are 2 rules in it, one is set to translate from Dutch to Italian and the other from Spanish to Italian.

Can you recreate the rule that had issues translating to German? I tried to recreate the rule, but for me, translation worked.

Regards.

Hi, can you crawl the draw result from https://www.check4d.com/ ? the result will update on every Tuesday, Wednesday, Saturday & Sunday, around 7.30pm, possible to crawl and duplicate it to my own site? thanks

Hello,

Thank you for contacting me.

Yes, the crawling of this page is possible. To do so, please configure the plugin like this:

‘Main Settings’ -> please check the ‘Update Post If It Is Already Posted’ checkbox

‘Web Crawl to Posts’ menu -> create a new importing rule ->

‘Scraper Start (Seed) URL’ -> https://www.check4d.com/ ‘Content Query String’ -> magnum4d ‘Content Query Type’ -> Class

Like this, the plugin will import the listing from the page, and update it automatically, each time it runs.

Please give the plugin a try using it’s live demo, here: http://wpinitiate.com/crawlomatic-test/crawlomatic-test-generator/

Also, please check the plugin’s tutorial videos here: https://www.youtube.com/watch?v=7BYDa72zY7c&list=PLEiGTaa0iBIgcqNzVBaoTCS4ws47vNMuQ

Regards, Szabi – CodeRevolution.

nwa1982

nwa1982 Purchased

Hi,

I have some problems scraping a site from the waybackmachine. I watched your tutorial couple of times and i do excatly what you show, but it doesnt comes out how i want.

Can you tell me what settings i need for these 2 sites:

https://web.archive.org/web/20140826033633/http://www.hausfrauen-sex-bilder.com/

and

https://web.archive.org/web/20160216131602/http://www.geilerteenblog.com/

regards

Hello,

I checked, and for me, importing worked.

For the first, please configure importing like this:

‘Content Query Type’ -> Class ‘Content Query String’ -> PostContent

For the second:

‘Seed Page Crawling Query Type:’ -> Class ‘Seed Page Crawling Query String’ -> beitrags-titel ‘Content Query Type’ -> Class ‘Content Query String’ -> beitrag

If it is still not working for your, your server’s IP address might be blocked in way back machine for some reason, and because of this, you cannot access it.

Regards, Szabi – CodeRevolution.

Hi, is the plugin support posts from medium.com, I’m trying to scrap some links and each time it’s scrap the same link different. If it’s supported, what setting you recommend for it? Thanks

Hello,

First of all, thank you for your purchase.

This plugin will be able to scrape content from Medium, however, because it is a dynamically generated website (which generates content using JavaScript, also after the page is loaded) – the content might not be fully imported. To fix this, there are 2 options:

1. You install on your server the phantomjs extension (if possible) and enable it’s usage in the plugin – like this, the plugin will be able to import also dynamic content.

2. I have another plugin, specially built for Medium: https://codecanyon.net/item/mediumomatic-automatic-post-generator-and-medium-auto-poster-plugin-for-wordpress/20780459 – this one should be able to import content from Medium in a better way (using it’s default settings).

Regards, Szabi – CodeRevolution.

Do I have any limited on the websites or blogs to post on my websites if I buy the plugin? Like can I crawl from unlimited sources(websites) and create unlimited posts?

Hello,

Thank you for contacting me.

The plugin can be used to create unlimited posts and to crawl an unlimited number of websites.

The only limitation is that if you purchase a ‘Regular License’, it can be installed only on one WordPress site/domain. If you wish to get an unlimited domain license, please check the ‘Extended License’ for the plugin.

Regards, Szabi – CodeRevolution.

Sorry about the translation, I don’t know English and I’m translating with the DEEPL.

I have a website in prestashop and I want to spend the whole WP. https://sumarbil.com/ I want to know if this plugin can collect data both images and text from the web

If so, I’ll buy the plugin.

Hello,

Yes, you can also point the plugin to single items, they will be scraped.

Please check the tutorial videos for the plugin, to check it’s workings: https://www.youtube.com/watch?v=7BYDa72zY7c&list=PLEiGTaa0iBIgcqNzVBaoTCS4ws47vNMuQ

Regards.

I continue with the doubts of amazon

Can I put a link to my amazon id and keep it in the scraper ?

what I want is to take amazon products, make a page of the best sellers or a category and put it on a page of wordpress, if possible with the id mio of amazon

Hello,

Please check this other plugin, that is specialized on Amazon product importing: https://codecanyon.net/item/amazomatic-amazon-affiliate-post-importing-money-generator-plugin-for-wordpress/19763851

Please give it a try.

Regards.

Hi I have pre-purchase question suppose I want to create posts from a url like example.com/category/house I want to crawl all posts under this page is that possible ?

Hello,

Yes, this plugin is able to crawl all posts listed in an URL.

Please check the tutorial video for the plugin, here: https://www.youtube.com/watch?v=7BYDa72zY7c&list=PLEiGTaa0iBIgcqNzVBaoTCS4ws47vNMuQ

Also, it’s live demo, here: http://wpinitiate.com/crawlomatic-test/crawlomatic-test-generator/

Regards, Szabi – CodeRevolution.

i’m doing the demo and in crauling helper i see the whole site i want to scrape, but when i do the test campaign i just get texts no images.

What does crauling helper really do? I want to take the whole web with the photos and the shape of the theme

Hello,

Crawling helper can get you the HTML class or id that should be used when setting exactly what content the plugin should import.

You can use it by loading the target page, hovering your mouse over the page and clicking on the part of the page you wish to import. You will get a popup with the class/id.

It’s usage is not mandatory, it is only a tool to help you get ids and classes.

Regards, Szabi – CodeRevolution.

Hi there I don’t want that “Read More” button when limiting content size by setting maximum words count. Any way ? Also I don’t see that it actually working.. I am getting full content with “read more”..Though I set 28 words max. Thanks

Did you get my email ?

Hi there, Here is the issues I am facing. I got modified plugin for “Read More” button removal. If you send me another one, please keep that changes as well. Also please tell me where did you have made the changes to keep those changes in updated plugin version. 1. Price issue. I have noticed that the filed saving numbers with dot instead of comma in database. Can you please just modify it so it take price as text instead of number ? 2. I am not saving feature images locally. My site and the source have SSL. But the images coming from a subdomain without https://. Subdomain have SSL too but in html source, images just linked without https. I need all those external feature images link to be rewritten with https anyway. 3. Is there any way to remove some predefined texts from title ? For example: Source Title is: “Category name: Post name” In my website: “Post name” Here I want to predefine “Category name:” to remove from post title. Thank you

Hello,

I responded by email, please check.

Regards.

Hide CodeRevo… awesome plugin… i want to ask you if there is any way to strip css without remove all html from a post…

For example, this code:

<style type=”text/css” rel=”stylesheet”>.tg-marca td,.tg-marca th{font-family:”Roboto Condensed”,sans-serif;font-size:14px;overflow:hidden;word-break:normal}.tg-marca{width:100%;border-collapse:collapse;border-spacing:0;border-color:#aaa;border-width:0;margin-bottom:0}.tg-marca td{text-align:center;padding:10px 5px;color:#333;background-color:#fff;border:1px solid #e8e8e8}.tg-marca th{text-align:left;font-weight:400;padding:4px;border-style:solid;border-width:0;border-color:#aaa;color:#fff;background:linear-gradient(to bottom, #595959 0%, #353535 100%)}.tg-marca tr{border:1px solid #e8e8e8}.participant-name{white-space:nowrap}.assist{color:#B0B0B0}#goleslocal,#golesvisitante{font-size:25px;width:16px}#escudolocal,#escudovisitante{min-width:65px;height:55px}#torneot{height:17px}#tiempo{min-width:30px}#local,#visitante{width:35%}</style>

I dont want to have this in my post. Thanks.

Great. If you like the plugin, please give it a rating, it is really appreciated. :)

Regards.

cabenaguis

cabenaguis Purchased

yes, i will give you the best rating right now. one last question i have this another code that i want to erase same way, with regex, i tried to figured out myself but im awful with coding:

https://pastebin.com/QYURebr8

Thank you!!

Thank you for your rating. :)

Please try this Regex, it should match both:

<style type=[”'"]text\/css[”'"](?: rel=[”'"]stylesheet[”'"])?>([^<]*?)<\/style>

Tip: to learn Regex and to get the right one that matches your case, you can use this site: https://regexr.com/ – you can use it’s Cheatsheet from the left side to learn.

Regards.

Hi, can this crawl all this forum questions and responses contents? https://bit.ly/2j2mzER

Hi Szabi, the wp site is ready. How can I provide you the login information? Regards,

Hello,

My email is kisded@yahoo.com

Regards.

Hello Szabi, email sent, please check. Regards,

Guiz21

Guiz21 Purchased

This comment is currently being reviewed.

Guiz21

Guiz21 Purchased

Ok I just understood, I downloaded the installation files only and it works… ignore the previous message: D

Hello,

Thank you for your purchase.

Regards, Szabi – CodeRevolution.

strovolo

strovolo Purchased

Bought today,

all works good except from featured image grabbing. Tried all possible options and automation. Still cannot get the image to be posted.

I am trying to scrape from, https://www.brigitte.de Please advice

Hello,

First of all, thank you for your purchase.

Please configure importing rule settings like this (for me, featured image importing worked with the below settings):

‘Featured Image Query Type’ -> Class ‘Featured Image Query String’ -> m-image-box

Please let me know if this helped.

Regards, Szabi – CodeRevolution.

Presales question:

1. I’m using a multisite and for my blog, I’m using the Divi theme from Elegant Themes. Will there be any compatibility issues?

2. I have a standardized template for all my blog posts. Is it possible for every crawled article to match the format? If not, how should I do it since I’m using a website builder?

Cos it seems like all the articles are in one post: https://www.awesomescreenshot.com/image/3930688/71e47678ba2bfe302cab129244c9ba04

The plugin doesn’t create like, 10 posts for 10 articles. So I can’t change the content or layout of the articles individually right?

3. Is this compatible with plugins like Yoast for SEO purposes?

4. Does it come with spinwriter functions? Like Spin Rewriter or Article Builder? Or provides integration?

thank you and I hope to hear from you soon!

Hello,

Thank you for reaching out.

1. Yes, this is supported.

2. If you create manually a post (without assigning to it any custom settings) – and the post will get the standardized template automatically, then posts created by the plugin will also get this template automatically. If you set a settings during manual publishing to show the custom template, you need to get the slug of the custom meta field, to set it also from the plugin.

3. Yes, Yoast supported.

4. SpinRewriter, The Best Spinner and WordAI supported – also, the plugin has a built-in, free, text spinner (more simpler than the premium ones).

Please give the plugin a try using it’s live demo, here: http://wpinitiate.com/crawlomatic-test/crawlomatic-test-generator/

Also, please check it’s tutorial videos, here: https://www.youtube.com/watch?v=7BYDa72zY7c&list=PLEiGTaa0iBIgcqNzVBaoTCS4ws47vNMuQ

Regards, Szabi – CodeRevolution.

by
by
by
by
by
by