Discussion on WP Content Crawler - Get content from almost any site, automatically!

turgutsaricam

turgutsaricam supports this item

Supported

This author's response time can be up to 2 business days.

220 comments found.

Dear Turgut

I would like to know how to delete some invisible tag

<- like this ->

and tags which come along with the images, usually comes with the original site url

srcset= / alt= / width= / height= /

Please help me solve this issue Thank you

Hi,

In the current version, you can delete them using find-and-replace options with regular expressions. In the next version, there will be options that you can use to remove specified attributes of elements.

To remove “src” attribute with its value, you can use the following:
Regex: Checked
Find: (<[a-zA-Z]+\s.*?)src="[^"]+"(.*?>)
Replace: $1$2

You can replace “src” with the name of another attribute that you want to remove.

To remove comments, you can use the following:
Regex: Checked
Find: <!--(?:.|\n)*?-->
Replace: Leave this empty

Thank you for your kind support. I am not a programmer, I cannot achieve this without your support. I bought another license as a thanks for your help.

Thank you :) As I said, in the next version, these kind of manipulations will be much easier.

When the crawler is getting categories, it is only getting the final category, and not the parent categories. Is there any setting that will make it use the parent categories as well? Thanks.

Hrm, I have all the categories listed on the site already. The crawler is placing things in the correct category, but not marking the parent categories.

I enabled cookies as well, though I am not sure why that would affect marking parent categories.

Now I understand what you mean. I thought you were asking something else, sorry. The plugin puts the posts in only one category, as you select only one category in category map. Unfortunately, there is no way to achieve that currently.

OK, thanks. I appreciate your response.

If anyone has any ideas on how to achieve automatic checking of the parent categories I would appreciate it. I am trying to get this ecommerce store up. Its broken down like a normal site.

PARENT CATEGORY —>Child Category—->Sub Child Category.

So when the crawler runs, it only puts it in Sub Child Category. So when Woocommerce displays the categories on the store, nothing shows up.

Editing manually with over 600 products is not an option. I just need something to set those parent categories. If I find a solution, I will post it here.

drayim

drayim Purchased

Merhaba Turgut Lisans süreniz bitmiştir uyarısı alıyoruz. Lisans süremiz nasıl bitiyor. Lisans süresi biz alırken belirtilmemişti. 6 ay 12 ay 1 yıllık 3 yıllık seçenek yoktu.

Merhaba,

Lisans kontrolü yapan sunucuda bir problem olmuş olabilir. Lisans ayarlarınızı yeniden kaydederseniz büyük ihtimalle sorun düzelecektir. Lisans sürenizin bitme gibi bir ihtimali yok. Aldığınız lisans ömür boyu geçerlidir.

Hi,

There might have been a problem with the license server. The problem will probably be fixed if you save your license settings again. Your license cannot expire since it is a life-time-valid license.

Mugen88

Mugen88 Purchased

Merhaba, youtube üzerinden de yazmıştım. Orada da bahsettiğim gibi güncellemeyi yaptıktan sonra eklentinin içerisindeki hiç bir sayfa açılmamaya başladı. Yardımcı olabilir misiniz ?

Lütfen profil sayfamı kullanarak FTP ve admin panel bilgilerinizi email gönderin.

Parse error: syntax error, unexpected ’.’ in /wp-content/plugins/wp-content-crawler/app/vendor/illuminate/events/Dispatcher.php on line 334

Please send me an email so that I can send you the previous version of the plugin. The current version works with PHP 5.6 without any problems. Sorry for the inconvenience.

I upgraded to PHP 5.6. It works well. Thank you

OK. Sorry again. I’m glad it works now.

kjcheon

kjcheon Purchased

HI, how can I set the crawler for the page not with categories but by keyword search result?

You can use the search result page’s URL as a category URL.

I sent an email about the 1.6 version bug and translation.:grin:

This comment is currently being reviewed.

The plugin is sold only on CodeCanyon.

This comment is currently being reviewed.

Thanks. I’ll consider implementing your request.

Hi there, if you make the plugin able to catch number of fb shares and display it at the backend and arrange posts in list of importance I will buy for sure!

Hi, need to click on a link, to show me a hidden info on a page that o need to craw, ( i wanna do this as i load the post page ). Im trying to do this on Post config to Manipulate HTML: <script type=”text/javascript”> $(function(){ $(’.versomentetelefone’).trigger(‘click’); }); </script> or $(’.versomentetelefone’).click(); And put the class on Item: Find and replace in HTML at first load

but its not working can u help me?

If help, the class is versomentetelefone, i need click to show me the full phone number, this is the page: https://www.webmotors.com.br/comprar/volkswagen/gol/1-0-mi-8v-flex-4p-manual-g-v/4-portas/2010-2011/18860378

Thank you!

Hi,

The plugin cannot be used to click on links or run JavaScript. If the HTML code you want to retrieve is hidden on the page, you do not need to click anything. You can just get it by writing its CSS selector. However, if the data is loaded with an AJAX request, you cannot get it with the plugin.

Selam Daha önce de sizinle plugin için görüşmüştüm. Plugin’i almak istiyorum ama ayarlarken biraz sorun yaşıyorum. Ayarlamam konusunda yardımcı olmanız mümkün mü?

Merhaba,

Takıldığınız bir yer olursa yardımcı olabilirim fakat site ayarlarını sıfırdan yapmıyorum.

Curious, will this provide source or credit to the original poster for each article?

Can this be used in conjunction with adsense?

Hi,

If you want, you can add original post’s URL to your posts. Using templates, you can decide how you show the URL, such as showing a link under the target site’s name.

You can use the plugin in any way you like, including showing advertisements. This kind of stuff is more related to themes.

This comment is currently being reviewed.

Hi,

Thanks. I’m glad you like it. You can try using cookies option under Main tab in site settings.

Great work :-) GL

Thanks :)

Hi, how can i record crawling date and time (not post date) into a post meta?

Hi,

Currently, it does not seem possible.

Hello, I buy the WebCrawler 15 minutes ago – and always comes the error:

Your Content Crawler license has expired. Please get a new license until 27/03/2017 12:02 to continue using Content Crawler.

Message: The license could not be checked with the server. Please try saving your license settings again in a few minutes. If the error persists, please contact the developer.

Complete the setup now.

I put in the Purchase Code from Envato Market but it doesn`t work?

Hi,

Please try saving the license settings again a few times. If it does not work, please send your license key to me through my profile page so that I can check what is wrong.

What are few good free websites to copy content?

Sorry, this question is out of scope.

ayronika

ayronika Purchased

Hi Turgut, I am getting such a response about the license:

Your Content Crawler license has expired. Please get a new license until 27/03/2017 19:09 to continue using Content Crawler.

Message: The license could not be checked with the server. Please try saving your license settings again in a few minutes. If the error persists, please contact the developer.

Can you provide a new one?

Hi,

Please try saving your license key again. Your license key does not have an expiration date, i.e. it is valid forever. The server might have been a little bit busy when your license was checked. Please just try saving your license key a few more times with interval of a few minutes.

Hello, in last 2 days I am trying to make the cookies works, but can’t do, I have inserted the cookies in firefox manually and cookies is working but in my site doesn’t , I tried in your demo http://wpcontentcrawler.com/demo, and is not working too, I tried the same cookie in another WordPress crawler and there it is working, the browser from another plugin crawler is logged in, please can you check if it is working , I can give you the cookies for you test , thank you

Hi,

Yes, I would like to check the cookies. Please send them to me through my profile page.

already sent the message, thanks for your support

That website forbids accessing them. So, the plugin does not save them.

How do you found that, so I don`t lose time with similar websites.

The error is handled silently. I figured that out by debugging the URL response, which you can only do by investigating plugin’s source code. You can just observe whether the image is saved or not by using the tester in order not to waste time with similar sites. First, just write image URL selectors and just test it. If the images are not saved, then you will know.

by
by
by
by
by
by