Discussion on WP Content Crawler - Get content from almost any site, automatically!

turgutsaricam

turgutsaricam supports this item

Supported

This author's response time can be up to 5 business days.

438 comments found.

Hi in tryint to crawl a site but the images are using timthumb (i know, how dare they) and the image url comes like this ->

http://somedomain.com/wp-content/themes/pp/timthumb.php?src=http://somedomain.com/wp-content/uploads/2017/11/thumbnail.jpg&w=900&h=506

And i am not being able to get the featured image, i tried using the direct image url but it is not working :(

Hi,

If the direct URL is not working, there is currently nothing to do about it. I cannot check the link since you gave an invalid link.

Hi again, i sent you via message the link to the page im trying to crawl, can you take a look and tell me if its gonna be possible please ?

focus12

focus12 Purchased

Hello, I see ”
 
 
 
 
 
 
 
 
 



” when i use the browse. Please advice me what to do.

Thanks.

focus12

focus12 Purchased

I receive ” 
 
 
 
 
 
 
 
 
 
 ” when i click on the url selector. Wish i can send you image.

focus12

focus12 Purchased

Working now mbstring was disabled.

All right. I’m glad its solved.

focus12

focus12 Purchased

Active for crawl not working, only works after plugin is disable and enabled. You have to repeat that before it crawl again.

Hi

Make sure your CRON works properly.

focus12

focus12 Purchased

Hello, my CRON is working well but unable to auto crawl.

Could you please send me your admin and FTP login credentials through the contact form on my profile page?

Hi, I have one question before buying the plugin. Is there any option to name the saved images as the same post’s titles? If that option’s available, can the image be named as the translated title after the content’s translated (in case i use Google Trans Engine). This is all for SEO benefit, thank you!

Hi,

Unfortunately, it is not possible to rename saved images.

Can you bring that option to the next release?

I cannot promise that it will be included in the next release. However, it is in my todo list and I am planning on implementing it if I can find the time.

Please, please, please… We need a fix for the problem between your plugin an Newspaper 8 theme. As you can see here it’s one of the most used themes in the world: https://www.wpthemedetector.com/top-wordpress-themes/ and I need to use both, the theme and your plugin… Could you please talk to the people of the theme and give us a fix for that? In your support answer about newspaper 8 problem you can delete the first option, because the theme has this already: function _schedule_modify_add_three_days($schedules) (now on line 500)

Hi there!!! good news!!!

I found the fix for the problem… (I was a very bad boy and were looking for another plugin that makes something like yours but without the problem with the Newspaper theme but I found the same problem in the other plugins…) The best new is that one of the other plugin’s author have the fix for the problem!!!!

They write this:

“If you use ionMag WordPress Theme, please continue with this step otherwise you can skip to next step. You should modify the wp-content/themes/ionMag/includes/wp_booster/td_cake.php and file td_js.php.

function _schedule_modify_add_three_days() { to function _schedule_modify_add_three_days($schedules) {

After you edit and save the file, re-save your task to check if is it working as expected or not. If it’s not, please follow the next step. “

As you can see there is not only one file (td_cake.php) but there is another file (td_js.php) which is not fixed. (and the problem is the same with other themes) When I fix the second file with the $schedules parameter your plugin runs again PERFECT!!!!

I wrote to the theme’s creator and hope they solve that, but I think its a good idea for you to write it in your support section.

and now… is it posible to have that file in my child theme to avoid the problem in the next theme update in the case they don’t solve it? could I overwrite the file with my fixed one in my child theme? what would be the path in the child theme?

I’m glad you managed to solve the issue and thanks for letting me know about the solution. I am going to add this to the FAQ section, as you suggested.

For the child theme question, I checked WP’s Codex (Template Files section on Child Themes page). The page says

“If you want to change more than just the stylesheet, your child theme can override any file in the parent theme: simply include a file of the same name in the child theme directory, and it will override the equivalent file in the parent theme directory when your site loads. For instance, if you want to change the PHP code for the site header, you can include a header.php in your child theme’s directory, and that file will be used instead of the parent theme’s header.php.”

So, I think you can override the file by creating the same file under the same directory, which the file exists under in the parent theme, in your child theme. I am not familiar with themes. I hope it works for you.

Hi

How do I try the demo? When I click on it, it goes to the Wordpress login admin page. Is there any login information for the demo?

I have tried Scrapes – Automatic web content crawler and auto post plugin for WordPress. I cannot see the exact web page of the source when I click on the load values from source.

I would like to create a website similar to http://www.newlaunchportal.com.sg/

I will be editing the contents after the crawl. Is this possible?

Hi,

Login credentials is written above the login form on the login page.

I noticed that you have great support thats why I am considering getting this. Here are my questions before purchase: 1. Why you did not add your tool to the Black Friday offers in Envato, as may already did 2. Can I collect articles from multiple website or multiple categories in each website, and then publish it to one or two different news websites 3. Does it work with Arabic language 4. I do not know CSS or coding, and from the video it seems you need to have idea about coding and CSS. is this a simple user friendly tool

Re #2, i want to see how this works on your demo website before I order it. Thats why I requested form you to help me as I could not do them, these are just samples

You can follow the steps described in the video tutorial to learn how to create site settings.

That was not the reply I expected as I tried but your video is hard and not easy

Hi is there any feature in this plugin to have the same permalink as the source has?

Hi,

Unfortunately, the plugin does not have a feature like you mentioned. WordPress generates the post permalink by using the post’s title, automatically.

munyu

munyu Purchased

Hello. There are things I do not understand about plugin settings. The reason why the crawl of articles with pagination stops is because the setting is incorrect, is not it?

The pagination of my blog is like the screen shot below. One is to the next page. The other is the link to each page. There is no link to the current page.

https://gyazo.com/1eccb7c540b2fc4392350091d1db775d https://gyazo.com/1123bbd1faf29d507135115986728546

The plug-in settings are as shown in the screenshot below.

https://gyazo.com/1f5751e60e030e21ccecc87b24516047

When automatic crawling is done with this setting, it stops with the article being paginated, and the article is saved in draft state.

I tried each page with pagination at Site Tester. It seems to be working normally, but is it normal for Site Tester that the pagination link remains the original site’s URL?

I have read past comments,“you also need to use Post All Page URLs Selectors”, but I do not know what to set for Post All Page URLs Selectors.

Post All Page URLs Selectors is a link to a page that lists all the links of the divided articles? What should I do if I do not have such a page?

As I said, you need to make sure the next page selector finds the right next page URL. Also, for the last page, the selector should point to an empty value. Please make sure your settings are correct by using Tester by checking every page of the target post to see if correct next page URL is found.

If you cannot understand my explanation, I’m sorry but I cannot do anything about it :)

munyu

munyu Purchased

Thanks to you, the crawler was able to get an article with pagination.

However, the link URL of the pagination remains the URL of the original article. Is it not automatically changed to the URL of my blog? Could you tell me how to change it?

I do not understand what you mean. Could you please send your admin login credentials using the contact form on my profile page so that I can check your settings?

Hi there, me again. :) I would like to tell you one thing… there is another plugin that crawls pages (I will not say the name here because I prefer yours but i bought and tested it) that doesn’t need google or microsoft apis to translate the posts, perhaps you could look how it does it and do it in your plugin too… I think this is the only think your plugin need to be perfect… ;) If you want the name of the plugin just ask me. :)

Hi,

The right way to implement translation functionality is to use provided APIs, since other methods might not be available after certain measures have been taken.

soriax

soriax Purchased

Hi

how can i get the youtube image from iframe?

Hi,

I think this is not a plugin-related issue.

soriax

soriax Purchased

no i dont say it is an issue, but help me how to get it :-)

All right. I can only guide you since this is out of support’s coverage. Here is a way to get thumbnail images using YouTube video IDs. You can extract video ID using find and replace options. For example, you can create an img element next to the iframe element, and then write a selector for the img element so that the plugin can get its src and save it as featured image of the saved post.

mrb turgut, Öncelikle böyle bi başarıda imzan olmasından doayı seni kutlarım. Benim sorunum ise şudur. 3 site denedim. 70 tane filan çekiyor daha sonra duruyor nedeniyni anlamadım. yardımcı olursan seviinirm

Merhaba,

Teşekkür ederim. Kategori ayarlarında bir problem olabilir. Testçi’yi kullanarak kategori sayfalarında sonraki sayfaların linklerinin bulunduğundan emin olun.

soriax

soriax Purchased

why does the plugin not automatically check for new posts and copy them to my site?

Hi,

There can be a lot of reasons. Could you please give more details about the problem? Do your CRON jobs work properly? Did you activate scheduling from both general settings and site settings? Did you make sure your site settings are correct by testing category and post pages in Tester?

Signgoo

Signgoo Purchased

Hey – I have problems with the plugin. I have installed it on a subdomain “http://mailer.manymize.com/” – After initial installation, everything works fine for me. The plugin collected the sites properly. Then the plugin stopped working… when now deactivate the plugin, reset the license-code, and reenter my purchase code it runs for one cycle but then stops again.

Maybe this is an issue with the license server? Please let me know if you need an admin account or anything else to verify the problem.

Best regards, Kai

Signgoo

Signgoo Purchased

I will have a look on that, but I deleted the installation and database and build everything from scratch. Plugin doesn´t perform any action after second installation – only one time fora short period (seconds) when I reset the license code.

Signgoo

Signgoo Purchased

mbstring is enabled – libmbfl v. 1.3.2, oniguruma v 5.9.6

Could you please send me your admin and FTP login credentials through the contact form on my profile page so that I can check what might be wrong?

hayırlı satışlar diyerek konuya gireyim :) hocam eklentinizi satın alacağım lakin öncesinde sorum olacak, bulduğu içerikleri eklenti ayarları yapıldıktan sonra kendisi otomatik bir şekilde yayınlıyor mu ? 2. si https://themeforest.net/item/newspaper-news-magazine-wordpress-theme/16131379?s_rank=7 bu tema üzerinde kullanacağım, çalışmasında bir sorun olur mu ? veya temadan temaya bi farklılık olur mu diye sorayım.

Merhaba,

Teşekkürler. Evet, ayarlarını yaptıktan sonra eklenti otomatik bir şekilde içerikleri yayınlar. Bahsettiğiniz temada CRON’un çalışmasını engelleyen bir bug var. WP İçerik Toplayıcı her temada çalışacak şekilde tasarlanmıştır. Fakat kullandığınız tema WordPress’in temel fonksiyonlarını bozuyorsa bu temadan kaynaklı bir problemdir. Diğer taraftan, farklı müşteriler de bu temanın bu probleminden şikayetçi oldukları için, buldukları çözümleri de bana ilettiler. Ben de bu çözümleri Sık Sorulan Sorular bölümünde yazdım. Bu temayı kullanmakta kararlıysanız, şu SSS‘nin 1. maddesinde anlatılan değişiklikleri yaptığınızda, müşterilerden aldığım geri bildirimlere göre, sorun düzeliyor.

Pre sale question; Can I have the plugin create posts that, when the user clicks on the post link to open the article, it instead opens a new browser tab and opens the article on the source page? For example, I woudl scrape from Amazon and when the user clicks to open the post/article, it opens up a new browser tab and goes directly to the article on Amazon?

Hi,

This cannot be done on the plugin level. However, you can make the plugin save the posts along with a post meta that stores the post’s URL, or you can use the post meta key used by the plugin to store posts’ URLs, and then make your theme do the rest of the work, i.e. opening the links.

Hi, thank you for the reply. That is a shame, I´d already bought from your competitor and made the mistake of not realizing to begin with what I needed and it set my project back so badly. I am now trying to find an alternative that can do this but it doesn´t seem to be an option for anyone :( Thank you anyway!

All right. That’s because what you want can be done by modifying a theme, not a plugin.

mrb tekrar,

verileri çekiyor iyi güzel ama 24 saat sonra durum bu. sıra artarken kaydetme ya hiç artmıyor yada çok az Queue: 1692 Saved: 57

sorun nerde acaba?

Merhaba,

Kaydedilen URLler geçersiz olabilir. URLlerin doğru olduğundan emin olun.

How to input short Codes in the template?

wcc-main-title wcc-main-content wcc-main-gallery .......OK

Custom Content Selecter ......???

[wcc-main-test1]

Hi,

It is written in the explanation of that setting:

‘Write your shortcodes without brackets, e.g. “item-price”. Next, you can use it in the main post template by writing [item-price]’

by
by
by
by
by
by