SEOClerks

Majestic Seo not crawling all my pages



Write the reason you're deleting this FAQ

Majestic Seo not crawling all my pages

Hi,

on majestic, it only crawls 19 of my posts and really wants it to crawl all as I believe my TF will be higher. anybody know what is going on

Comments

Please login or sign up to leave a comment

Join
Cristian
How many posts do you have? Because if you only have 19 posts Majestic will only crawl those 19 posts.
Another good way to figure out what is going on is to write the following command in Google Search: "
site:http://www.yourwebsitename.com
Another good way to figure out what is going on is to write the following command in Google Search: "site:http://www.yourwebsitename.com"

This command will show you the exact number of pages of your domain indexed by Google. You can then compare that number with the number crawled by Majestic.

Also, I won't rely on web crawlers to much, if you really want to proper crawl a website I suggest you try Screaming Frog (the paid version) and I assure you if you configure it correctly it will crawl everything to the last link, image, and URL.

Another problem you may have that can stuck Majestic from crawling your whole website are robots.txt restriction you may have as well as robots meta tags with no-index or no-follow implemented on some of your pages.

If you don't have the money for a paid version of screaming frog, let me know your domain address and I'll do a full crawl and tell you where the problems are and overall result.

Cheers.



Are you sure you want to delete this post?

varleysheen
Thank you for your response. I have a feeling it maybe the robots.txt but wouldn't know to he sure.

I have 97 posts all together, the only reason I want TF is people seem to buy in to that when looking at paying me for a guest post.

my website is bustingmyfatbelly.com



Are you sure you want to delete this post?

Corzhens
This is the first time that I’ve read about Majestic SEO and I have to be honest that I want to know what it is really about. When it comes to crawlers, I understand those search engine bots that check the sites for their SEO concerns. But I don’t know about the indexing of the web pages that probably is the means of search engines to get the status of the sites regarding SEO requirements.



Are you sure you want to delete this post?