Reporting Forum Error

[Cardinal51]

I do not think that is correct. Guests can not see anything in these forums.

Tell google about indexing your site . . . . https://search.google.com/search-consol ... authuser=0

[Chacal]

Ah yes, same problem: I know we disabled access at some point, but I can't remember why.

[Chacal]

Now guests and bots can read the forum's contents.

[Cardinal51]

Just checked for guests, and the problem is fixed

Google Bot did not do her work yet though.

[Cardinal51]

Just noticed Bingbot is present.

Screenshot 2024-11-22 124136.jpg

Hopefully these will change soon:

Screenshot 2024-11-22 124406.jpg

[Chacal]

I created accounts for some crawlers, this lets us better control what they scrape.

[Cardinal51]

I created accounts for some crawlers, this lets us better control what they scrape.

When non-members have access you do not really need special settings for crawlers anymore. Crawlers are your friend, not the enemy. :-)

Unless you want to give the crawlers the rights of registered users. That has the advantage that for instance google will index images and you will get traffic from google images users as well.

[AmericanSteam]

I created accounts for some crawlers, this lets us better control what they scrape.

When non-members have access you do not really need special settings for crawlers anymore. Crawlers are your friend, not the enemy. :-)

Unless you want to give the crawlers the rights of registered users. That has the advantage that for instance google will index images and you will get traffic from google images users as well.

What if these crawlers do not close their connections. We are left with limited connections. Could there be a connection limit time out?

[Chacal]

Legit crawlers play nice. They follow your directives in robots.txt, they avoid impacting the performance of your site and they close their connections.

[Cardinal51]

I stopped being a forum admin about 8 yrs ago. Had half a million members. First we had phpbb and later switched ( LOT of work and scripting) to Vbulletin. Crawlers had Reg User access, was never an issue. Ofcourse you could also ask in the phpbb help forums, great source for info and knowledge

These great RWA forums have a lot less traffic than they deserve and a lot less than what one would expect. I assume that SEO has a lot to do with that.

[buzz456]

"These great RWA forums have a lot less traffic than they deserve". Why would we care? Is there a reward for it or something? My aim is to have a informative, educational, pleasant place for the lovers of our great hobby to enjoy themselves share all kinds of stuff without all the drama and snarkyness that infect so many forums where that kind of behavior if not protected by anonymity and remoteness would result in a poke in the nose.
This is supposed to be fun and a expanding hobby and we should just pledge to do our best to further that here.
Rant off.

[Chacal]

This.
SEO isn't high on my list of priorities.
Serious hobbyists know about us.


Still, I'm performing this experiment out of curiosity.
It seems like the crawlers, including Googlebot, have found us and are slowly starting their work.

[Cardinal51]

.....Why would we care? Is there a reward for it or something? ....

I think it would be great to have more grumpy people like you in these forums !

[buzz456]

.....Why would we care? Is there a reward for it or something? ....

I think it would be great to have more grumpy people like you in these forums !

Why is that grumpy? I think you are a little sensitive that I question a statement that i don't understand.

[Chacal]

I think he means that's your usual demeanor, not just for this conversation.