Page 1 of 2
Is Google excluded from crawling this board?
Posted: Thu Jul 24, 2025 2:07 pm
by NicTheQuick
Hi Fred,
today ChrigiGee wrote this post in the German forum:
https://www.purebasic.fr/german/viewtopic.php?t=33431
After a few tests I made with Google search it appears that it no longer indexes the board properly. Could this be related to the server upgrade you made recently? Since the built-in board search is pretty poor, it would be a shame if Google no longer functioned as the search engine for the forum.
Re: Is Google excluded from crawling this board?
Posted: Thu Jul 24, 2025 5:07 pm
by Bisonte
confirmed. Nearly no post is found by google anymore.
Re: Is Google excluded from crawling this board?
Posted: Thu Jul 24, 2025 8:30 pm
by ricardo_sdl
I use duckduckgo and can't find any forum posts and links on It either. The Domain is fone from the results.
Re: Is Google excluded from crawling this board?
Posted: Thu Jul 24, 2025 9:52 pm
by Caronte3D
Maybe with the last server change the robots.txt file is blocking all bots?

Re: Is Google excluded from crawling this board?
Posted: Thu Jul 24, 2025 10:37 pm
by Quin
Seems likely yeah, I used to see Bing BOT, google BOT etc as always online users, now I never do.
Re: Is Google excluded from crawling this board?
Posted: Thu Jul 24, 2025 10:57 pm
by Olli
@fred
You should check if a malware has modified the file
robots.txt (looking after the several access dates).
https://support.google.com/webmasters/a ... -robotstxt
https://developers.google.com/search/do ... k-indexing
Remark : check also the ad system, because any
fake news appear on the page queues...

Re: Is Google excluded from crawling this board?
Posted: Thu Jul 24, 2025 11:23 pm
by idle
you can look at the contents of robots.txt
User-agent: CCBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
so maybe there's another robots.txt higher up in the path or it's being blocked by dns
Re: Is Google excluded from crawling this board?
Posted: Mon Jul 28, 2025 2:06 pm
by Fred
I removed the bot check, it should be better
Re: Is Google excluded from crawling this board?
Posted: Mon Jul 28, 2025 3:35 pm
by NicTheQuick
Fred wrote: Mon Jul 28, 2025 2:06 pm
I removed the bot check, it should be better
Thanks, Fred. Let's wait a few days until Google is up-to-date again now.
Re: Is Google excluded from crawling this board?
Posted: Mon Jul 28, 2025 9:33 pm
by Olli
Google help tells us two other ways to block it : the keyword 'noindex' in the page, or in the response (post thing).
Note that maybe you are for nothing, but a malware in the ads, that Google detected, and cut the whole site containing it.
I mail an example... (to alpha send at 22:35) As you can see, I do not think you are agree with that type of ad...
Re: Is Google excluded from crawling this board?
Posted: Tue Jul 29, 2025 5:40 am
by Olli
Purebasic is back !
Re: Is Google excluded from crawling this board?
Posted: Sat Aug 16, 2025 3:49 pm
by spikey
This problem seems to be back again for me this afternoon. Can anyone else confirm?
Code: Select all
PureBasic
https://www.purebasic.com
No information is available for this page.
Learn why
PureBasic Forum
PureBasic
https://forums.purebasic.com
No information is available for this page.
Learn why
Duck Duck Go seems unable to crawl it too.
Re: Is Google excluded from crawling this board?
Posted: Sat Aug 16, 2025 6:42 pm
by minimy
spikey wrote: Sat Aug 16, 2025 3:49 pm
This problem seems to be back again for me this afternoon. Can anyone else confirm?
Code: Select all
PureBasic
https://www.purebasic.com
No information is available for this page.
Learn why
PureBasic Forum
PureBasic
https://forums.purebasic.com
No information is available for this page.
Learn why
Duck Duck Go seems unable to crawl it too.
Confirmed google is .... mmm? google?
Google= No work
DuckDuckGo= Work
Bing= Work
Yahoo= Work
Ecosia= Work
Yandex= Work
Gibiru= Work
Dogpile= Work
Startpage= Work
Qwant= Work
As i said... google is google...
Re: Is Google excluded from crawling this board?
Posted: Sun Aug 17, 2025 3:50 pm
by Olli
Strange to see such a treatment difference :
I go to this forum, only with Google search. And, actually, I do not see any problem...
I am curious to see anybody making a Google visibility "map" of the forum, with a VPN.
Is this message of minimy a presage of a none status in the next hours ?
Re: Is Google excluded from crawling this board?
Posted: Mon Aug 18, 2025 9:03 am
by Rinzwind
Fred wrote: Mon Jul 28, 2025 2:06 pm
I removed the bot check, it should be better
The Robots.txt needs adjustment, that is for sure. ChatGPT can't access it:
I couldn’t fetch purebasic.com directly here (blocked by robots.txt)
Too bad, because it is the main resource of PB syntax rules and solutions. @Fred!