Since When: Yahoo! Bot?!? MSN Bot?!? - Printable Version
-Shoutbox (https://shoutbox.menthix.net)
+-- Forum: MsgHelp Archive (/forumdisplay.php?fid=58)
+--- Forum: General (/forumdisplay.php?fid=11)
+---- Forum: Forum & Website (/forumdisplay.php?fid=13)
+----- Thread: Since When: Yahoo! Bot?!? MSN Bot?!? (/showthread.php?tid=38927)
Since When: Yahoo! Bot?!? MSN Bot?!? by matty on 02-21-2005 at 05:44 AM
Since when has this board recorded where bots are from?
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by WDZ on 02-21-2005 at 05:46 AM
http://shoutbox.menthix.net/showthread.php?tid=38791
I was just bored the other day and had some fun. 
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by Pipish on 02-21-2005 at 05:56 AM
Dodgy WDZ putting yahoo bots on the forums
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by WDZ on 02-21-2005 at 05:58 AM
quote:What? They've always been here.
Originally posted by Pipish
Dodgy WDZ putting yahoo bots on the forums
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by Pipish on 02-21-2005 at 06:26 AM
realy i thought this forums didnt use bots
oh well u learn somethign new every day
* Pipish slaps himself
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by WDZ on 02-21-2005 at 06:28 AM
Search engine bots crawl most of the sites on the internet. It's how they make indexes so you can get search results.
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by Mnjul on 02-21-2005 at 06:47 AM
Why is there no Google crawler? 
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by WDZ on 02-21-2005 at 06:53 AM
The GoogleBot doesn't seem as active as the other two. I guess it's not on constantly, and maybe it uses one IP address (so one guest) for multiple requests. I saw it online for a while yesterday...
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by fluffy_lobster on 02-21-2005 at 04:54 PM
Are you sure you added the whole GoogleBot IP range?
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by WDZ on 02-21-2005 at 05:03 PM
quote:Well, I added 2 full ranges that are registered to Google, and I know they're used for GoogleBot. They might have more ranges that I'm not aware of though. I'll look into it some more...
Originally posted by fluffy_lobster
Are you sure you added the whole GoogleBot IP range?
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by saralk on 02-21-2005 at 05:06 PM
why dont you use $HTTP_USER_AGENT?
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by WDZ on 02-21-2005 at 05:09 PM
quote:I could do that for current requests, but on the Who's Online page, it shows old requests that were stored in the database, and User-Agents aren't stored, just IPs.
Originally posted by saralk
why dont you use $HTTP_USER_AGENT?
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by saralk on 02-21-2005 at 05:17 PM
couldnt you add another range, called bot, and it stored if it was a bot, and what bot it was?
Im only asking because i used a program to change my user agent to googlebot's user agent
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by WDZ on 02-21-2005 at 05:19 PM
quote:That's another thing I forgot to mention; user-agents are easier to spoof.
Originally posted by saralk
Im only asking because i used a program to change my user agent to googlebot's user agent
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by Dempsey on 02-21-2005 at 05:20 PM
WDZ would you like to release the changes you made so that other MyBB users could have it on their boards?
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by WDZ on 02-21-2005 at 05:25 PM
Not really. 
It's not finished anyway... 
I made it in like 15 minutes.
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by Concord Dawn on 02-21-2005 at 05:30 PM
quote:
Originally posted by WDZ
Not really.
It's not finished anyway...
Go DZ!!! L33t PHP coder.
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by saralk on 02-21-2005 at 05:32 PM
does google use its own hostname? THat might be helpful.
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by WDZ on 02-21-2005 at 05:35 PM
quote:It's like crawlxx.google.com or proxy.google.com... of course, doing a hostname lookup on every IP can be very slow.
Originally posted by saralk
does google use its own hostname? THat might be helpful.
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by Smethead on 02-22-2005 at 07:25 PM
I've made a similar thing, but I don't know if it works good yet. I need some testers.
http://www.mybbmods.com/forum/showthread.php?tid=285&pid=1501#pid1501
WDZ, did you type all of the ip's, or do you use some kind of database? Because all ip's make a very long code.
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by WDZ on 02-22-2005 at 07:35 PM
Well, I generated lists of IP ranges, not each individual IP. The lists don't take up much space.
BTW, I hate it when people copy my ideas. 
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by toddy on 02-22-2005 at 07:36 PM
quote:
Originally posted by WDZ
BTW, I hate it when people copy my ideas.
well you'll have to live with it
(it won't be build asgood anyway
)
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by Jeronimo on 02-22-2005 at 07:42 PM
Yahoo are viewing my profile again trying to track me down 
Looks good though 'DZ 
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by toddy on 02-22-2005 at 07:44 PM
quote:
Originally posted by Jeronimo
Yahoo are viewing my profile again trying to track me down
no there not
dz maybe u should split these 3 posts
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by Smethead on 02-22-2005 at 07:46 PM
quote:It was a request. I didn't even copy anything exactly as you did. I made it myself.
Originally posted by WDZ
BTW, I hate it when people copy my ideas.
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by saralk on 02-22-2005 at 08:09 PM
but you copied his idea.
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by Smethead on 02-22-2005 at 08:16 PM
I didn't really copy his idea. It was a request and this board was an examples, so i had to see what he meant.
quote:
Originally posted by Dempsey
I was wondering if anyone could make a similar mod to that on the Messenger Plus! Forums so that Search Engine bots like Google, MSN and Yahoo are shown, rather than just as a guest.
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by Millenium_edition on 02-22-2005 at 08:40 PM
quote:that things existed on IPB since a long time ago
Originally posted by saralk
but you copied his idea.
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by WDZ on 02-22-2005 at 11:39 PM
quote:Nobody suggested doing it in MyBB until after I did it though..
Originally posted by Millenium_edition
that things existed on IPB since a long time ago
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by WDZ on 02-28-2005 at 07:51 PM
Update: I found and added some new IP ranges for Google and Yahoo... now maybe we'll see more of the Google icon on the online page... 
RE: Since When: Yahoo! Bot?!? MSN Bot?!? by Vilkku on 02-28-2005 at 08:41 PM
It works