atom feed5 messages in Server Side Call triggered by web...
FromSent OnAttachments
MarkOGSep 17, 2010 3:22 am 
Jeremy GeerdesSep 17, 2010 5:25 am 
MarkOGSep 17, 2010 6:36 am 
Jeremy GeerdesSep 17, 2010 7:04 am 
MarkOGSep 17, 2010 9:59 am 
Subject:Re: Server Side Call triggered by website crawlers
From:Jeremy Geerdes (
Date:Sep 17, 2010 5:25:18 am

Hi there. We kicked this around when you initially asked the question back in
July. Here is the link to that thread:

I would point out today that there are compelling arguments on both sides of
this question. On the one hand, the spiders, etc., that crawl your site will be
triggering automated queries which will be in violation of the TOS. However, on
the other hand, in the case of GoogleBot, we're not talking about you or a third
party triggering the automated search, but Google itself.

Since the TOS specifically require that the search results not constitute the
bulk of the content on your page or site, I will assume that the queries to
Google are simply augmenting things. Therefore, I would suggest using whatever
server-side app you're using to run these queries to detect the client's User
Agent string and, if it's a common bot, simply not running the requests to the
Google APIs.

All of this said, I am not a legal expert, and I don't know anyone on the group
who is. If you're truly concerned about this, then you should consult someone
who is (e.g., a lawyer).

Jeremy R. Geerdes Effective website design & development Des Moines, IA

For more information or a project quote:

If you're in the Des Moines, IA, area, check out Debra Heights Wesleyan Church!

On Sep 17, 2010, at 5:23 AM, MarkOG wrote:

Situation: A web page displays some data that is taken from Google via sever side implementation of the Ajax Api.

The page is requested by users who click on a link to get to the page.

Now a Spider has found the link to the page and starts requesting the page.

Is this now against the Terms of Use regarding requests only being made by real end users?

Does GoogleBot spidering my site and hitting pages like this mean that I am breaking the TOS?