chevron_left chevron_right
Login Register invert_colors photo_library


Stay updated and chat with others! - Join the Discord!
Thread Rating:
  • 1 Vote(s) - 5 Average


How does google know my bot is a bot? filter_list
Author
Message
RE: How does google know my bot is a bot? #4
Google does more user checking than a user agent alone. However, if it's something such as a search, just use Google's API. APIs exist for more than one Google service, in fact most of them, so if it's something that can use an API, in this case, it should.

As far as what else it does, I'm not sure. I myself have never been able to quite "crack the code" on what Google does. Perhaps it's worth a Google.

(06-12-2016, 01:23 PM)superMAUS Wrote:
(06-12-2016, 01:14 PM)TechSaavy Wrote: It could have something to do with JavaScript not being executed by mechanize, as far as I know, mechanize can't parse JS.

Very interesting idea, I'll check that out. Maybe ill try with urllib if that has JS parsing.

Also, no it doesn't have JavaScript parsing. However, I doubt this is what is causing Google to raise flags on your bot. If you feel the need to regardless, there is a solution, many in fact. There's an old module I found a while ago which actually enabled the parsing of it, but it required Node.JS to be available on the system. You could also use the QT python bindings for webkit, call the evaluateJavaScript function, but I'm not sure how much "coverage" that would offer, i.e. if it can actually be the same as using a browser to execute JavaScript rather than a bot. Webkit is the renderer, so I think it would be able to.
(This post was last modified: 06-12-2016, 01:30 PM by Stocking.)

[+] 1 user Likes Stocking's post
Reply




Messages In This Thread
RE: How does google know my bot is a bot? - by Stocking - 06-12-2016, 01:26 PM



Users browsing this thread: 1 Guest(s)