[Challenge] Sinister.ly on the commandline.

Eclipse · 08-20-2015, 04:27 PM

(08-20-2015, 04:13 PM)Shebang Wrote: Well, every time I noticed something wrong and I went to change it, I obviously had to re-login and use the command again, so 2 requests for every change I made. Probably did that about every 30-90 seconds.

Maybe get the full response from the server instead of just the code. It's most likely just cloudflare's browser integrity check. Make sure your HTTP request is correct.

Equinox · 08-20-2015, 04:36 PM

(08-19-2015, 07:52 AM)Oni Wrote: Hehe. If someone made a client to browse/post for Linux that would be interesting.

Are you implying none of us here are on anything GNU/Linux based?

(08-20-2015, 08:31 AM)Lux Wrote: You know what, let's not make this a challenge, but a project. I'm using NodeJS. Anyone wanting to assist? @"Stocking" maybe?

Why are you using NodeJS exactly? -SNIP-. Aside from that, Python has a pretty good workflow to go with it in this sort of project, and it's far easier to test with.

Shebang · 08-20-2015, 04:37 PM

(08-20-2015, 04:27 PM)eclipse Wrote: Maybe get the full response from the server instead of just the code. It's most likely just cloudflare's browser integrity check. Make sure your HTTP request is correct.

I am obtaining (at least attempting to) the full response. I don't see how my request would be "wrong", because it was working initially. It was just after using it a bunch I hit that "wall".

Equinox · 08-20-2015, 04:39 PM

(08-20-2015, 04:37 PM)Shebang Wrote: I am obtaining (at least attempting to) the full response. I don't see how my request would be "wrong", because it was working initially. It was just after using it a bunch I hit that "wall".

You have to specify a user agent header, if you haven't figured that out already (pretty sure you did though). Here, use my user-agent if you're too lazy to fetch yours.

Code:
Mozilla/5.0 (X11; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0

Shebang · 08-20-2015, 04:42 PM

(08-20-2015, 04:39 PM)Stocking Wrote: You have to specify a user agent header, if you haven't figured that out already (pretty sure you did though). Here, use my user-agent if you're too lazy to fetch yours.

Code:
Mozilla/5.0 (X11; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0

Yep, already have a user agent set.

Akane · 08-20-2015, 04:57 PM

(08-20-2015, 04:42 PM)Shebang Wrote: Yep, already have a user agent set.

Weird. Try again in half an hour. Or try proxying your request.

Equinox · 08-20-2015, 05:06 PM

Well, we're definitely getting somewhere.

Master Antrax · 08-20-2015, 05:15 PM

@Shebang The 403 forbidden error is coming from the CF page itself. You might be able to use a Keep-Alive connection to get by that.

Shebang · 08-20-2015, 05:18 PM

(08-20-2015, 05:15 PM)Master Antrax Wrote: @Shebang The 403 forbidden error is coming from the CF page itself. You might be able to use a Keep-Alive connection to get by that.

I'm currently using a Keep-Alive header, unfortunately no dice.

lux · 08-20-2015, 05:20 PM

(08-20-2015, 05:18 PM)Shebang Wrote: I'm currently using a Keep-Alive header, unfortunately no dice.

https://github.com/Anorov/cloudflare-scrape

Merp.