+- Sinisterly (https://sinister.ly)
+-- Forum: Coding (https://sinister.ly/Forum-Coding)
+--- Forum: Coding (https://sinister.ly/Forum-Coding--71)
+--- Thread: [Challenge] Sinister.ly on the commandline. (/Thread-Challenge-Sinister-ly-on-the-commandline)
RE: [Challenge] Sinister.ly on the commandline. - Eclipse - 08-20-2015
(08-20-2015, 04:13 PM)Shebang Wrote: Well, every time I noticed something wrong and I went to change it, I obviously had to re-login and use the command again, so 2 requests for every change I made. Probably did that about every 30-90 seconds.
Maybe get the full response from the server instead of just the code. It's most likely just cloudflare's browser integrity check. Make sure your HTTP request is correct.
RE: [Challenge] Sinister.ly on the commandline. - Equinox - 08-20-2015
(08-19-2015, 07:52 AM)Oni Wrote: Hehe. If someone made a client to browse/post for Linux that would be interesting.
Are you implying none of us here are on anything GNU/Linux based?
(08-20-2015, 08:31 AM)Lux Wrote: You know what, let's not make this a challenge, but a project. I'm using NodeJS. Anyone wanting to assist? @"Stocking" maybe?
Why are you using NodeJS exactly? -SNIP-. Aside from that, Python has a pretty good workflow to go with it in this sort of project, and it's far easier to test with.
RE: [Challenge] Sinister.ly on the commandline. - Shebang - 08-20-2015
(08-20-2015, 04:27 PM)eclipse Wrote: Maybe get the full response from the server instead of just the code. It's most likely just cloudflare's browser integrity check. Make sure your HTTP request is correct.
I am obtaining (at least attempting to) the full response. I don't see how my request would be "wrong", because it was working initially. It was just after using it a bunch I hit that "wall".
RE: [Challenge] Sinister.ly on the commandline. - Equinox - 08-20-2015
(08-20-2015, 04:37 PM)Shebang Wrote: I am obtaining (at least attempting to) the full response. I don't see how my request would be "wrong", because it was working initially. It was just after using it a bunch I hit that "wall".
You have to specify a user agent header, if you haven't figured that out already (pretty sure you did though). Here, use my user-agent if you're too lazy to fetch yours.
Code:
Mozilla/5.0 (X11; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0RE: [Challenge] Sinister.ly on the commandline. - Shebang - 08-20-2015
(08-20-2015, 04:39 PM)Stocking Wrote: You have to specify a user agent header, if you haven't figured that out already (pretty sure you did though). Here, use my user-agent if you're too lazy to fetch yours.
Code:Mozilla/5.0 (X11; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0
Yep, already have a user agent set.
RE: [Challenge] Sinister.ly on the commandline. - Akane - 08-20-2015
(08-20-2015, 04:42 PM)Shebang Wrote: Yep, already have a user agent set.
Weird. Try again in half an hour. Or try proxying your request.
RE: [Challenge] Sinister.ly on the commandline. - Equinox - 08-20-2015
![[Image: CU1ei3G.png]](http://i.imgur.com/CU1ei3G.png)
Well, we're definitely getting somewhere.
RE: [Challenge] Sinister.ly on the commandline. - Master Antrax - 08-20-2015
@Shebang The 403 forbidden error is coming from the CF page itself. You might be able to use a Keep-Alive connection to get by that.
RE: [Challenge] Sinister.ly on the commandline. - Shebang - 08-20-2015
(08-20-2015, 05:15 PM)Master Antrax Wrote: @Shebang The 403 forbidden error is coming from the CF page itself. You might be able to use a Keep-Alive connection to get by that.
I'm currently using a Keep-Alive header, unfortunately no dice.
RE: [Challenge] Sinister.ly on the commandline. - lux - 08-20-2015
(08-20-2015, 05:18 PM)Shebang Wrote: I'm currently using a Keep-Alive header, unfortunately no dice.
https://github.com/Anorov/cloudflare-scrape
Merp.