I was bored between jobs NOT AI SLOP.
Posted by ubreakitifixit@reddit | Python | View on Reddit | 9 comments
I'm currently between jobs, so in my boredom I built scraper that can bypass Reddits notoriously difficult to gain API.
I'm not a software dev or have any skills in the field nor have I worked in the field, I'm a mechanic who likes coding lol.
Its a simple python script that looks for anything you ask it. I used to find new or up-and-coming memes, I was planning to use that and funnel it's to a print on demand service which mocks up and auto uploads ready to buy clothing to a self built website for sale, once I built the script I couldn't be bothered to build the rest of the backend (ADHD is great lol)
it uses stealth ingestion that mimics Chromes TLS fingerprint then appends .json to old Reddit URLs which forces Reddit to hand over clean raw data, then finds metadata explicitly tagged as an image by Reddit by looking for file extensions in image format (jpeg, PNG ect)
Once the script has found a clean list of images it's calculates a viral score based momentum from upvotes, comments, engagement and rising posts.
On first run it takes the score and saves it to a JSON database, subsequent runs finds new content and cross-references old content on the JSON file and flags it if has risen over 15% from previous runs.
Just a simple project that works to a point but I'm too lazy to actually use it, besides I'm sure Reddit will find it one way or another and patch my loophole.
As I said I'm not a dev or a trained python powerhouse in the slightest. A pure greenhorn amateur.
Python-ModTeam@reddit
Your post was removed for violating Rule #2. All posts must be directly related to the Python programming language. Posts pertaining to programming in general are not permitted. You may want to try posting in /r/programming instead.
TheDivinityGod@reddit
" r/python "
Either_Extreme_2236@reddit
That's pretty solid work for someone who isn't even in tech professionally. The TLS mimicking and using .json endpoints is clever - most people would just bang their head against the official API limits
Your viral score calculation sounds interesting, especially the 15% threshold for flagging rising content. Too bad about the ADHD brain hitting after you got the fun part working though, happens to best of us
Reddit probably has bigger fish to fry than one mechanic scraping memes but yeah they'll probably close that loophole eventually. Still cool that you built something functional just from boredom
Sensitive_One_425@reddit
Great AI comment
ubreakitifixit@reddit (OP)
Maybe, but it's better than nothing
wRAR_@reddit
That's sad.
ubreakitifixit@reddit (OP)
I appreciate that man, thank you. Although I'm not in this for the praise or distinction. Barely a hobbyist lol who took a dip but I've leaned a lot
Sensitive_One_425@reddit
So you’re just scraping the site but using the JSON instead.
ubreakitifixit@reddit (OP)
I'll level with you, I had no idea what I was doing past the point of scraping so any critique is valid.