If this project is an ad for their product (Upstash, promising "Highly Available, Infinitely Scalable"), then the last thing they'd want is a hug of death :/
/api/hn -> 502 {"error":"Search entry should have an initialized schema, command was: [\"SEARCH.AGGREGATE\",\"hn\",\"{\\\"$or\\\":[{\\\"title\\\":{\\\"$eq\\\":\\\"anthropic\\\",\\\"$boost\\\":5}},{\\\"text\\\":{\\\"$eq\\\":\\\"anthropic\\\"}}]}\",\"{\\\"by_month\\\":{\\\"$dateHistogram\\\":{\\\"field\\\":\\\"time\\\",\\\"fixedInterval\\\":\\\"30d\\\"}},\\\"top_authors\\\":{\\\"$terms\\\":{\\\"field\\\":\\\"by\\\",\\\"size\\\":6}},\\\"by_type\\\":{\\\"$terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":4}}}\"]"}
Our little startup got slashdotted in 1998 or so. I was the only one randomly awake at the time. Remember watching all the logs from our poor little server in realtime, ready to immediately kill anything or anyone threatening the overall availability.
512 kbps uplink, I think. Even accidental DoS was trivial. We had a self-hosted little data center at our office with the only available stupidly expensive commercial connection.
Felt some dread having to restart the main (async, single-process) web server a few times to keep things going due to bugs in our code. So many people on dial-up patiently waiting for the page to load.
Its funny that these days the bottleneck is usually the data layer. Servers are so powerful now that even your average $5 server can handle HN levels of load if configured correctly.
It's a HN clone, that syncs with HN that allows you to basically establish smaller private communities who can discuss anything that's on HN without actually being on HN.
It also indexes and let's you search through the DB which I find is really useful to find things that peak my interest.
One useful feature would be to normalize by total so that I can see changes in something as opposed to just total site growth. Right now I have to chart a single generic parameter but if I pick poorly it’ll confuse the issue.
one subtle consistency bug that made it hard for me to interpret when I was clicking around: the small thumbnail plot vs the full plot often (always?) seem to use different colors.
The blue / orange gets assigned to the opposite labels in the A vs. B when you click, which made it confusing to understand.
IMO, using AI to assign keywords to a broader group of strict synonymous keywords would make the comparison much more helpful.
Because in general we want to know the trend of categories more than of a word, asking for “auto pilot” for ex. should include “self driving”, FSD etc.
I would not like this. This is the kind of change that made google search so annoying. (Eg what if I want to track the history of 'self-driving' vs 'auto pilot' in sales pitches? Or more basically, what if the system wrongly interprets me wrongly?) Better to support | or similar old-fashioned search engine syntax and dwis and not dwim.
I also have a seperate page for the "Who is Hiring?" posts, here is the distribution of programming languages over each monthly "Who is hiring?" post in HN ever.
https://hackernewstrends.com/who-is-hiring
A minor suggestion - I'd like to be able to render the current graph taller (full height of my browser window).
Also some sentiment analysis on the "people" graphs would be very insightful (particularly for the likes of Edward Snowdon, Julian Assange, Elon Musk and Sam Altman). Perhaps colour the area under the graph red-orange-green based on the sentiment?
Just my idea. I'm working on a side project https://newsavista.com that aggregates news and tracks news trends and changing sentiment on the major stories. With cheap cloud LLMs (and "free" local LLMs) it turns out to be a trivial feature to build.
Hello, /api/hn -> 502 {"error":"Your database has been temporarily rate-limited, please contact support@upstash.com for further details."}
Hug of death
` /api/hn -> 504 An error occurred with your deployment FUNCTION_INVOCATION_TIMEOUT cle1::c8vgv-1782399959042-aeba3cae05ff `
If this project is an ad for their product (Upstash, promising "Highly Available, Infinitely Scalable"), then the last thing they'd want is a hug of death :/
Oof that would be hilarious/tragic
Downstash
Must stash
/api/hn -> 502 {"error":"Your database has been temporarily rate-limited, please contact support@upstash.com for further details."}
A cache would help.
I get
/api/hn -> 502 {"error":"Search entry should have an initialized schema, command was: [\"SEARCH.AGGREGATE\",\"hn\",\"{\\\"$or\\\":[{\\\"title\\\":{\\\"$eq\\\":\\\"anthropic\\\",\\\"$boost\\\":5}},{\\\"text\\\":{\\\"$eq\\\":\\\"anthropic\\\"}}]}\",\"{\\\"by_month\\\":{\\\"$dateHistogram\\\":{\\\"field\\\":\\\"time\\\",\\\"fixedInterval\\\":\\\"30d\\\"}},\\\"top_authors\\\":{\\\"$terms\\\":{\\\"field\\\":\\\"by\\\",\\\"size\\\":6}},\\\"by_type\\\":{\\\"$terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":4}}}\"]"}
back in my day we called this a good ole' fashioned slashdotting.
Our little startup got slashdotted in 1998 or so. I was the only one randomly awake at the time. Remember watching all the logs from our poor little server in realtime, ready to immediately kill anything or anyone threatening the overall availability.
512 kbps uplink, I think. Even accidental DoS was trivial. We had a self-hosted little data center at our office with the only available stupidly expensive commercial connection.
Felt some dread having to restart the main (async, single-process) web server a few times to keep things going due to bugs in our code. So many people on dial-up patiently waiting for the page to load.
It was exhilarating though :).
One of the things I love about HN is having stories like this in the comments from otherwise random unassuming usernames
Its funny that these days the bottleneck is usually the data layer. Servers are so powerful now that even your average $5 server can handle HN levels of load if configured correctly.
We will be with you shortly :)
yeah we killed it :(
Reminds me of this side project I'm working on.
https://gitlab/here_forawhile/torum
It's a HN clone, that syncs with HN that allows you to basically establish smaller private communities who can discuss anything that's on HN without actually being on HN.
It also indexes and let's you search through the DB which I find is really useful to find things that peak my interest.
Fixed link: https://gitlab.com/here_forawhile/torum
One useful feature would be to normalize by total so that I can see changes in something as opposed to just total site growth. Right now I have to chart a single generic parameter but if I pick poorly it’ll confuse the issue.
Very cool!
one subtle consistency bug that made it hard for me to interpret when I was clicking around: the small thumbnail plot vs the full plot often (always?) seem to use different colors.
The blue / orange gets assigned to the opposite labels in the A vs. B when you click, which made it confusing to understand.
This is actually very cool@
This is actually very cool!
The colors of the lines of the big graph are inverted compared to the smaller ones.
The 'flash vs html5' chart looks strange juxtaposed with that conclusion
IMO, using AI to assign keywords to a broader group of strict synonymous keywords would make the comparison much more helpful.
Because in general we want to know the trend of categories more than of a word, asking for “auto pilot” for ex. should include “self driving”, FSD etc.
I would not like this. This is the kind of change that made google search so annoying. (Eg what if I want to track the history of 'self-driving' vs 'auto pilot' in sales pitches? Or more basically, what if the system wrongly interprets me wrongly?) Better to support | or similar old-fashioned search engine syntax and dwis and not dwim.
Synonym functionality is good as long as there's an easy way to disable it, either globally or by wrapping the term in quotes.
Very cool idea. Shows programming language trends pretty well.
https://hackernewstrends.com/?q=Nim&q=Rust&q=Zig
This is great, I was just hoping to find a tool like this and specifically scoped to "Show HN" posts? Is there a way to do that?
Great idea actually, I'll add that as well for sure
We had to take the site down for a second, it'll be online in a few minutes. Thanks for trying it out
Very cool!
I'd love to have some sort of normalization option to separate more subtle positive trends from the general increase in number of posts.
I'd be interested in "google ngram for hacker news" instead
What is missing from it? I've used ngrams as well and I this was partly inspired by that.
Really cool! Where would you get the data for something like this? Is it open, or its scraped?
Hello HN,
This was a small project of mine after I've found out that I can simply the whole hackernews archive (~48GB) and play around with it.
You can compare terms just like in google trends and you can also see the exact posts & comments from that time.
I like that you can discover what went crazy in the timeline, they just come up as small burst of activity, it's quite fun to play around with it. https://hackernewstrends.com/?q=litecoin&q=dogecoin&q=solana...
I also have a seperate page for the "Who is Hiring?" posts, here is the distribution of programming languages over each monthly "Who is hiring?" post in HN ever. https://hackernewstrends.com/who-is-hiring
Any kind of feedback is welcome.
Honestly the HN archive is very valuable. If you had it all on a local db with everything indexed you basically end up with a offline search engine.
Where is this archive located you speak of?
It's on firebase, per https://github.com/hackernews/api
This is excellent.
A minor suggestion - I'd like to be able to render the current graph taller (full height of my browser window).
Also some sentiment analysis on the "people" graphs would be very insightful (particularly for the likes of Edward Snowdon, Julian Assange, Elon Musk and Sam Altman). Perhaps colour the area under the graph red-orange-green based on the sentiment?
Thanks for the feedback, noted the full-screen request.
The sentiment analysis is very interesting, I can do that easily. Could be a new page as well. Did you see this anywhere else or just your idea?
Just my idea. I'm working on a side project https://newsavista.com that aggregates news and tracks news trends and changing sentiment on the major stories. With cheap cloud LLMs (and "free" local LLMs) it turns out to be a trivial feature to build.
Woah, great work!
I am really liking the trend for "linux": https://hackernewstrends.com/?q=linux
Funny how closely that tracks with windows
https://hackernewstrends.com/?q=linux&q=windows
Does the trend only show absolute numbers? Because I think it should be divided by the number of posts during the time frame (day?).
Nice. Is the data points y-axis normalized by total amount of comments at that time?
Edit: Nvm seems like absolute count if you click the graph.
Love this, seems to struggle with newly indexed words. Will try again when the FP load is gone
The example comparisons made me smile. Well done!
Really beautiful, informative, and functional layout. Great work!
But can it discover new trends without having to type the keywords?
insane ! I don't know if it's possible but it would be huge if we had access to the localisation of the trends
Yup your upstash is rate limited
too slow or broker right now
nice. i guess AWS still had nothing to fear from GCP/Azure. ty for this
COOOOOOOOOOL!!!!!!
This is the only HN submission I ever upvoted because it is amazing
Thanks, it was my first ever post here as well, would you look at that
If more people spent time on /new looking for awesome stuff and vouching for dead items, HN would be a better place.
I know right
ooh this is sick! really nice ui too!
https://hackernewstrends.com/?q=furries&q=furry
Hmm, did I break something?
love it