Recent searches
Search options
@berkes @helgek @darnell @volkris @nickapos Rails is not the main reason for the scaling issues. Mainly background jobs where Ruby is just a wrapper for transcoding libs etc. You can scale Puma and Sidekiq horizontal by adding more compute/workers. Same for storage. Meaning its not the language or the framework, its the protocol and overall architecture which leads to more resource consumption.
@jan @berkes@bitcoinhackers.org @helgek @darnell @volkris @nickapos I also don’t think it can be so much an issue with raw compute power. That only affects how the server performs within it’s own workloads, not between computers at all.
Network IO takes magnitudes longer than compute. But while doing it, the thread often waits most of the time, doing very little compute.
Maybe a layer of distributed caches? Delivery is the key.
@gimulnautti @jan @berkes @helgek @darnell @nickapos
Remember, this is not theoretical. This is actual experience hearing from real people running instances and finding themselves having to unexpectedly shell out more money for higher hosting prices that they weren’t expecting.
The protocol requires poorly scaling processing and bandwidth.
And that’s not even getting into expensive design decisions that Mastodon in particular put on top of everything else. For example the intentional decision not to redistribute image previews but instead require each instance to go out and pull its own image preview, duplicating that effort throughout the whole platform.
It seriously sounds like [almost] nobody involved in this from protocol design up through platform implementation gives a second thought to what’s going to happen at scale.
And I may have said it in this thread, but when I was in school for computer science we were hammered with big-O analysis of algorithmic scaling but someone recently told me that’s not emphasized in school these days. It sure looks like that’s the case.
@volkris @gimulnautti @jan @berkes @helgek @nickapos We definately need to figure out how to operate #Mastodon / #Fediverse at scale.
@Sujiyan revealed that running Pawoo (https://pawoo.net), MSTDN Japan 
(https://mstdn.jp) & Mastodon Cloud (https://mastodon.cloud) surpassed $1 million over 2 years (legal, staff, servers, etcetera).
https://darnell.day/zooming-with-pawoo-net-new-owner-suji-yan