Ramageddon, The PC Paradigm Shift, and Federated Learning

Ever since I lost my martyred laptop (coupled with the traumatic baggage of dealing with repair services), I've come to the conclusion that buying a new laptop is simply too expensive. There are additional reasons why I shifted back to a desktop PC (in this case, a Mac Mini). First, performance. With roughly the same benchmark results, a PC costs significantly less. Second, screen size. There’s no denying that as I approach the big 4-0, my eyes don't lie. A 14-16 inch laptop screen now looks ridiculously small. Give me at least 24 inches and a 2K resolution. Then there are other side reasons, like how my mobility has drastically decreased (yay, WFH), right down to the absolute truth that "there is no such thing as a mechanical laptop keyboard"—and I refuse to sacrifice my tactile brown switches.

But the questions are getting incredibly complex. Is the current spike in RAM and SSD prices by design, orchestrated by certain parties, or is it just market mechanics doing its thing? Looking ahead, are we doomed to keep upgrading our PCs (getting more expensive by the minute), or will we just surrender and let giant servers do all the heavy lifting?

The PC Paradigm Shift

Historically, there has been a massive divide between two camps.

First, the centralization camp (thin client). This school of thought believes that the internet and bandwidth will only get faster and cheaper, meaning people will only need "dumb PCs." As long as it turns on, has an input (keyboard/mouse), and an output (monitor), that's enough. The rest is handled by massive, high-capacity servers. In this camp, we have IBM, Sun Microsystems (if you remember this company, congratulations, you're old), Chromebooks, Cloud Gaming, and the like.

Second, the decentralization camp (thick client). This camp believes that hardware will get cheaper and faster thanks to Moore's Law, leading people to build god-tier, sultan-level PCs in droves. Aside from craving low latency, these folks believe data privacy shouldn't be leaked to monopolistic megacorporations. This idea was born as a rebellion against the hegemony of mainframe companies or giant corps (like IBM in the 1970s). Enter the gang: Steve Jobs, Steve Wozniak, Bill Gates, Paul Allen, and Gordon Moore from Intel. The Apple I and Apple II, for instance, carried the vision of bringing computers "into the home."

In today's modern edge computing era, we're watching the historical pendulum swing once again. We see tech giants divided—some are pushing hard for Cloud Computing, while others are aggressively pushing for overpowered PCs just so we can run local AI.

Federated Learning

In the midst of this RAM apocalypse (which is predicted to last until next year), a "middle ground" emerges, bringing a concept that feels awfully familiar to existing technologies: federated learning. This concept was introduced as a distributed machine learning approach that allows AI model training across a massive, decentralized pool of data. The raw data belonging to each client stays safely local on their respective devices and is never exchanged or transferred out. The clients train the model locally, and the only thing sent back to the server is the model update (like the learned weights or gradients).

Illustration made by, as you can guess, ChatGPT

This idea hit the mainstream around 2016, championed by several Google researchers, notably H. Brendan McMahan, Eider Moore, and Daniel Ramage. But if you look closely, this concept is essentially a carbon copy of the "gotong royong" (communal crowdsourcing) spirit we've seen in the following things:

BitTorrent.

The pioneer of the communal spirit for bandwidth-starved digital peasants in the early internet era. Say we need to download a 20GB file; we don't download it from one central server (which would inevitably crash the network). Instead, we grab tiny chunks of that file from 100 other people's computers (about 200MB each) simultaneously. In exchange, our computer seeds the chunks we already have to others.

Waze or Google Maps.

This navigational crowdsourcing shares a similar philosophy. Our phones act as data "donors," which the central server then aggregates with data from others in the exact same location. We get the final conclusion: a beautiful map showing us exactly where the traffic jams are.

SETI@home or Folding@home.

This is brute-force crowdsourcing. Scientists have massive homework assignments—like searching for alien signal patterns (if they exist), or finding DNA/protein combinations to cure cancer. Every massive task is sliced into millions of micro-tasks. Our idle PCs act as volunteer number-crunchers to hitchhike on the calculations, and the results are sent straight back to headquarters.

Federated Learning might just be an alternative workaround for these skyrocketing chip prices. Which is pretty absurd when you think about it, considering silicon dioxide is literally just sand. You could scoop some up at Parangtritis beach—and if it’s your lucky day, you might just bump into Nyi Roro Kidul.

PS: this article is previously written in Bahasa Indonesia.