Code News

Explain the Cloud Like I'm 10


    Explain the Cloud Like I'm 10


    “Todd, can you explain ‘The Cloud’? What is it?” I was asked this question at lunch by Joe, a fellow tour member on a recent trip Linda (my wife) and I took to France.

    It was not a question I was expecting on vacation, but with many years of experience as a programmer, a lot of it spent in cloud computing, it’s a question I should have been able to knock out of the park.

    Except I didn’t. My answer stank. I hemmed and hawed. I stuttered and sputtered. I could tell that nothing I said was making any sense. I gave a horrible answer, and it has haunted me ever since.

    While talking, I noticed a lot of other people at the table were interested in my answer as well. It seemed a lot of smart people were confused about the cloud.

    When I got back home I did a lot of research. I was trying to redeem myself by finding the perfect book to recommend. I couldn't find one! So I decided to write Explain the Cloud Like I'm 10. It's the answer I wish I'd given Joe in France.

    Here's what I came up with:

    • What Does the Cloud Look Like?
    • Why is it called a ‘cloud’ anyway?
    • If You Use Online Banking You Already Understand The Cloud
    • What is the Cloud?
    • What is a Service?
    • What is a Cloud Service?
    • Facebook Messenger is a Cloud Service. How does it Work?
    • What isn’t a Cloud Service?
    • What does Stuff Mean?
    • What does Stored in the Cloud Mean?
    • What is a Device?
    • What is a Program or App?
    • What is the Internet?
    • What is a Datacenter?
    • What is Cloud Computing?
    • The Good, the Bad and the Ugly of Cloud Services
    • Kindle: Amazon’s Cloud Service for Reading Ebooks
    • iCloud: Apple’s Cloud Service for Syncing Data
    • Google Maps: A Cloud Service for Navigation
    • Cloud DVR: TV in the Cloud
    • Netflix: What Happens When You Press Play?
    • Fighting the Ecosystem Wars in the Proactive Cloud
    • Does stormy weather affect cloud computing?
    • If I say Something is in the cloud, do you know what it means?

    Beginners will find the cloud explained from the beginning. Little prior knowledge is assumed. I use lots of pictures, lots of examples, and many somewhat questionable analogies. We build our way up to understanding the cloud idea by idea.

    At the end, we'll use what we've learned to understand the inner-workings of Kindle, iCloud, Google Maps, Nest, and Cloud DVRs. You will understand the cloud and what it means for something to be in the cloud.

    I also think even intermediate level people will learn something. I cover a lot. You can read an example chapter at Netflix: What Happens When You Press Play? 

    If you are someone or know someone who could use a good book on the cloud, then I'd appreciate giving Explain the Cloud Like I'm 10 a try. And please write a review. That would really help.


    1 month 5 days ago

Stuff The Internet Says On Scalability For December 15th, 2017

  • Hey, wake up, it's HighScalability time:


    Merry Christmas and Happy New Year everyone! I'll be off until the new year. Here's hoping all your gifts were selected using machine learning.


    If you like this sort of Stuff then please support me on Patreon. And I'd appreciate your recommending my new book—Explain the Cloud Like I'm 10—to anyone who needs to understand the cloud (who doesn't?). I think they'll like it. Now with twice the brightness and new chapters on Netflix and Cloud Computing.


    • 157 terabytes: per second raw data output of the Square Kilometre Array; $11 million: made by a 6 year old on YouTube; 14TB: helium hard drive; 1: year education raises IQ 1-5 points; 10: seconds mining time to pay for wifi; 110 TFLOPS: Nvidia Launches $3,000 Titan V; 400: lines of JavaScript injected by Comcast; 20 million: requests per second processed by Netflix to personalize artwork; 270: configuration parameters in postgresql.conf; hundreds: eyes in scallop from a unique mirroring system; $72 billion: record DRAM revenue; 20: rockets landed by SpaceX; 

    • Quotable Quotes:
      • Bill Walton: Mirai was originally developed to help them corner the Minecraft market, but then they realized what a powerful tool they built. Then it just became a challenge for them to make it as large as possible.
      • Stephen Andriole: The entire world of big software design, development, deployment and support is dead. Customers know it, big software vendors know it and next generation software architects know it. The implications are far-reaching and likely permanent. Business requirements, governance, cloud delivery and architecture are the assassins of old "big" software and the liberators of new "small" software. In 20 years very few of us will recognize the software architectures of the 20th century or how software in the cloud enables ever-changing business requirements.
      • Melanie Johnston-Hollitt: There is not yet compute available that can process the data we want to collect and use to understand the universe. 
      • Brandon Liverence: Credit and debit card transaction data shows, at these businesses, the average customer in the top 20 percent spent 8x as much as the average customer from the bottom 80 percent.
      • @evonbuelow: After looking at the source code for a series of k8s components & operators, I'm struck by how go (#golang) is used more as a declarative construct than a set of procedural steps encoding sophisticated logic.
      • apandhi: I had a run-in with CoinHive this weekend so I did a bit of research. Most modern computers can do about 30/h a second. Coinhive currently pays out 0.00009030 XMR ($0.02 USD) per 1M hashes. For a 10 second pause, they'd mine 300 hashes (about $0.000006 USD). To make $1 USD, they'd need to have ~166,666.66 people connect to their in store WIFI.
      • @matt_healy: Went from zero clue about #aws codepipeline and friends yesterday, to setting up an automatic Lambda and API gateway deployment with every git push in production today. Awesome!
      • lgierth: Pubsub is probably one of the lesser known features of IPFS right now, given that it's still marked as experimental. We're researching more efficient tree-forming and message routing algorithms, but generally the interface is pretty stable by now. Pubsub is supported in both go-ipfs and js-ipfs. A shining example of pubsub in use is PeerPad, a collaborative text editor exchanging CRDTs over IPFS/Pubsub
      • Manish Rai Jain: Given these advancements, Amazon Neptune’s design is pre-2000. Single server vertically scaled, asynchronously replicated, lack of transactions — all this screams outdated.
      • There are more quotes. So many more. More. More. More. Yep, there's even more.

    Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

    1 month 1 week ago

Netflix: What Happens When You Press Play?


    This article is a chapter from my new book Explain the Cloud Like I'm 10. The first release was written specifically for cloud newbies. I've made some updates and added a few chapters—Netflix: What Happens When You Press Play? and What is Cloud Computing?—that level it up to a couple ticks past beginner. I think even fairly experienced people might get something out of it.

    So if you are looking for a good introduction to the cloud or know someone who is, please take a look. I think you'll like it. I'm pretty proud of how it turned out. 

    I pulled this chapter together from dozens of sources that were at times somewhat contradictory. Facts on the ground change over time and depend who is telling the story and what audience they're addressing. I tried to create as coherent a narrative as I could. If there are any errors I'd be more than happy to fix them. Keep in mind this article is not a technical deep dive. It's a big picture type article. For example, I don't mention the word microservice even once :-)


    Netflix seems so simple. Press play and video magically appears. Easy, right? Not so much.


    Given our discussion in the What is Cloud Computing? chapter, you might expect Netflix to serve video using AWS. Press play in a Netflix application and video stored in S3 would be streamed from S3, over the internet, directly to your device. 

    A completely sensible approach…for a much smaller service. 

    But that’s not how Netflix works at all. It’s far more complicated and interesting than you might imagine.

    To see why let’s look at some impressive Netflix statistics for 2017.

    • Netflix has more than 110 million subscribers.
    • Netflix operates in more than 200 countries. 
    • Netflix has nearly $3 billion in revenue per quarter.
    • Netflix adds more than 5 million new subscribers per quarter.
    • Netflix plays more than 1 billion hours of video each week. As a comparison, YouTube streams 1 billion hours of video every day while Facebook streams 110 million hours of video every day.
    • Netflix played 250 million hours of video on a single day in 2017.
    • Netflix accounts for over 37% of peak internet traffic in the United States.
    • Netflix plans to spend $7 billion on new content in 2018. 

    What have we learned? 

    Netflix is huge. They’re global, they have a lot of members, they play a lot of videos, and they have a lot of money.

    Another relevant factoid is Netflix is subscription based. Members pay Netflix monthly and can cancel at any time. When you press play to chill on Netflix, it had better work. Unhappy members unsubscribe.

    Netflix operates in two clouds: AWS and Open Connect.

    How does Netflix keep their members happy? With the cloud of course. Actually, Netflix uses two different clouds: AWS and Open Connect. 

    Both clouds must work together seamlessly to deliver endless hours of customer-pleasing video.

    The three parts of Netflix: client, backend, CDN.

    You can think of Netflix as being divided into three parts: the client, the backend, and the CDN. 

    The client is the user interface on any device used to browse and play Netflix videos. It could be an app on your iPhone, a website on your desktop computer, or even an app on your Smart TV. Netflix controls each and every client for each and every device. 

    Everything that happens before you hit play happens in the backend, which runs in AWS. That includes things like preparing all new incoming video and handling requests from all apps, websites, TVs, and other devices.

    Everything that happens after you hit play is handled by Open Connect. Open Connect is Netflix’s custom global content delivery network (CDN). When you press play the video is served from Open Connect. Don’t worry; we’ll talk about what this means later.

    Interestingly, at Netflix they don’t actually say hit play on video, they say clicking start on a title. Every industry has its own lingo.

    By controlling all three areas—client, backend, CDN— Netflix has achieved complete vertical integration. 

    Netflix controls your video viewing experience from beginning to end. That’s why it just works when you click play from anywhere in the world. You reliably get the content you want to watch when you want to watch it. 

    Let’s see how Netflix makes that happen.

    In 2008 Netflix Started Moving to AWS

    1 month 1 week ago