AWS re:Invent: Effective FinOps Optimization with Erick Carlin of Prosper Ops
Eric, welcome. Glad you're here. Yeah, pleasure to be here. It's been a while since you were at rackspace. Uh Yeah, I, I left in 2018.
So was that five years? And when did you join? How long were you there? I joined in 20 I'm sorry, 2008. So was there a good run? What was your first job? Uh I worked. So, John Gates hired me who was the CTO R and I worked in a group called Rack Labs, which uh was the group that basically ended up building the racks space public cloud. Wow. So the, the cloud files technology came out of rack labs which then was contributed to open stack as the swift project.
And then took that and went on to launch the first support for a public cloud that Rackspace had and that was for Aws. Yeah. So there was a team already working on storage. I joined and sort of took up the mantle of compute and then uh we ended up acquiring slice host and Jason seats and I kind of joined forces and built uh cloud servers and then had to just sort of turned into open stack over time. So, yeah, it was, it was great. I, I love those years of rack space. It's amazing. And open sack alive
and, well, yeah, it's great and, and, uh, Mark and Jonathan are still running strong and, um, it's great just to see that. Yeah, continue to progress. I love how Lanham used to call that Rack Spaces gift to the world. Yeah, because it really has. I mean, it's got used in some really
amazing places and still being, I just talked to a customer, uh, you know, here at the show that's said they're, they're still running up stack. So it's amazing. Cool. Well, and it had, um, we stopped talking a lot about it at rackspace, but it's, it's back and alive and well, and there's a lot of reinvestment going on there. I mean, the, the foundation is still going strong. So, and then went on to help launch our Aws practice at, at Rackspace. Yeah. So in 2015,
Rackspace made the decision to, um, partner with Aws as well. And so I had the privilege of being one of the sort of internal founders of that business. And in fact, I remember the, you know, I was downstairs in the expo floor looking at the Rack Space booth, which looks incredible. It's like a really nice booth with the, the sort of led lights and very slick. I, I remember our very first rack Space booth in 2015.
It was back in the corner, it was a small booth and I remember our marketing guy bought in a wingless airplane because our offer was called Aviator. And, uh, he was able to somehow get this airplane uh, into the showroom floor. It had no wings. Uh, so we would joke with him about that, but, uh, it's just great to see like how far things have come. And, um, it's great. Yeah, and there's still a lot of that original, you know, the thoughts about how we, how we look after customers and manage those customers. A lot of that heritage is alive and well, totally, um, and great booth traffic when I've walked by. So, yeah, but, uh, but you left, of course, because you recognize there was an opportunity or a problem really in the world of, of cloud computer. So, so when we were
serving customers in the managed Aws business, we were primarily helping them with two problems. One was around security and the second was around cost and, you know, we were, um, integrating sort of leading tooling, third party tooling as well as building our own tooling and sort of combining that together with the power of rackers and this care for customers you're talking about and delivering outcomes to customers. And, you know, fops wasn't even a thing back then this 2015, 2016 days. And, but we did realize that, um, there was sort of a, an opportunity that was missing in the market, which is really um rather than take kind of a visibility approach that you kind of combine with, with human expertise to deliver outcomes.
How do you make this a 100% you know, computer science problem? Like how do you automate this entirely sort of end to end? And no one was really doing that in the market. This was also the dawn of sort of A I and this idea that software can deliver, not just, you know, um visibility and recommendations but can actually deliver outcomes for customers. And so we said, we think there's, there's something to this and ended up uh you know, starting prosper ops, you mentioned that, that fin ops really wasn't a thing because if we think about how computing was acquired and used for decades before that, it was a, it was a procurement cycle, you know, budgeting and procurement cycle. So in August, we figure out budgets for the next year, the next year would come along and it had a pile of money to spend and they would go and they wouldn't go spend it, but they would go talk to the finance team can really spend it, they would talk to procurement and they would go and spend it and then it would show up and it become this multi month multiyear process.
But with cloud computing, that's a different world. I mean, you get it with servers or even serverless, it's a different world of, of not only utilizing the tooling but that, that, that buying cycle, one is different and two, you know, when we would buy new servers, we would get rid of the old ones. A lot of times we would decommission and move them out of the way. But that wasn't the case with cloud computing. Yeah. What's, what's novel and powerful
about cloud computing is the fact that it's utility based, right? So I'm uh you pay as you go. Uh and you pay per hour per second in some cases per resource and you have effectively unlimited capacity. Now, obviously there is a capacity limit, but for any individual, for all intents and purposes, it's infinite, it's effectively infinite.
And uh you know, that that's wonderful in terms of being able to, to test cost effectively, to be able to, to kind of scale and have your costs adjust to your, your actual usage. But it introduces a second order problem. And that second order problem is the, is the idea of waste because you can, it's so easy to, to instantiate a server and then forget about it or uh you know, dump a bunch of storage here and leave it there forever. And so you can imagine just that activity over time in any sort of reasonably large organization just leads to this massive kind of waste problem.
And you know, that's really what prospers is focused on. And I like to sort of help people understand the magnitude of the problem. Uh If you look at the cloud providers, most recent kind of earnings reports, uh, Amazon has about a $92 billion run rate. If you take their Q three revenue times times four. Google is at about 32 something like that. I can't remember. Exactly. And Azure doesn't break out or Microsoft doesn't break out Azure.
Uh exactly. They combine it with a small company, you may have heard of called github. Uh and, and other things, but it's estimated to be in the high fifties. So you put all that together, that's about 100 and $84 billion uh you know, per year spend on cloud and growing and growing. And by multiple estimates that I've seen uh cloud waste is about 30% of all cloud spend. So if you, 30% times 100 and 84 billion is over $50 billion that's estimated to be sort of waste and waste is defined as basically resources that you're uh deploying that you could otherwise, you know, uh successfully run your application without. So it's craft,
it's, it's oversizing, it's not getting the sort of the rate discounts that you could otherwise receive. So all the things that you could otherwise do to sort of achieve your outcome in a more efficient way. Um And so that's just a tremendous amount of money. In fact, I, you know, 2023 is the year of generative A I so II, I gave a talk at dinner and I, I said, I'm giving a talk and I need the audience. I want the audience to have sort of a creative illustration to understand how much $50 billion is. And Chet GP T came back and said, if you take dollar bills and you stack them end to end $50 billion would be to the moon and back 10 times. Oh my gosh. So
uh it's just a crazy amount of money and it's not like the cloud providers are somehow uh you know, nefarious. Uh It's just that this second order problem exists, there's so many benefits and so many good things that happen. But waste is just a, a second order effect of the utility computing. Uh what an opportunity to get in and help, help solve. So let's define it. Uh And, and how you're identifying and help creating then solutions around um different areas. So you mentioned storage,
we talk about it on the compute side and then there's the economic side. Is there a better way to buy it kind of prosper ops that? So there, there's uh a whole bunch of ways to kind of attack the the waste problem. Uh what we focus on and, and where we've started is on a concept known as rate optimization. So there's sort of two primary levers. You have one is uh usage optimization which basically says use less stuff like don't use smaller stuff, use less stuff. The second lever is rate optimization which says for the stuff that you use, you wanna get the best price possible for it. And so we focus
uh right now on rate optimization. And so within the world of uh Aws, uh you have ways of making commitments to Aws uh in exchange for discounts. And uh you know, Amazon has created some really cool products, uh reserved instances and savings plans.
Uh and our platform effectively will um completely automate that for customers. So, uh right now, you know, the I, I've never met a fed ups practitioner who it just kind of kicking back and has nothing to do. There's always more things on their plate than there is time to do. And there's a lot
of high order, uh you know, problems that they want to solve. Uh but they're spending time doing things like sort of R I and savings plan uh management. And so what our software does is we will uh take very, very lightweight permissions to basically uh monitor customers usages in real time. And we build a model of uh their environment. So what is it, what does it look like?
And then we have a set of algorithms. Uh We use uh you know, optimization algorithms, some lightweight A I. And we basically figure out what the optimal set of commit discount instruments is to maximize their savings. And then we have an execution engine that implements that for customers. And so what they get is effectively an autonomous savings outcome where they just point our software at their environment configure some settings one time. And then our platform is just watching in real time and continually optimizing their environment to ensure that they ensure that they get.
Well, you made the point earlier and now you've really unpacked why, why it's impactful. And that is, you know, even Aws has built some visualization tools for, you can see where the spend is and opportunities, but that requires human eyes to look at as opposed to what you just said. And that's your software which is continually day and night wrenching the platform. Yeah. And it's not that visibility
is unimportant. It's absolutely important, sort of necessary but not sufficient to really attack the waste problem, right? If, if visibility, because visibility tools have been around for a long time, Amazon has some great native visibility tools. There's some, you know, great third party tools as well. Uh
You absolutely need visibility to understand your costs and show back your costs and you know, be able to detect cost anomalies and all these things. But if that was sufficient to eliminate waste, we wouldn't have $50 billion of waste, right? And so the idea is the cloud is so dynamic, it's it's changing. Um It's very, it's a very difficult problem set for a human to kind of address because you know, things are auto scaling at three o'clock in the morning and you know, in, in any sufficiently complex environment it's always changing. And so you really need to have algorithms that are watching in real time and just making changes. So we always say that,
you know, if you think about Fin Ops is like a fly wheel and you want to get that thing spinning where you're really understanding your costs, optimizing your costs, et cetera. It's really the combination of kind of visibility plus automation. And if you can get those two sort of uh categories, you know, working really well, you can get that fins, fly wheel spinning and really, you know, have a, have an efficient FIN S, you know, practice and organization. OK. So you're saving people money. How are you, how are you making money? Uh Well, that's, that's an interesting question. So uh we charge a percent of savings.
And so, uh you know, some of the other models that exist in the environment for tools that are not generating outcomes are based on a percent of spend. Um And you know, that's, that seems a little backwards. Well, uh uh II I sort of liken it to in real estate. If you are
buying a house, you might have a buyer's broker that's representing you. Generally, he or she will get a percent of the sale price of the house. Well, his or her job is to negotiate the lowest price possible for you, but they actually make more money higher the price. So there's, there's some slightly misaligned incentives. That's kind of the way it is for tools that charge a percent of spend because you're buying these tools to optimize your cost. But the better you do with their tool, the less revenue that they're gonna make. So, so we
reverse relations. It is and so what we wanted to do is do something different and we wanted to perfectly align incentives with our customers. And because we generate outcomes, not just visibility of recommendations, we can basically stand behind the results that our platform delivers. And so we take
a percent of savings. So if our platform doesn't generate savings, we make no revenue. Uh and when we do generate savings, the customers get the vast majority of it, you know, we take a small percentage of it as a saving share, but the vast majority of every dollar flows to customers. And so,
uh you know, as our, the, the better job we do, the more our revenue increases, but the more the customer saves. And so it's kind of a win, win, everyone's aligned relationship. Yeah, total total alignment. And uh it's been great, you know, thankfully, our churn is extremely low and you don't want to keep saving money because cloud spend is going up there. People are using more, they're migrating more and they're coming up with more workloads there and, and once they, uh you know, I was talking to a customer who just signed up, they, they came to one of our breakfast and he said, when I was signing this contract, I thought this is a vendor for life, uh which is like a say that, get your recorder out to say that just watch it when you feel.
I think there's some skepticism when, you know, you know, it's sort of like autonomous driving, right? Like there's some skepticism about, I'm, I'm gonna hand off, you know, control to software sort of execute and, you know, what happens, etcetera. So we find there's a little bit of and rightfully so, you know, skepticism at the beginning, but once they see our software and see the results and see it over time, um you know, they, they are very happy to sort of move on, you know, what we wanna do is empower the Fin Ops practitioner. They get so many things on their plate. And so how do we come in and just start chipping away at this waste, chipping away at just, uh you know, work on their plate and just elevate them to sort of be able to handle sort of higher order activities. II, I tend to think of Fin apps
today as like playing whack a mole, right? There's just all these cost anomalies and things that are popping up and it's, it's, you're just attacking them. Um And, and it's sort of like an like a sis and task of like continually when you can apply software to handle the majority of that and then to be a tool in the practitioner. And so we wanna just continually chip away at that over time and just elevate the famous practitioner to, you know, be able to do more impactful things that, that help. Now you'll know this from, from being at Rackspace. But when you
know, we had managed hosting and we would say it's better if Rackspace manages your server than if you do or when we had the support for Aws. And you could say it's better that we move you in and migrate and manage you because of all these reasons. But our part of that was convincing them that they had a problem, then our solution would be a solution that would solve that. But I would think in this context, people are looking at that bill and they know they have a problem. You just have to convince them that your solution is one for it. And uh
yes and no, actually, we, we would think you would think and this is kind of what we thought that um it is sort of an easy sale, so to speak. But, you know, there's natural skepticism, but there's also a sense of, well, how do I know that your software is gonna do a better job there? There's also rightfully so too. There's spinoffs practitioners that are very proud of the results that they're delivering. And so the idea that sort of software is gonna sort of age old question software is gonna replace them. Uh
We, we find that's not so much an issue more and more because, um, you know, as I think a I, and the idea of software delivering outcomes is sort of becoming more of an accepted concept. And plus it's just a task that, that people want to sort of automate. So we've seen more and more like when we first started, it was a, it was a novel concept. It's becoming more, more sort of mainstream. But um you know, this is actually a problem because in the cloud, there is no standard KP I to measure the effectiveness of your savings strategy with our savings plan. So
when we started the company, we said we're gonna have to sort of convince customers that, you know, our approach is going to deliver better outcomes, certain net of fees. You know, if you look at the world of investing, they have like RO I and and things where you can basically go look at a, you know, a venture capital company or P company and look at the historical results that they've achieved. Well, there's nothing in cloud that basically is the equivalent of what is the effectiveness that of these instruments that you're using. So one of the things we created
when we started uh prosper S as a metric called the effective savings rate. And that is a KP I, that is an metric that basically tells you when it's all said and done. You have, you know, different discount rates, different products, you have different coverage, different utilization, all of this complexity, you have cyclical workload patterns. So there's, there's many, many things, a lot of go into this. But when it's all said and done, what are you saving off the on demand rate? And that's what effective savings rate is. And so um the the way our process works is we come in and we run what we call savingss analysis for customers and it measures their effective savings rate over the trailing 12 months using whatever they're using today. Like some customers
are using a visibility tool, they write scripts, they might be using another third party tool, whatever it is. Uh what, what is your rate? And then we take our algorithms and we play it on their data forward and we project that of our charge, what the effects rate will be in the future. And you know, our approach, we, we're kind of data type people, we're like data nerds. So our our sales process is extremely data driven. Like it's generally no
lies in the sale in, in a data driven process. So we, you know, we, we present data to customers. Here's what your savings was, here's what we projected it could be. And you know, we, we kind of arm our customers with data to just go make an informed decision. That's really fantastic. And what's really cool is, you know, we've been promoting effective savings rate, you know, it's useful for us uh as a way of sort of being a report card, you know, like how is prosper doing? How do I know? Uh and I always tell our customers as long as you're post prosper ops esr net of our charge is above your pre prosper ops esr you're effectively in the money and our platform is writing our customers a check every month.
Um But it's a useful metric for anybody, even if they're not using Prosper ops, you should know your ESR. And so we've been promoting ESR uh you know, uh our team jokes with me and sent me a Texas license plate that says Doctor Esr and this kind of stuff. So I've been out there sort of promoting it. Well, uh it re invent uh Amazon launched a new um dashboard cost dashboard that includes effective savings rate. Yeah. So you're, you're impacting the world. We're starting
to see it. Uh the Fins Foundation, which is another group that we're greater lead position um is also in the process of adopting effective savings rate. So it's really becoming a standard uh vernacular. Yeah, we, I I met with I DC while we were here and you know, they, they said that this is one of the, you know, five or 10 KPIS that we recommend everybody measure. So if people don't know their esr they can actually go to a, we set up a micro site at Effective savings rate.com.
And there's a completely free Google sheet that they can access. And we explain to you, uh you know how you can calculate your effective savings rate on, on Aws. And of course, if you just want to come to prosper ops and run a savings analysis, we can also do that. And that's a totally free process. How much you've saved people over we do, we've saved. Uh In fact, if you
go to our booth, there's a check, a big check in our booth. It's, it's uh $750 million 750 million dollars is what we've saved customers since our inception. And we now have about $1.5 billion of annual computer usage under management. So incredible and not that many years because you left Rackspace in 2018 to found the company. When did the first customer sign? Let's see, we left, we started the company in the fall of 2018. It took us about
six plus months to get MVP. And then, uh I think we had our first customer. You know, I don't know, a couple months after that, maybe May June of 2019. Well, I'm proud to say we're one of those customers. It's a privilege for us to, to serve rack space.
So we help uh rackspace and rack space. Customers save money on their, on their fantastic. So, uh, what's your booth number while you're here? Oh, gosh, do you know that number? I believe it's 46846468.
Check the directory, check the directory. Now as in all things big conference, there's always a little bl and you're not shy with the bling today. No, not at all. Let's see those shoes through the audience here. Oh, yeah. Yeah, these are, uh, you know, all the things that I could wear. I get so many compliments on
these shoes people. Yeah. Well, they've seen the audience knows the shoes. So I'll tell you the story on those at some point, all the detail. But my son made them for me for Christmas last year. So, so
these, uh, do you know who Nolan Bush is by any chance? I do not. He is the founder of Atari. Oh, ok. And, uh, also Chuck E Cheese randomly. But, um, he actually came and spoke at rackspace headquarters at the castle for a tech block event. And, um, he came and he was wearing these shoes and when I saw them, I thought those are the coolest shoes I've ever seen. I have, I have to get a pair. So, where did you
find it? Just Amazon glowy shoes? I don't remember. Our, our marketing team bought a whole bunch. I don't remember what to search for. Light up shoes. Something's gonna show up. Fantastic, Eric. Thank you so much for coming
and chatting with me today. It's been great. It's good to see you again. Good to see to you, Jeff.
2024-03-04 02:56
