Bringing Retail Trading to 34 million Cryptocurrency Wallets (Cloud Next '19)

So. I'm Luis tough, I had a platform at blockchain. Today. I'm gonna be talking about how. He brought retail, trading to 34 million cryptocurrency, wallets, globally. So. First off who. Is blockchain. They. May have heard of the term Brenda. Durant so. We are blockchain comm, television. Company and actually the. World's leading digital. Asset platform. So. We started out life as a. Blockchain, Explorer, specifically for Bitcoin and. Then we've evolved, and grown many products over the last five years. One. Of our most popular is our cryptocurrency wallet. We've. Also launched a number of our consumer, products including. The ability to trade for two currency, and. We have data. Products market, research, ap, is for. Live streaming data on blockchain analytics, and, we've also launched a number of institutional, products I'll go into more details on some of these later. On the presentation. So, how do we compare it to the market and who is our kind of main competition. So. You may have heard of a company called coinbase. So. The difference between us and them, is that, they. Focus more on the custodial offering, which, means they, are taking custody. Of your digital assets and funds and in, return you can use our platform and services for. Us we've taken a noncustodial. First, approach which. Means we're empowering users take, full control of their private keys and digital, assets at no, point do we take ownership, of those and compromise. Your security, or privacy. So. That means you are no longer reliant, on a third party a centralized. Entity or risk. Of any one person. Losing. Or being compromised with, your digital assets and funds. So. Our key aim really is to empower users and educate them to allow them to take control in a secure and simple, manner. And. Our tagline here you can see is B, or n Bank we, want every user to consider becoming their own bank and. That is surprisingly easier, said. Than done. So. We have three main clients. We. Have an iOS Android. And web. Platform. Here, are some screenshots of, our iOS. Application. And I've. Highlighted some of the key features functionality, that. Our mobile. Applications, offer. As. I mentioned these are owned long custodial, so. They're fat clients, or wallets so. A lot of the heavy lifting is done actually on browser. Side or on the client themselves our. Back-end. Is purely the store some metadata and to. Interact with the. Underlying blockchain, network, so. Here you can see you have an overview of your cryptocurrency. Assets, your, current balances the live market price you. Have the ability to buy and sell cryptocurrencies, will, fit on and off-ramps you. Can exchange your cryptocurrencies for any other asset you, may be interested in and get live pricing, and I'm going to list a bit more detail on the architecture behind that and, it. Finally. Common. Use case you can send and receive cryptocurrency. Funds. Very, easily and very securely, from. Right within our mobile, apps or from. Within the web wallet and as, I say you never have to give up custody, of your assets to do this. So. Where are we today, so. Right now we have 34, million wallets, globally, and growing. We're. Doing over 10 million new wallets a year currently that.

Equates To around a hundred thousand, transactions, per day that, are being processed through our software. Platform, in. Total. To date that's, two hundred billion dollars worth of transactive ahlian has, flown through our software application, and platform and, I. Think for me most excitingly that, actually accounts for 25% of, the global, network traffic on the Bitcoin network. And. Finally. We. Can't have a user base that spans over 140, different countries worldwide. So. First up our Explorer. So. Here on, the slide you can see we. Have this common interface. Into. Many different underlying, block. Chains, notably. Bitcoin, etherium and Bitcoin cash right now there's. Three main section on Explorer the, first is the input to look up your address your, transaction hash or a block. Second. Middle section is key, stats in blockchain, metrics. Such. As the, current difficulty or. The mempool size and finally. We have a real-time view, of incoming transactions, and blocks, that are being mined. So. I want to delve now bit deeper into what, does it take to run a. Web. Platform like that and how, do we achieve in real-time. The. Stream of data from these on the line chains and. More. Importantly how do I do that at scale for, getting around two and half thousand requests per minute consistently, and that kind, of Peaks even, higher with. Kind of significant, price movements. So, how does it work. So. This is obviously simplified. Overview. But. As I mentioned we have three main consumers, of our platform, so. That is our mobile wallets on iOS Android. And the web we. Have our Explorer. Which I just showed on a pure slide and. We also have a number of public api's where any individual. Company can build on top of our platform and. They. Can actually use it for creating. Transactions, sending receiving and also, just consuming. And adjusting, the, blockchain data directly without, having to do the kind of hard work of integrating, with the underlying chains. And. We're in the middle we have this data ingestion service platform, so. Internal, to us we built out a number of services that are, responsible for interacting, with the underlying chains such as Bitcoin etherium cash, stellar. Etc, so, we create a wrapper around these that. Interacts. With the, low-level API is that they provide we. Actually spin up and run all of our own notes in GCP and. We have a number of our servicing, opponents, that then ingest, all of the real-time blocks, and transactions. Extract. The key insights, and metrics, do. Any kind of transformations, we need and persist. That within a relational database. We. Then serve that back up to our clients, on. Demand.

At, Scale. Between. That we do have caching, layers and a number, of other technologies to. Mitigate. Issues. At scale which, I will go into a bit further. But. A key thing here is that a lot. Of the complexity, of, pulling. Out a very simple attribute like your balance is. Abstracted, away from, the user both. In terms of the UI and UX experience, you get from within our clients but also for our unified, API so. You no longer have to spin. Up your own node come, a part of the public network understand. Their api's, which, are often pretty low level interpret. The data which, again is usually. At a, lower, level of abstraction. Now. You just have one API to interact with to. Seamlessly send, receive and process, all. The blockchain data. So. How do we evolve. So. I wanted to set a scene on our scale of our infrastructure, to support our. Platform, both the Explorer could critic, currency wallet and a number of other products, so. You see here we're running over two and a thousand, cores currently, in DCP and around. Nine terabytes, of virtual memory. You. Can see the graph the. Wallet accounts. They've been created over the last five, six years have, grown tremendously especially, since 2017, when, a news kind of hit mainstream, and cryptocurrencies, were suddenly being discussed by every single person on the street I. Think. Astonishingly, and something. That we realized, wasn't going to scale was. That a third of our infrastructure, was, dedicated, to our in-memory. NDB. Cluster and. This. Was running our. Explorer, and our cryptocurrency wallets, to the backbone, for all those and. It. Was done ourselves. By, our engineering team so. We're using. Most. Predominantly compute, engine and running all this stuff ourselves as scale. As. Many of you will know that doing that beyond a point becomes, extremely costly but, from a monetary perspective but. Also from the operational overhead of maintaining. It upgraded, it and dealing. With the fallout of failures. So. We ran over 24, nodes the bacon are in redundancy and make sure we our fault torrent. And. To. Do any kind of scaling we cannot scale linearly, or on demand, we. Had to actually spin up a whole new cluster. Alongside. Our production cluster. Restore. Each, of the underlying chains, for. Example the Bitcoin blockchain from. The. Beginning and, now take around one week all, being well and. Then. Once it caught up we can then flick the switch and move our applications, over to, the new Russian, cluster because. Of the pain involved and the time it took to do this we, were just last year just doubles our. Infrastructure. Overnight. Just, to support our exponential, growth. Not. As quickly realized that this is, not sustainable. So. Beginning of last year we started to consider what, are the other options we have here and how can. We reduce. The operational overhead of running this at scale and. Ensuring. That our, infrastructure. Was going to support us on our, mission. To grow and, we. Define, can, a financial infrastructure of the world. So. In comes Clyde, spanner. So. We looked at a number of different products. And. We. Decided to try a prototype, and PSC, on Clyde spanner, that. Impressive thing here is that, within a few minutes we were able to spin out the production cluster and have. Strong, consistency, globally. Distributed at. Scale. And, all. Of that for a marginal. Increase in cost as to, running just compute engine and management infrastructure ourselves. So. Early signs were promising. We. Then switched and we architect our application. Started. To reinvent. Our schema and it. Started to actually go all-in on Clyde spanner. Initially. For our Bitcoin. Blockchain ingestion, process. One. Of the key things we did early on which, really benefited, engineering. Team, is. That we connected, with the class banner team directly, so. We had a few conversations with, technical PM Deepti and a number of engineers. In her team and they've, already helped us through the initial hurdles of getting set up on, a brand new technology and really, understanding the nuances of, where, is gonna work and some, of the constraints of, running at scale, we. Had to make some. Design. Decisions early on that, we've had that help would. Have been costly to change later I. Think. One of the key features that really. Benefited us during, this iterative development process, was. The fact that we can now the. Full import and export in record, time so.

We Could back up our whole database in just, 30 minutes. As. I mentioned before it took one week to restore that we, can now do it in a batch job overnight. In just 1/2 hours. So. What this meant is that we. Could rapidly, develop, our application and services. Do. A lot of testing and playing, around with different schemas, and architectures, and also. Once we're happy we could then promote that through each of the environments, dev, staging, pre fraud and then finally to production and. Now it's simple quick and efficient. You'll. Notice here as well that, we have. This. Is our production, comparison. To have under. Half, of the original nodes that we needed before now just ten nodes and all. The reasons for that is because you're getting redundancy. High availability strong. Consistency and global. Replication, as part. Of the package. And. If finally we could scale on-demand so, at a touch of a button we, could double our infrastructure. We. Could also set predefined thresholds, and have that elastic. On-demand. Final. Thing is we, actually leveraged. Table, interleaving, which, meant that we could take some of the complex queries, we're running against our NDB cluster previously. And reduce. The time it took to run those from 30 seconds down to around 15 milliseconds. So. The whole migration was, hugely impactful for us and. Our products and, now. We've been running this for, rent six months in production. Overall, on average I just did a comparison before it came and. We. Are saving around 30% month, on month in terms, of the at Dollar Man which. Is pretty impressive when usually. Some. Of the hesitation of I'm using to software. As a service and, Sony's. Cloud services is the expense it's going to just creep up as you grow but, actually we've seen the opposite we've not only reduced, the operational, overhead of managing mr. Skell we've, also made. Our CFO very happy. So. How else is this data. Set used. And. One of our products do we offer. As. I mentioned earlier we, build out a crypto, the crypto exchange, product. Again. Leveraging, a number of client services and, operating. At a significant, scale. So. This allows you to exchange, a, cross, chain. Transactions. And get, live exchange like pricing, right, from a VIN you're, noncustodial wallet. So. Again you never have to send your funds anywhere, and. Give. Up the custody of your assets, to any, one individual, company, or entity. Your. Private keys remain with you on your device or in your browser. Here's. A quick demo of the product so. This is on one of our mobile applications, you'd, simply just type in the amount you wish to sell or buy. You'd. Get a live pricing, right from within the app in.

Real. Time and you. Can exchange between mobile pairs that we offer you. Finally have a summary screen confirm. And that's, it. So. What's really going on and what are some of the complexities, with building out this. Hair. For your degree smooth, user, experience. So. The key challenges. Security. So. For us blockchain, as a company, security. Is our number. One priority in everything we do. You. May have heard of some of the horror. Stories from cryptocurrency. Companies, in a space that have been hacked compromised, or lost. Their users funds we. Don't be a next headline so, everything we do goes for a very rigorous process. Volatility. So, in the marketplace if any of you follow the, cryptocurrency pricing, you'll, see that it's extremely volatile, maybe. Swinging by 20 to 50 percent in a day, so. How do you create a smooth, user experience. When. The price is jumping around all over the place. The. First challenge, public. Network I think it's an important one now. We are relying on infrastructure, that. Is outside of our control, we're. No longer just relying, on our private. Network. Within GCP and the, services that we've built we, now have to broadcast, in centers actions over. Network and many. Nodes globally, distributed. So. Whether can some of the constraints when you do that and. It. Finally high availability I think, it's kind of a given for any challenge. Company now but it's worth noting how do you achieve that at scale and again. What are some of the design decisions an architectural, decision you have to make early, on to, ensure you achieve that. So. The interesting part how does it work. Someone's. Give you a little bit of information. On, what's, under the hood and how does the architecture, actually look, this. Is a slight simplification but, it gives you a very good idea by, some of the core components and interactions. Between those, we. Do leverage a number of cloud services to achieve this and I'll. Mention where they fit into the stack. But. As you saw on the previous slide. First. Of all we start with a client so Android, iOS or web. You. Didn't have a secure connection to, our back-end while our client gateways and. The. User will connect to the swap product. Initiate. A secure, web site connection, and as, we'll start streaming live prices, to, their device, these. Are types in the volume they wish to sell or buy and. We. Then adjust our pricing based upon that input. We. Have a liquidity an execution engine which, is the driving force of all of our trading activity and. That has connections to many different exchanges. Market makers and brokers and also. Our. Internal imagery, that's. Managing risk portfolio management and pricing. Internally. As well we've also built, our own custodial, solution there's a multi-tiered, custody, solution on which. The, hot wallet is at. A table stack so. This is the only part. Of it, or layer that, is programmatically. Accessible and, secure, in our internal network all. Lovers are air-gapped and have, a number of other security. Policies and pastes. So. Once the users happy of the price. The hit exchange what, happens next, so. Our client gateways will go off to our liquidity engine, and say hey I want to execute a trade to sell half, a Bitcoin. Liquid. The engine will then lock in the price and, send, back. How. Much of the, counter currency they wish to receive so. That's for example say I want, to sell, Bitcoin for aetherium in.

Parallel. We initiate, a request to our hot wallet. Which. Is the gateway, to all of our nodes so. These are the blockchain nodes, stellar. Xlm. Bitcoin. Beacon cash etherium etc. Will. Then generate a deposit address which, the hot wallet service will persist. And start, listening on for any incoming deposits, we. Then combine these transactions, send it back to, the client and, the. Client name returns, to the user the, final amount they. Need to send, us to. Receive their, etherium and amp. Reliant, side the. Browser and/or. The, mobile client will, construct that transaction, and sign. It using. Their private keys and then. Broadcast that out to the, underlying respective, network. None. Of that touches our back-end that's well done client, side as mentioned earlier. Now. It, will be sent and broadcast, across the public network and we, wait patiently for, that, to. Hit. A network. Broadcast event and a confirmation event, and the. Hot water is responsible for that. Once. Those have been received off the public network we. Then use Kafka, as our main message bus. So. You have an advantage of an architecture, so, this sends an event over Kafka. One. Of the consumers is our client gateways that picks it up and in purses that. We. Then compare that against our state machine transition. Through the lifecycle say hey deposit, received we. Now need to Pender. Withdrawal. We. Update the user and say hey we receive your deposit we're, now going to initiate the request for you we then make a call back to the, whole wallet and send. The amount we wish to withdraw and the, deposit address that was sent from the client originally. Again that's them broadcast out on the public network. The. Client will then receive the funds in due course usually. Within half, an hour and we. Notify them via push by email SMS, I. Mean. One of the interesting cases that happen recently is. That these public blockchains can, have many different use cases apart. From just transferring. A store, value and one, of them is actually being, a leverage for marketing. So. What that means is actually every. Time anyone sends a transaction, on a Bitcoin cash network and. Someone is listening because you have this public ledger they're listening for this address. Any. New address that appears they didn't send you a little marketing message. And. What does that mean materially, it means that actually. We. As a company on a repositories, receive.

More Funds than the user actually sent and, also. The user receives slightly, more funds than we, send them, they. Also receive two transactions, that are one so. This was kind of a very interesting use. Case where we had to be flexible and defensely. Program for this, in the future. And. Finally. The the note gateway to the bottom are really. The data, ingestion service platform, that I mentioned previously and our, our way. Of wrapping each of the underlying blockchain. Notes. So. One of the other interesting things that we've, managed to do is. Build. Out a hardware, device which. Is a secure. Way, of storing your cryptic currency assets offline. This. Has full instruments. With. All. Of our platform, and software so. You can use our swap product as I show, you before and. Your. Keys never have to leave the physical device. So. This is hugely beneficial, to. All users who are security, conscious and maybe. They'll trust the terminal or workstation, they're using and. We've. Managed to achieve this. Leveraging. Our existing, interface, and software. Some. Of the key benefits of using Google so I mentioned a few of these already but. I wanted to kind of summarize and highlight these, before. Finishing up. So. One of the things that GSP is really allowed us to do is, scale. Up our infrastructure on demand. When. You're dealing with tens of millions of what. Accounts, globally, it's. Important that you're. Not constrained by the, infrastructure that you are using. We've. Had many cases when the prices moved significantly, and suddenly. The, number of users that were onboarding everyday has. Gone up five or ten times. And. We've. Had to actually scale poor infrastructure significantly, during, those periods, I mean. In the peaks of the, last. Few months we've. Seen over a hundred thousand, new wallets, being created every single day and up. To 101 million in a given week. So. The significant strains on our infrastructure that, we. Have to account for, and. One of the things we do do as I mentioned previously is, adding. A number of caching layers. Take. A lot of advantage of CDN and do. A lot of, load. Balancing across many instances for a most part all of our infrastructure is stateless so, means we can horizontally scale, it on demand. Also. On the previous. Architecture, we're, leveraging, cloud. Services where possible it, means that our team. Can focus, and hone in on some, of our very. Specific domain complexities. And challenges and, don't. Worry about running, our infrastructure scale and solving, the problems that Google have been doing for many years. Today. It spend less time energy and effort, Imaginarium. Structure and more, time focusing. On building, what. We believe to be the future of Finance. As. I. Mentioned we have high availability so. We're leveraging cloud sequel, that we're, leveraging spanner. We. Use a number of other services, quite, broadly across our architecture, as. You. Can imagine we have a lot of American micro services within our architecture, so. Being able to just quickly spin up a. Highly. Available redundant, and scalable, database. At. A click of a button it is very powerful and, we're using both Postgres, and MySQL, engines. For that. Our. Third point is that it's. Engineering friendly I. Think. All of our engineers whatever, their skill, set or focus, enjoy. The GCP experience, whether. That's because of using the CLI and. Programmatically. Writing scripts around that or, using. The interface itself to. Dive into problems, and debugging, issues. And. Finally the fourth one which I mentioned earlier of our spanner story is the. Technical support I think. A lot of people didn't, realize the amount of support that Google offer you. As a company. We. Can tap into a wealth of knowledge and domain, expertise, through. Our camera managers and customer engineers and we, do that pretty, regularly. We've. Had many conversations comparing. Google, products to other open source technologies, and the, pros and cons of using. Them. So. Another story. That happened. Last year we. Were just trying to decide on the, correct message bus architecture, for. This training product and for many others, one. Of our key requirements, was sequencing. So. We, then spoke with the. Technical PM's from the pops, up team and had, a real candid conversation, about what. Were the pros. And cons of using pub/sub, above. Kafka. And, a. Net result of that conversation was, that we used Kafka and that, was their recommendation I, mean.

It's Quite refreshing when you can actually connect, with the engineers at, a cloud company and they recommend a product which, they do not support or sell themselves. You.

2019-04-17 15:40

Other news