Микросервисная архитектура, как в BigTech (микросервисы vs монолит)

today we will talk about how writing microservices in bigtech companies This is the topic of our lesson today. first, let me introduce myself as someone who doesn't know me knows My name is Oleg I work in Avito in the Back of As Service platform we do all sorts of cool things to make life easier for product developers all sorts of things these authentication routing policies access, etc., and before that I worked in Ozone for a couple of years there in three teams visited logistics in Ozone Moments in marketplace also taught there school 256 and here more is included on the slide fun fact last fall I became the presenter of the conference which Yandex spent 2 days it was interesting exciting 8 hours of live broadcast buzz What will we talk about today first? microservices don't exist somehow isolated from everything else you need to see what it is first monolith and we'll be a little contrast microservice monolith Let's look at the pros and cons first Monolith when it should be used when no need to talk about why later most companies refuse Monolith pros consider microservices What is this anyway? Well, let's see how write microservices in bit-tech companies what difficulties they face and how they they decide this way And about questions in the chat, I periodically I will be interrupted along the way stories and rest But what if I'll definitely miss something at the end We We'll talk to you some more, let's go and we'll go to general storytelling next We have the code is actually my code if anyone doesn't knows his name is Kotkin and he One day thought about it and said, well, airplanes are cool cool thing, transport is necessary organize your search service cheap flights he obviously was inspired by something Well, apparently I flew with by someone in the same salon but he says In short, I will do everything to make the best website in kind of obvious for cats Yes cats must also fly otherwise you know how it happens Well, it’s terrible while the cat was transported to you'll spend all your money on a plane, and he says we'll do everything cheap air tickets and this was the beginning of the journey He so well, that means he's got ready, he says I need a website, so why not get a website? air CS search for cheap tickets obviously I bought a server I bought more a server for a database and such talk Well, everything is basically fine on one server everyone deployed the base and went to in general, well, services It seems so they are still built in their own time by a master there was a code and ours So everything is according to the canon and then In general, he bought server Yes, he bought the server there and in general picture Well, a diagram of all this the product looks like this some browser clients there are mobile phones and they go through DNS to ours web services for such a solution it is essentially classic monolithic application did firstly the ease of development Well like there’s no need to worry about it, I just did it there is some code in one repository sketched it all out and deployed it as one service E is not necessary there are a billion of them Settings are easy to make in principle radical changes because well that everything is at hand all in one repositories, again, everything is connected If you want to change something, take everything rewritten, testing again is not difficult and that I covered all the code nearby with units there is no network interaction deploy too, I already mentioned that on Easy And it’s easier to scale than in a store go Well, he bought a few stuck the RAM sticks into the server and ready But all these points they work in in this key only up to a couple of times A and here's the thing that means the code is running his own project Well, in general, he has it here kittens ask, can you tell me where? By the way, here are the cheapest air tickets write to the chat where they are cheap, damn it the cheapest air tickets I'm at the beginning by the way mentioned Well, in general, everything was blocked by traffic A he says it's all great service begins to gain popularity and one evening he lies on the bed he's so high, he says business is booming let's go cool orders pond service is growing but suddenly he realizes that the clients it became too much and the service stopped cope and that was the beginning microservice era, he began to think Well, what to do, what the hell is the problem? It’s easy to decide that the servers are melting there literally from the influx of these kittens generated by a neural network something like this burned because they were not there end generate it just for me here to he said that he saw everything a and So that’s where they surfaced cons of Monolith and development begins slow down as the project develops because the more code there is in one the more difficult it is for an idea to cope with repositories to somehow index it there It’s clear that modern ones are there too trying to adapt to it and stuff this but it wasn’t always like this Yes and again not in all cases, they are normal there can process all this I remember in my own way time some Monolith opened in the IDE Well basically my IDE died after that it was sad but those were the positives then the big Time to Market that is delivery time for features before sale Because if the system is gigantic and we are in it we make some change and it turns out we need to check the whole system again A This even trivial tests take a long time to run it might work and then deploy everything long because everything weighs who knows how long are there any images? virtual girls will gather and roll out. Horror scaling is also difficult because A monolith with some consists of modules and not every module equivalent in its load Well, that is relatively speaking, some kind of showcase obviously receives more traffic than some payment module because look much more often than buy I would like to isolate them scale because otherwise we we’re just getting started with these dies throw The reliability of the application suffers because they are monoliths, well, as if fell then fell everything fell all system I don’t know the error happened again in the payment module and the whole site crashed Well, that’s a perspective. Well, that’s it here again they start playing aside the fact that if monoliths are most likely in at some point they wrote him further it was difficult to saw and gradually the stack on which it was written begins become obsolete because the larger it is, the it's harder to change anything because everything tied to each other, that is, it’s difficult take some Monolith on I don’t know on PHP and such But let's take a piece under the edit Nago we will rewrite it like this Well this the piece is monolithic and was already there then to saw it somehow you have to be cunning some building blocks here's a plus everything else We are good at scaling that something just came to him clients and this logic What kind of now we are operatives throw it on maybe a terabyte of RAM? Well, in short there is a limit and yes And plus to everything every step in scaling this kind it is significantly more expensive than the previous A and m I don't know if I can find a few cheaper, well simple pieces of hardware and how to use them somehow one healthy cool one Well, in general, he sitting there, I realized all these disadvantages And then it means the world of microservices some of these dockers are starting to crash Although he seemed to be his industry connected What does the port have to do with something? heard all sorts of things What’s so incomprehensible about this? and in general, they also showed him what It turns out that all your functionality is possible divide into component modules Well, in this case here just as an example presented what he divided I don’t know the service issued the order as a separate one, that is work with orders payment service work with payment and nofi service of some kind and so they seem to exist in isolation they communicate from each other over the network there somehow, well, it doesn’t matter whether it’s brokers messages or some kind of synchronous data protocols but the trick is that what can we do? Regardless of these modules develop, deploy, test and the cool thing is that there will be teams too isolated in their work, that is, we have a team that deals with ordering service it is only for them and is engaged, of course, there are some related integrations but at the same time they are all concentrated on this product A I when they break something in their service it does not directly affect others services A And this is an undeniable plus of course and so It’s easier for the work team to divide zones responsibilities are more clear And due to this and the speed of development increases because and test that it also doesn’t take a million years to check everything less functionality trite means what are the advantages he understood microservices from all this descriptions of the advantages are as follows, as if I had already said yes Let's repeat the difficult ones one more time applications are simpler, there's just less code there are going to be some containers there then the services will be available faster small and easier to develop too because well make some changes in the code that is there intertwined and deployment services Independently of each other friend's services scale independently from each other is also an important aspect because as I already said Well for example for the same service as for the storefront answers it can be filled there I don’t know so that it is allocated there for him RAM is conditionally 10 GB and under the server orders 1 GB because he doesn't have any more it’s necessary, well, that’s cool, well, autonomy teams I already mentioned Yes, they are big opportunities to experiment with technology stack because if we for example, we want an ordering service rewrite it in less volume anyway than rewriting the entire system and plus We can do everything with a stack experiment since they are online interact with each other and in general, I don’t care at least there’s one the component on the pluses is written differently on they communicate over the network no matter It’s clear that the technology zoo Well, that's the story. But

there is some variability in this regard 1D there are three languages, that’s localization problems are of course better. That is, you you're not looking for a problem in this giant monolith Where did it happen here And in small service and it just speeds up time to find the problem this one but the cat seemed to be inspired with all this history but comes its own development department and sees the following picture monitor broken with its ingenious design and developers are fucking with each other a friend, in general, one tells how says How crazy is it to test it’s not clear when your system is there from with a bunch of microservices they are somehow distributed over the network there interact there too. It’s not clear what’s wrong prodom Who is rolling what where because well the release cycle has shortened and they are all quick cut in pro maret reduced they can be there every day I don't know fuck five times in a row Well, in short, a cat looks at this whole story. There’s also one cat is actually dead and speaks out of fear I say I don’t know the payment went through the order no, that is, in the same database one service has this data, another no, it's all go away what the hell is going away like this the whole thing is to coordinate, well, in short developers who worked for him mono application encountered the reality of microservices and it’s not like that either everything is fabulous, like it was in the picture with pros because there are also cons And here’s an important point, by the way Picture from Martin Fowler's article who wrote a lot in his time writes and wrote and writes about microservices and he just notes that But namely the choice of what to use for your system monolithic architecture or microservice also largely depends on from the complexity of the system itself Because if your system is not complicated Well, I’m not I know some kind of ped project you are doing there Well, you're sick of this one infrastructure for microservices It’s really easiest for you to build just one there is some kind of zahir repository Set up some kind of deployment for the UB action everything has rolled out for you, but in some way moment But this complexity is growing and monolithic system and which is green marked Yes the performance of this the system begins to fall according to the indicated the reasons above are simply not taken out there horizontal scaling in prot longer catt and so on and in some microservices are starting at this point plan to save Well, here you also need to understand that everything there is not free everything with reservations and so on That is, it is impossible so to say that definitely yes microservices always rule when available complex system, that's where it comes from in general highlighted the disadvantages the following our code says firstly, it is difficult to divide the system adequate because there are a number of problems with this And firstly it could be like this the situation will turn out that you will not succeed system of microservices A distributed monolith it happens like this the situation when you have microservices they for each other too much gears in short the same Monolith just simply distributed over the network Secondly there are more Well, there are many different ones like this technologies, how to divide them correctly there I don’t know about domain zones there based on business features and so on Here and to this cash register basically, if you ask like this some questions fit well there all these approaches are clean architecture doi dri design and other things which precisely regulate history what you need to try microservices highlight I don’t know not by size what like oh microservice there should be 500 there no more lines, otherwise this is no longer microservice Are there such strange ones? take or something like that should develop by two people or whatever microservice must be able to develop koto team to feed which I need one pizza No, it’s better to go from business features, that is, uh look some domains, that is, if you have there is some need to process users Well, here you have a service for users answers And if you have any history of orders from you they answer and it doesn’t seem necessary that this service will be small. Just you

clearly divide the roles in this system of yours who is responsible for what roughly speaking distributed and here just some kind of universal Light it simply cannot be because everything depends on the specifics of the business company specifics and so on therefore here you need to cut before all these really sit and think through options architecture and moreover most likely something is not taken into account and in the future it’s just will have to be redone Well, that is, conditionally There is such a situation that, well, you are like this Let's make two services here. Then these done and oh but in general Well, you can do it in one just collapse something was unclear first some new circumstances have arisen, are you doing well, or another situation you are doing some functionality within one service but then this functionality starts to build up and you realize that from this functionality Quite two new dependent services are highlighted also an option and just the same development in style There are purely architectures and other things She allows leave you the potential or something for maneuver Well, it’s just trivial that it will be easier pull this logic inside the application Yes, for several services then again look at the disadvantages difficulties of distributed systems And their plenty firstly How to develop like this testing in general Well, in the sense of debugging at least locally if you have unal for two service is tied, you need to somehow have two services raise it on your computer so they are there We communicated over the network It’s all real like this somehow functioned test too It’s not particularly clear, you’re like, how will you be? wet or something These are the dependencies network or somehow Negotiate with the neighboring team to we got some fake data provided deployment too, well, it requires certain settings for all this environment so that a bunch of services are deployed at the same time somewhere there are all these I knocked the containers into the rest and put them there I don’t know they flew into the cube and blah blah blah in short again requires separate specialists PS specialists who understand everything correctly can do it and these are grandmas accordingly Well, we are already starting understand that if the project is small, what the heck you all this pandemonium with microservices and you are stupid to spend money a lot uh next means uh what I have written Deployment Deployment function which cover several services difficult and requires clear approvals between owners Well, yes, it's a classic history Yes, when some service changes contract if it's there I don't know coordinated all this with the neighboring team who comes to see them there, everything is renamed it breaks but this is again about the question about the use of Best Practices of all kinds developing that we don’t break contracts, we create new ones then if you have something changed radically in the second version and so on and again it’s not easy to decide when is it time to switch to microservices? Well, like, where is this one, there? this clear meaning Yes, like this we wrote so many lines of code in our it's time for the monolith Well, that's it. Again, on some Metrics is being built if you have severe subsidence something you see that is significant there the pace of development has slowed down Monolith's stability collapsed Well, here it is all sorts of things can be reversed attention and new services are obvious reduce system availability why is this happens because they are microservices communicate over the network and where there is a network there there is a possibility that something somewhere Lant, well, there’s something about a timeout somewhere will fall off somewhere accessibility will fail and for example, if you have ae instead one Monolith which the user answers Yes A and he is like that for me stability I don’t know there 990 99% E 99.9 yes about Oh well, not in these ones relative numbers 0.99 like this

let's call it stability if you saw for two services that are mutually exclusive walk then you get it in one network interaction 0.99 and in the second 0.99 and for overall stability Calculate all systems need to multiply these numbers and the number will be less than 099 here Well, as I already said, infrastructure we need a much more complex one. Well, what the hell there will be fun with transactions, that's just it back to the issue of data consistency When we have change the value of some kind I don’t know the same order affects data modification in different services they all have their own database data So somehow you need all this do it carefully so that plus or minus everything was agreed upon in a nutshell minuses in general and pluses Well, that’s it contradictory decision Well, he called their developers and they are like this So what is it now should I read a little boar? probably Let's Like Yes it says or a plus sign in the chat who read it hog hog - in my opinion this is ours everything is in these distributed systems highly loaded another says And me Are they going to fire me or am I monolithic? programmer and here are some microservices I don't understand anything about it at all I'll drop it, he says, well, this one is simple states the base says it seems it will be it hurts true Well, in shock he simply says that we will do something. Well, what can I say?

we have a new chapter in our epochal microservices saga The Hidden Threat Well then The hidden threat will help us outer space the ship is obvious Yes, which is all of us will save ONT same piled up In short, this one was just ll kek uh In general, he sits like this and says, uh, well in general, the first problem is exactly this we are discussing, what can we do? be too confusing e system and that's it this will turn into a microservice Monolith distributed Monolith which I am mentioned this is where we need to start get rid of Yes in those ways that voiced Well, like plan adequately system design constantly m you know just like saying here there is this one here Continuous integration Continuous Delivery type continuously We are all there change the data in prot skinny I would be here He also said: How's the design going? in English, you know how to speak Well in short, you have continuous design there must be more because it is always necessary mmm, read feedback from some metrics have you built an adequate system? maybe it's better to think about decision And let's maybe something here interrupt Let's do something here let's redo it Well, it's also fairly understandable case here's the next one moment this is our decomposition as well, these are related things with the fact that I said how right this is distribute once again I remind you that microservices have this principle that each microservice has its own base data and only he can go there the neighboring service cannot access this database data to walk he can walk in it only through the API, that's it decomposition is real, well, to some extent probably even art, because that t and experience begins to influence and all that and experience manifests itself as, well, like you worked, tried it, ate shit and Damn this needs to be done differently tried it differently and you got some experience is already beginning to accumulate in what situations is it best to do here? books of course too. It’s great to read this helps expand erudition That's all anyway, experience will not replace it because books using a specific example it's all showing you're dating a little with another and such Oops let's let's try this ANO herak not SBO Maybe it's a billion factors which converged in the following situation in general everything will be different like this this is the situation so let's get on with it Let's take a short break and chat look through 132 people have already written to you It's amazing how joyful it is now we read here they ask where to donate so obviously you go to the course website buy the course here before on boost yes yes well I'm here by the way I plan to start creating content on boost uh interesting let's see And so so so so scale from recoding bits e to si and DSL to Sla where go what I don’t know where go go is in the lead obviously in the top, but will there be a course on concurrency in Go Well, I don’t know, maybe you can think about it, but here we have The monolith is at war Well, okay, but the topics are just honey Of course there will be a recording blah blah blah like they go Oh by the way no forget to like Stream Who hasn't done this yet? made it interesting it's important it useful advertising in the first 5 minutes advertisement What is this? like - this is the development of a cat AV completely Prim order with no photo of a cat with a cup coffee with soy milk the story is incomplete not bad at first job on pluses build project about 40 minutes and by the way I have I had the same situation with my computer came with a laptop that barely lived there inclined the project and he's done and he's there Bill Bill Bill Bill I was still kind of stupid Do you know if I make any mistakes there? like from a series somehow there is no variable so he left and well, some idiotic things did and to understand that I'm a fool it would take half an hour, I'm like, Lord, well then they gave me a normal laptop and docker is short hat here Shaki tucked up I'll fill it up the day after tomorrow and continue Delivery pizza delivery Not bad not bad there will be another Kotkin dely some Koto I'm tired of being a jackass. What is it? leim Okay, so here you already wrote it Yes ok The more microservice is more complex the cat has a snack to eat all the wires sharish normal topic So what do we have next? what about other options distributed systems with some have you come across any options? If so, can you do it? share my experience in a moment ri V something like a prototype of microservices are that we somehow share it's sitting on the domain, it's sitting crossed he seems to be very good with them here ate so much here Coziness that the cats are here at mom's asked to ask so Okay so so so genius on who made e sticks sticks B I mean, it needs to be put into a microservice Deploying on a separate server is simple base and then people like that come Lord why do we have 283 microservices What did you do I did cool So we will use common ones contracts from food buff where we will use Well we can use there are such there is such an option transaction transactions crazy fun at the front there is a reg are these theses related or how I didn’t understand Well transactions yes definitely ve Is it possible consider that fnn and backend are two microservice you know what they say in smart books these are two levels Well, yes So we are now Front in backed CD Well Why not Well if it has its own specifics we're all here now too we do interesting things with the front there In short, we decided to remember here zero purely on a bare script of some kind template geri pages all sorts of stuff there registration other Well, they are light there It’s convenient for us to just fill them there content from Well by which he knows And that's it And like Fuck I'm on your channel for someone's hog Mom advised Well, that's cool and a hog okay hog, take it off little man But here's just a book with like a wild boar if you see a bottle of wine with wild boar is also quite good choice at yes, by by Go is better to start with. So what?

it's about Design Well, if in this context Yes, how is it standing there? dadada hog base they tell us Lord I lost my comments again. Let's I’ll read something else and continue microservices are top but annoying to roll 10,000 releases with each microservices yes this is there is such a topic here Novi By the way, there is this topic which is auto PR creates And you're like you All that's left is smr live essentially with some branch next door or separately his Oleg hi I really want to see Saturday and the whiskey won't let me do it it's good that there will be a recording Well, there you are hold on, the main thing is that they don’t take over your patronage is this whiskey drinker. What if one connects to the other by this microservices Well, basically, whatever it is this is if this crack it looks like a duck if it sounds like it duck It's probably a duck it's like that bad microservices Yes, this is a maxservice, they tell us, and also a noser there is such a fashionable theme of the 2000s, it’s crazy How do you maintain consistency? data vdshka is essentially a problem Are there no microservices? That's what we're talking about let's talk let's go later further down in Amazon there is also an MRF generator they tell us when I left there they just washed it down, I remember them teli koche and we flew into this engine bridge 40 notifications like this PSP and we are like that Lord, well, like, do you have some platform dependencies Here they are such Updated and so that this does not happen what do you know, someone is seriously behind there they are in every service of some kind passes them ace updates automatically in the project and creates with this change for you too All that remains is smr Essentially, well, let's go. What other ones?

We still have problems and the problem is the cat sitting in anticipation Well, we need to set up FL C in the World Cup soto The point is that we have when We are developing an application and we want it this way Here release the code passes from the developer's computer to automation of the working server by to the maximum, therefore all sorts of things are under control version, that is, there we pushed it in git tla some kind of no matter Yes he is everything was saved automatically then this launched the scripts that built it all I don’t know, the docker image is being assembled there linter is passed Unit tests are launched then from the rest he shot down this one piece of how is this image deployed in test some environment there either testers Well, when if then development begins Yes, they are there with their hands first they can poke then they have to Another good way to cover the autotest Well so that the already existing functionality Every time with every deploy that he it didn’t fall apart but just automatically are also checked with a type of code What for? Python or Go, whatever you want in Java depending on what language, but well, when, like we're all convinced that it's good in production in production there are also different strategies ploy there is a green-blue ploy when we there we are trying to replace the existing one version which is in the product of the new version canary deployment when we are by percentage roll out like roll out a new one functionality for 5% for 10 for 20 look at Metric Everything is fine no If everything is fine that's ok fuck and then we get an incident Well, in general, all this here too building this whole pipeline developing a delivery pipeline even I would so he said it also falls on the shoulders individual specialists can also call it a common word dogs Yes, this is it something about infrastructure Dudes it turns out you don’t need to hire them pay grandma and all this, all this organize deployment And the code is like Okay, to hell with it deployment Let's organize what else you have prepared for us in your microservices of these there is such a thing as ses Discovery feature it is that look at us We they said that microservices are cool there everything scales Yes and that’s why most often in Production there is more than one a copy of the application and several copies are well the logic is very simple, they can be even on different servers Well if we distribute the load across several copies of the application collectively brought more traffic But there is a nuance which is connected with the fact that, for example, U we have some kind of service with orders Yes and he is processing something there and he wants when he processed the order Send message contact service notifications what a dude Send notification to the user via SMS or more somewhere and he's like Yeah I'm like where from I find out where this notification service is then we actually need to contact know the guy and the port, but considering that microservices they are constantly re deploying that is, some new versions are being rolled out then obviously uh pods well if we are in these in cuber terms we say they are shorter new ones die and IPs are created constantly changing, but how in such an environment like keep track of this whole thing and here Sis Discovery comes to the rescue ses Discovery is essentially a mechanism search for this address Where contact and this ses Discovery he There are two types: there is server and client client is that in general we have some kind of remove some storage that's there some kind of database under it that stores Well Relatively speaking, the map is like this Yes for simplifying the name of the service and its list food that is here and when Let's say some client wants to go to neighboring service He goes to this one first Reg is like what is there at the addresses E he to him In short, he sends this list and this one ok and continue on there on the client itself load balancing logic implemented there I don't know the round by Robin or any other options and he balances and the advantages are shorter than this the approach is that you have a network hop less Well, I mean you have one thing total network interaction you to contact us for some service you just need to go to the restaurant and that’s it, but the disadvantages are that you get you need to implement the logic on the client load balancing, which in general is not It’s always convenient to eat as opposed to this server balancing you from the client The client doesn't know shit at all. He's stupid. and he just goes by name where he goes and so and it doesn’t go directly to the restaurant somewhere else he comes here is a conditional balancer Yes some there is no point where some the component is coming and this component is already he's already on his way to the restaurant something is there finds - and then the route request Well then IP address that was found a plus this approach means that the client stupid and no need to worry about it at all once there is this component which balance the load, they wrote everything cool But the downsides are that there are extra network hop you go first to this balancer and then also to stri And if there is an additional network hop that's a little bit more delay more stability risks sags and that’s it this means you need to start from the text Now, if you give, for example, these public cases that are described the link will be nahar in Ozone in Ozone client balancing is used there written just the same utility that it's called Arden, it's just ses Discovery provides microservice all this Wednesday let's go Next he's like Okay Discovery normal topic explained protected but then another one comes running to him engineer and says everything is lost brother Nam need new ones functions there is such a pattern in distributed systems, its essence is that we have a number of needs first need For example we have some client Well in this case the front there's some kind of thing there, or it doesn't matter or just a third-party client there is no mobile phone mobile phone It doesn’t matter who wants ours send the system somehow given and not always far from it always the data this one needs to the client coincides with the distribution microservice data, that is, in there is a microservice, relatively speaking. contracts on some handles they let's say they contain fields Well, I don't know For example, there is some kind of order and you want to display a list on the front orders Yes, but at the same time you would also wanted to add it at the front information about whether there was any information about it notification sent and received then you will need to go from the front to one service for the order was stolen then for notification their status understand and so further Here iway, in principle, he can accumulate some data from different services, that is, relatively speaking, it some API provides to the front says Okay brother I leave it to you API that you want front but at the same time I will hide behind the implementation of this API what I climb into different microservices and collect data is a plus to everything. What other things Can I decide since this is such a gateway through which all traffic rushes into our we can centralize the system solve problems that need to be solved in in general, throughout the entire system starting there from authentication, uh, make sure there's no I know the person is authorized authenticator, that is, like this there is some frequent Request that Yes to it flies to you and instead of every time you can just get into your microservices with Ksha throws in this regard by the way interesting, I can also recommend the article in my Telegram channel a couple of days ago posted again with me I know azona, she's just like that I wrote how they were there in Ozone all this they coat the façade with porridge ozone because the traffic there is crazy the limiter can also be implemented on AP G Zaza because well, maybe you have ddosyat Yes and you want to chop off in some moment above some threshold otherwise everything will melt there the monitoring itself can be viewed in In general, this pattern is useful and it is often used also used in conjunction with rever Proxy and but just reverse Proxy it's a bit like say something AP gway over a lot of uh reverse Proxy that is reverse Proxy provides everything the same as AP gway just AP gway more extra provides all sorts of functions Well I I’ll send you the links to read. In general, if

it will be interesting to figure it out you will see in more detail And so I gway he so Okay gway good topic At the same time The client will not know where it is inside. We it's safer to get your data, but but of course, yes, probably someone has it you might think that this is a bottle neck type of narrow neck of our system e and v overall a single point of failure But here it’s already the question of how we will fault tolerance AP gtv m create maybe some kind of ballast AP gway hang Well, that is another copy of AP gove that just sits without traffic But if this crap isn't there, all the traffic is on it will translate uh maybe just eat right away several copies and balance between them traffic in general there are many options from Again, everything will depend on the context whichever is more convenient for you And the cat decided uh after all this news go to monitoring department and ask the guys Well what do you need and what do we have needs in our system in monitoring in general and they stuffed him there a bunch of demands at once says first It would be nice to collect metrics because what are the metrics firstly they can show the technical system state How is the CPU eating like RAM is consumed as network RAM load and blah blah blah because if I don’t know, some kind of routine flowed there Fuck us, you got it there memory will begin to survive in the second place without there are owl metrics, that is, well, it’s necessary track how many orders there were how many people visited the site? conversion and blah blah blah analytics here joyful ones will come running, we need data more data so that we are Nero and btw these cat air tickets are for you in the morning by force And then of course we are interested in dashboards all sorts of fun and of course that’s all it was more visual because the metrics Well, numeric Okay, but it’s not always convenient you can see it’s better to draw a plus to everything The metric also includes the alert history because if you have understanding it is clear that, for example, your service is not owes 50% of requests from five hundred Not bad then hang him up so that he caught fire and somewhere there I don’t know further by your logic I would call for example wrote to someone by email, in a chat somewhere this information fell, what a dude seems like some kind of system hat logs Of course they are needed because they are the same metrics detect any actions then you more likely to understand that in generally occurs Where exactly is the problem happened needed in raden systems I think they are it's very hard if anyone doesn't know this Well, you can say yes, bread crumbs according to which reserves the request Well like if you have a request, but with client yes oh some hits microservice and that microservice goes to the second one in the third one in the fourth one fifth and so on and so forth distributed query looks like this for 10 microservices that trace is in the end forms a picture of the way along to whom the request was sent. How long has he been in each microservice hung out and this way you can do it faster too localize the problem because Completely probably what are these 10 services? answered by 10 different teams Well done understand who to insert stars and where Is there anything to fix? Well, here I am on alert mentioned In short, all this is an observation observed by our system he says Okay everything is already cover everything with all the patterns How much maybe the cats are already here developers yes so and so and so but here it is there seems to be some problem with the database we didn’t decide to eat there certain points that this is the data they're going to hell, how can they even work? so with databases there says we are in monolith that we had one database We're like, we're going to fuck it up and there's no trouble at all knew and there are a bunch of them and he says Well here you know, in general the situation is as follows firstly, I would also like to say that their problem is not only that data is dispersed and also and e in fault tolerance is of course written here about consistency But I'm still just I'll add a small top of the list It would be nice to have backup copies of the database keep data This of course also applies to monolithic architecture but I kind of expanding your horizons If you suddenly don’t have you heard about replication when? we just keep copies of our databases data on different servers A on in case if someone there, I don’t know, will fall off or we We’ll lose the data and that’s just the replicas there are different there is a synchronous replica there is an asynchronous replica and the difference is that we always write Well if we take this system classic with one master and a bunch replicas then we always write writing all sorts of loads, these updates are lit we fuck the master and the reader We distribute the load across replicas Well to again reduce the load and usually These are the web applications Yes, this reading load is significantly more, they just update the page there That's when we write the load on master master must distribute changes in the database to others replicas and if the replica is synchronous then he fell asleep first and only after that he writes to The master tells the client that all the data signed up If you can't sign them up nothing is added to the synchronous replica will be recorded Well, the error is just displayed to the top and in asynchronous replication it’s different the story he writes in the Master backfill But there is no expectation at all answer and immediately answers And that’s how it is once it turns out this is a story about two chair you've probably heard about the cap theorem cap - it's like acronym kon avab Partition tolerance The bottom line is this: Like if you have Rakhila Internet Well, this is what I mean network communication between replicas then you have two chairs, the first chair is you in short koche is preserved data consistency but availability starts millet Why Because you don't let me write nothing new to the Master because you have a synchronous replica not available but also data consistency you don’t get lost in the Master, you don’t write in don't write that one because it's not available the second chair is accessibility systems you are like this I am like this data sent to the asynchronous replica what they didn’t make it there. Well, sorry, but you the error was not returned from Master No consistency may diverge and that's it usually in such systems when in short super critically this is the whole story with permanent, so instant consistency uses the term eventual consistency that is, like the final consistency is just the same she says that we are right in at the moment something may move apart because there's something up there the asynchronous replica did not arrive and so further but we are like that Well okay Like if the person placed an order, well, you think from him there 1020 seconds will be shown the status will appear incorrect later normal that is Event consistency says that ultimately the data is consistent just maybe some delay everything Further about more consistency data and these distributed each microservice has its own transaction database and we need to enter changes to several databases that refers to different mixes and that's all it should be transactional there are patterns there are two-phase commit there are Sagas that are in basically on the screen given they solve the problem plus or minus the same thing but in different ways Let's Let's talk about Saga. Saga is shorter chain of local migrations Well there is migrations Lord transactions You you can imagine how it is there all kinds of transactions in microservices are confused Well, not in microservices, just in code when interacting with the database you're messing up the transaction Yes, that's it in short You have this logic in every service code that, well, rolls out migration the code that m rolls back the migration and It turns out the next story is what you have the transaction must affect for example three service you in the first shorter it rolls If everything is ok, you just go somewhere There the message queue is dumping a message that I rolled everything next let's read the second service listens to some line like this it's time to roll himself if it's shorter for him failed to download He sends message This is a fail Well, also this one message queue And the first service is like this oops it didn’t work out, that means he made those changes takes it rolls back and everything is as if consistency appeared But at some point moment she may well not be because in the first one already downloaded the data and in the second one is not there yet, but if everything is ok then just the second one will also send a message to Reti roll and here they are at slightly different times but eventually in the end these changes rolled in and this There are simply two types of saga: either called choreography or orchestration choreography is when this is what I described there is no external observer they just exchange with each other messages and everything And in case of orchestration there's just one more extra one here this orchestrator which tells each of the services when his transactions work well then there's like he's taking on Bert's role here this component that decides what there will be order on the migration rink But usually, of course, with sagas, well, they cool powerful allow in general achieve the desired result But this extremely difficult, complex systems of course increases with the use of these things So let's say a little translation read and watch and Oleg’s chat is only for interesting ones messages and not for everything, well, you know sometimes it crashes if something important I duplicate I duplicate And I said duplicate duplicate God, we need to duplicate the database Yes yes Yes, duplicate this is a base and a cringe too you need to duplicate it like this Canary ploy to repair uh Not bad not bad also a Maldivian ploy Yes, and a canary for Kafka la Vita made Oh I don't know I don't doing this And so so so admin you're drunk Go sleep it off he he he he the tea drunkard just like Basta so the balancer is k or Gosh so they are different, like balance then you can do it with anything it’s already from the specifics depend on who uses what So So why is the admin drinking, he’s over there channel mo advertises this is exactly what it is need to [music] so Hello everyone, hello to you also master It would be a good idea to warm up the planner Well, that’s already Yes, that’s already the nuances gone like there to work with the base It's me I agree, something has become difficult. Well,

I also said that it’s easy already I'm also in shock He thought so now I'm like trendy guy microservices got knocked up I was born and everything is ready, but in reality everything turned out it's much more difficult, you know, I'm here promotional post for this open lesson one person did this too, in short the feeling that something is hard on him offended in these microservices he is so wrote something from the series you are still young finally someday you will understand that microservices are [ __ ] nothing at all they decide it's just terrible. It's a stench then it turned out in short Well, I’m polite I wrote some shit from the context of the sun depends on the matter, so he says yes, I just now come to the company here these are shit code two senors got fucked the problem is with people, not with microservices so the entry will be on this channel everything is so cool happening where is that guy with whiskey uh guy with whiskey Well apparently he is happy we are scratching our heads here Yes the best from the worlds we are here with some sagas we break it chills calmly will you tell me something about the sessions? I would be glad to hear about the sessions. Well, you can at the end when the question is answered let's go Well, answers to questions like this wasn't planned Let's go the cat has already dealt with the sagas just in shock, look at the cat already he's just crazy about what's happening, he says what's wrong with these microservices? they say Well, here in general this is what it is there is a history there is such a thing as a sesh call In general, this is technology which provides more flexible interactions between microservices and other things, for example, well, in short, classic This is the diagram that is built from two dat components and that is, some kind of know how to control component and component of this data and usually the EU in we speak in terms, that is, according to there there is a container with the application one who has heard these terms Saika called auxiliary in short container and it actually becomes well, it takes on all the traffic which goes into the container something with it can be done later in a container send In the container Our logic is there works out the answer and it comes again can do something with this traffic and the convenience of such a thing is that There is uh important things uh that you need in general implement in these microservices but they would code heavily cluttered in all places copy-paste back and forth is not very convenient secondly in Overall, it's not very convenient support Here we are such components we can bear it here in this Saika A what are these components firstly That's all what is related to security all sorts of mtls uh, they can just give a square on this site be serviced by collecting metrics Maybe there happening Or maybe Rey again limiter at the level of some kind of pods you will return it. What else could there be? Well, in general, these are all the things that we previously voiced them in principle here you can stuff it in plus the service MSh cool topic And when are you still routing When you pulled the handle of this service, it there, like, according to its logic, it should the second service to go and so that it doesn’t go to the master version and to the version some kind of uh which is some kind custom curl, well, another version Here it is without any service mice you wouldn’t be there I know there would have to be something in the config change sit like Now I'm on time I'll change the address, I'll mix something up mshm service is simpler It’s simple this is configured at the expense of abbi A and that’s it and it automatically balances traffic Where are you you want to change it later, here's how since everything is controlled at the expense of the hay these here proxy is not easy technology but promising but understandable the thing is she has a lot of advantages which I wrote But there are also disadvantages which in general consist in the fact that first of all, the system is just banal it becomes more difficult which in turn imposes certain risks secondly Well, again, an extra VM link is added interaction which may delay increase and less fault tolerance becomes but it's again a balance between that something has gone down there and The flexibility of this system has increased. So what doesn't happen here As they say something absolutely white and something absolutely black sers we figured it out and the cat is already this one says, well, it’s like I’m just okay already done in order to become the best service and agree with him they present the prize, says lushi search engine I developed cheap tickets for cats handsome just praise and honor to you but here to some greedy investor comes to him and says Well, let's do it somehow let's agree I have some money here yes, but the cat doesn’t fall for this provocation and he does the right thing because he is an investor then this one was illegally closed A the cat celebrates the victory that he developed best distributed system on the planet for animals obviously But what conclusions can be drawn from this? everything to do Monolith is good when the load is not so high Well, in general when the function then you also have a Well system itself has not grown but functionally microservices are easier to develop and scale but microservices increasing infrastructure requirements it is also worth observing patterns and the best microservice building practices architecture Because if we don't talk about them at least you can be in the know seriously get burned and there is data they'll leave and you'll be there later collect them uh Or maybe you won’t be able to collect Well, in general, horror And of course that's all depends on the context Who knows the joke will understand about the parrot, but who doesn’t know I I’ll tell you once they taught a parrot say It all depends on the context and he became architect But it arises immediately question: How to make a profit in these microservices at least like this fill it in so you won’t be ashamed to code anymore normal code And you know we are here with they thought like a cat, consulted and banged course which is next week starts now, I'll tell you more about it I'll tell you And after I tell you about I asked him I'm tearing it off yes bong na Goshkevich few syntactic constructions everything is mastered and It’s enough just to go to stepik I can also attach the link later there you take a course for a week for an experienced person to some person who Well, somehow speaks the language Quite enough Well, Of course you need to know how to use git Well, we go on and on, it sucks microservices for those who just rolls in I probably wouldn't I recommended this course because...

it will be too tough if you are not used to it it hurts and so on And who is there on the way from Jun to Medlad he wants something like this increase your technical capabilities this seems to me the best thing to do If I don’t know, I’ve already answered the course in general adapted to common problems that We encounter In development first learn what you need for development all sorts of services goc posts Prometheus I think the rapids too and well, you can obviously be distracted some cool vacancies from these fashionable the words poured in are coming to you, obviously find out The nuances of this practice are built on life hacks microservices Well, that is how it is architecturally it is better to build which There is we’ll write down the nuances of all this later, which means microservice scalable scalable due to this architectural techniques and even several of these microservices I will now tell you in more detail how this system will look like this there will be something to fill the course program with look first week first get acquainted secondly deal with ATP protocol how it all works in GO and jpc we immediately stir up a contract for our twist for working with users and then I check with whoever needs it Well who will be there to check and then spank that is, we already have some kind of the skeleton of the project is built in the second week we start working with preg som because, damn, they did it, describe it, yes it is necessary somewhere immediately save with poso worked understand how to write these queries like roll out migrations at the same time immediately in the microservice area, that is, we’ll try there options how to just pss log into the server and Upload the version of the code there How to upload there image How it is there I don’t know how we are yet let's do a cool thing naha action type you just send it to github, it’s all there rustles rustles and your hras has been renewed the version on your server is already new the application is running in production You You can knock on his door, it’s very interesting Well, let’s write configs for different environments Well that is, I’ll say right away that there will be configs there on environment Well here you go for local there for sale in the third week Well, I’ve already told you everything about migrations. By the way, let’s also see how migration from roll a separate container A third week Of course architecture because if you don’t talk about her right away take care sometime further Everything is over and it will be uncontrollable chaos Let's look at this layered architecture how best to implement it, why and step by step we write these layers Let's develop a Di container, that is let's talk about all these dependencies injection invers back and forth and let's work firstly with pgx directly and we with the base data then already in the second week we worked here, now we have transactions we’ll add how best to do with them and write your transaction manager and uh Interesting option e how can you make a wrapper I hope he's even cooler extensible like this let's say in code Saika is so small uh nuno only in this is the fourth week of the code, how's it going? don't test it, you need everything first this is a matter for us to test in units Let's cover the functionality we need and figure it out How to work with smokes, and let’s highlight a part components to the platform library we'll look at it in week five additional ecosystem system options jpc let's first talk about jpc gway Why is he where is he? By the way, I have a video about this came out recently, you can too get acquainted like this if you're interested say demo access, let's talk about proto validation Why how Why let's look at the sgr as a server server in general raise How in general what else was there A what is Corsa? Well, what if these Corsas cannot be registered and Swag does not work will be In general, let's talk about it all and We will also consider interceptors because Interceptors are an important thing without them in the work of microservices and Nata Neda with us We are dedicated to safety first let's talk about TLS This is transport we generate encryption and connect jpc there I'll take a second and then we'll talk about questions authentication authorization think Let's think about the example of a token, that is, we First, let's make the code that gives us these generates tokens and checks later he gives out some of these roles thanks to which you can access to some part of the system Well, there you are you decide this way that everything will be okay Of course, all sorts of server-client interceptors for working with tokens too we will write so that our services can communicate safely with each other seventh the week is dedicated to monitoring, let's talk let's see about the logs how Sberovsky pledge work then we'll look at the poso metrics We’ll display different metrics in Prometheus Let's build beautiful dashboards, everyone and where without tracing Of course, also with we

2024-02-08 14:11

Other news