Cloud native Azure deployments with Terraform - BRK3306
Thanks. For coming to my session my name is Neil Peterson, I'm a cloud, ops advocate with Azure or with Microsoft working in Azure, in, this session I'm, going to talk about deploying. And managing our. Resources, with terraform. But. Before, digging directly, into terraform itself I want to do a couple little things so. Really the agenda here is that we're gonna look at kind. Of the high level methods. For, deploying. Resources in, Azure so. Manually, deploying resources, using scripts, and SDKs, and then, template, based deployments, kind, of talk about some of the pros and cons of each we're. Then gonna like focus in a little bit more on just, template. Based deployments. What. It you know some of the benefits and declaratively. Writing. Azure deployments, versus scripts, and manual deployments, and then we're gonna focus in even further on and. Terraform. So. Just. Quick show of hands is anybody, using terraform, currently in the organization, oh sweet. Ok cool. So. You, know when considering. High-level. Methods. For deploying Azure resources, as. I just stated we've got a couple different options we. Can manually deploy resources a, resource, being a VM or, cosmos DB or Azure Container instance, in. This case we can manually deploy these with the azure portal. We. Can also write scripts. Using. Power shell or, bash in the azure CLI, or any, one, of the various, ash or SDKs, that we have and. Then finally we can automate deployments, using a couple different templating. Methods. So. Let's kind, of dig into each one of these talk. About the pros and cons and and there really is no you. Know this. Is the best way to do it I think they all have their use cases, so. Let's talk about some of those, so. You, know most fundamentally, the azure portal can be used to. Manually, deploy, and configure, resources, and kind of a you, know point-and-click, fashion, I actually. Use the portal quite a bit I like it when. I'm testing new. Or, kind of just going on an exploration. Mission. And, just kind of want to like kick the tires on some stuff in Azure I find, the portals a you know a great place to wrap. My head around a. Particular resource, or a resource, configuration. I, also, use, the portal quite often to validate some of the automation that I've done so if I am deploying resources with, a script you, know I quite, literally might have the, portal open on one, screen and vyas, code on the other screen and I kick, some stuff off and I just switch over and kind, of noodle, through what's been done just to validate that you, know what, I want to happen has. Occurred. Another. You know cool thing about the portals you know for the most part it's, fully featured, there. Are not many, resources. Or, configurations. In Azure that. Cannot, be be, made through the portal so you, know if it's an azure rest. Assured I should have access to, deploy it or configure.
It From, within the portal I mean. That said manually. Deploying, resources. You. Know is not quite suitable for modern day or cloud. Native deployments, where you might be deploying something over. And over again. When. Manually deploying resources not only you, know do we have the human body there you know so if. I'm gonna deploy something in the portal I need to be present. But. Also I. Make, mistakes. So. You. Know. The. Human element. Can. Really be a crux when when looking at production ready deployments, using, something like the portal, you. Know not to mention in my experience. Using. The portal is not necessarily, a whole lot easier than. Writing, a script with. Something. Like the azure CLI, or, a. Terraform. Configuration. So you, know I don't know that you, know hey I'm new to azure so, I'm just going to start using the portals isn't necessarily really the, right move because you know as we'll see, terraform. And even, things like the as your CLI or or powershell do make, deploying. Resources in Azure relatively. Simple, and. Then, finally you know I really think the big one when looking at manually. Deploying resources. Into. Azure or any cloud provider or any system, at that point you know one of the big disadvantages. Is you. Know we really don't have that ability to. Integrate. Those, deployments, or configurations. Or. Updates. Or even deletions. Into. Existing. Process, in. Automation. Processes, such as you. Know change management, problem management or, any kind of DevOps workflows, so we're, really limited when, manually, deploying resources in, hooking. Into those automated. Business processes, so. That's. The portal. Moving. Down one level we. Have scripting, languages such as PowerShell. Bash. Using, the azure CLI, and then. Multiple different software development, kits like the Azure SDK, Virgo and. Python. Additionally. You know if there's if you get into a situation where, you, know you're you don't want to script, and there's not an SDK, available you. Know roger has a series of rest. Endpoints, that really, give us total flexibility. When when working with Azure and managing. Resources and measure, so. You know a lot, of the the pros here are pretty obvious and, the. Big one kind, of in stark contrast to, manually deploying resources is, that, ability to automate these deployments and then. Hook into. Process. Automation, again. Change management, problem management and, he kind of DevOps, type workflows. But, additionally, you, know when we start automating stuff particularly when we're looking at at deployments, where we're we're deploying something over and over and over again you, know removing, that human element is, is. Pretty spectacular. Not. Only is is the human not necessary. Necessary. To trigger those deployments but, also you know we remove mistakes, and in. Kind of the stupid. Human. Component. Of these deployments.
Another. Thing and we're, gonna contrast', this with templates that. I think you know scripting, has going on for it is. Total, flexibility. In in. Our automation so. For example if I'm automating. The deployment, of virtual, machines and, I want to do something like you. Know once a VM has been deployed. You. Know I want to you know check it and maybe send an email I mean I'm just making something up you, know I have the flexibility, to do this because I'm basically, inventing. This. Deployment and I'm inventing all this stuff so my step might be deployed, VM. Validate. VM send. An email and I can do that relatively easy with. A script, or or using SDKs. However. There are you know there are some things to keep in mind here you know when we do take on scripting, deployments, there's definitely. You know some. Knowledge. Scripting. Knowledge that's necessary you, know we need to bring. Ourselves up to date with the SDKs, and, you, know there's some work that goes into that, as. Well. You, know from my experience, there. Are some things that you, know, scripting. Just, doesn't do out of the box that I may get as we'll see with, things like as your resource manager templates and terraform. And that is you, know when, I'm deploying resources to Azure it's. Not just like create, VM, I. Might want to do things like I want to create a VM with, this name and this resource group but, I need to check if that you, know if a VM already exists, or. Create. 10 VMs, you. Know so if I'm scripting, this it's up to me to write that loop in iteration. Logic. To. Deploy. 10 VMs, give them unique names give, them unique public IP addresses, if that's what's needed so. I have to build this logic into my script so there's some overhead, in kind, of these you know validation. And iteration steps when deploying stuff to in. Any cloud provider at that for, that matter. Let's. See. So. That's scripting, in SDKs. So. Lots, of benefits over manual, deployments. Yet. Some overhead some. Learning curve and then, you know we need to think about these these, complex scenarios. And. Then finally let's talk about you, know template-based deployments, in, for the sake of this portion of the talk let's just you know what we're talking of. A, Glee about. Declaratively. Deploying, things to Azure let's just keep in mind as your resource manager templates and terraform. There, may be others and I'm not gonna like compare, these things side-by-side, but, let's just like keep those in mind because they're definitely two of the big ones. So. When deploying Azure resources. With. You know either either arm templates, or terraform, we, definitely capture, many of the same benefits as using. Scripts. Or SDKs. You. Know again. One of the biggest ones there being that the ability, to integrate these deployments with process automation. We. Remove the human. Less. Error-prone. We. Can deploy. Over and over and over and over again, and. In you know rest assured our deployments, match each time. But. One of the you know major benefits, that I see here is that as. A resource manager templates and terraform. They.
Handle, Some of this complex, logic for us that that we didn't see with with scripts so. You, know with, a with, a template, you know I basically declare hey I want this, VM in this resource group. And. If I deploy a as a resource manager template or a terraform, configuration. And that. Configuration. Already, exists, you know, as a resource manager in terraform is smart, enough to know, like hey we're. Already in this state so, there's nothing else to do here or. If, I, want to deploy ten VMs, or ten container instances. As. Your resource manager templates in terraform, have, built in systems, for managing this so instead, of having to write you know a bunch of loop and iteration logic, it's, just a couple lines to say hey I need ten of these. And. Then you know even even. More, importantly than that in my opinion is like deletion. Like I have ten of these but now I only need three of these, terraform. And we're gonna dig into this a little bit more but. Template. Based deployments. Handle, a lot of this this complex logic for us. However. There are some you know disadvantages. As well the, first one here in the slide is very similar to scripting. You know there is some. Syntax. And, you, know as, we'll see these these declarations, are, kind of written in JSON or JSON like format you, know there is some you know you need to become familiar with how. To write the templates and there was some overhead there. Unlike. Scripting, and using, SDKs, though. There. Is some. Lack of, flexibility. That scripting, gives us so going back to the example that I just talked about where. I want to deploy a VM check, it and then send an email there's no native, system, you know in as your, resource manager templates or terraforming configurations. To do that that's. Not to say that you can't you know wire or something up or get creative around it but, you do lose, some, of that flexibility. That we have with, scripting, when when moving to template based, deployments. All. Right so. We've. Taken a quick swing through the pros and cons of each deployment, method let's. Now kind of like focus in a little bit more on on, templates, or. Template, based methods, for deploying eyes or resources, or, declarative. Based methods, for deploying resources, we'll, talk through them and then we're gonna get into some examples and, some demonstration. Alright. So we've already seen that you know template based deployments, provide automated. And repeatable, deployments. When, writing. Template. Based deployments, rather either in Azure resource manager templates or terraform, we're, basically writing a file. That's. JSON. Or JSON. Like and. Inside. Of that file where we're declaring the, desired state, for our deployment. So. We're, doing something like stating, I need a, storage. Account name X a. Cosmos. DB instance name Y and. Three. Container instances. We. Then deployed, the template or the terraform, configuration. The. The as. A resource, manager, terraform, kind of evaluates, the state of our environment. Compares. That with the declaration. Inside. Of the template and then, just like makes it so if. It's a new deployment it's gonna say like hey I don't have the storage account I don't have this, DB I don't have these container instances, let's deploy them all, or. It might say like oh hey I've got all of this but only one container instance, let. Me go ahead and add two more because that's the the declared, state. And. Then. Finally you know I already talked about the the format of the files and we'll see some examples here, but, when using, template based deployments, you know we basically have deploy. Update. And delete. Actions, built. Into, our deployments. So. I am. Gonna talk a little bit about as your resource manager templates will see an example then. We're gonna talk about terraform, configurations. And we're gonna see an example this isn't like a super-duper. Bake-off, like I'm not saying one over the other or anything like that I just want to compare them side-by-side a little bit some, of the pros and cons of the each and even.
Though This was a terraform. Session, I felt, it was important, to do this because there, is as. We're, gonna see there's it there's a couple differences, between the two and one, of them in my opinion is pretty major in, that you know it's something to consider like when you're putting together a strategy, like how, are we going we've, decided we want to do, declarative. State deployments. Now, we need to decide on a platform whether, that's as, your resource manager terraformer. Or maybe, some others there. Are some big decisions to be made and I wanted to kind of you. Know do, a quick brief, comparison, just to talk about some of these big, big. Items, so. What, a resource manager templates so, a resource, manager template is a just. A quick raise of hands anybody used, arm templates, alright. Cool so just. Quick brief you know overview resource manager templates are written, in JSON, like like, total. Like compliant. JSON. We. Have tooling, for visual, studio and, Visual Studio Visual, Studio code. And, you know again you're declaring like hey I need a VM in this resource group I need four of these five. Of these there's. Ways to kind of hook stuff together or, or pull data from one resource and use it in another resource. One. Of the cool things about as your resource manager templates is there is some native portable, integration. And in. Kind of two aspects, to that so if I've got an arm template, like the JSON I can, actually go into the portal and click like, new, deployment. Or new resource, template. And just paste, my template, into the portal and then, deploy a resource that way it's, kind of just an easy way to like just get something done pretty quickly, but. I think more exciting, is I can take an, existing. Resource group, that may have like 18 resources, in it and I, can export, that to. An azure resource manager template, now. This process is not perfect, you. There, there are some funkiness in it I mean it's a deployable. Template, but it's, not. Exactly super, pretty, every time however. When I am writing resource, manager templates, and I need to see an example of like man, how is this cosmos DB configured, you know there's a lot of configurations. In here and I want to see an example of how this looks in a, resource manager template form or that or the JSON, that I need to deploy this it, is helpful you know you can actually go into the portal deploy. Of Cosmos DB, export. It to a template, and then go in and like zero in on that JSON, and get a pretty cool example of what, it needs to look like and. So it's definitely you know that's definitely pretty cool in terms. Of resource, manager templates, but. Then I I mentioned. Kind of a big one that I want to really like zero in on and and. It's this last bullet point here so as your resource manager templates or our, or, the the language around, it is derived. Directly from. Azure. Rest endpoints, and swagger so, you, know if you, need to make deployments, using Azure resource manager templates, you, know you can be pretty you know rest assured all. Resources, and configurations. Are going to be available for you in those, in the in that templating, language. And. Then so here's just a quick example I hope you can see that yeah I think it looks alright we're. Not gonna go line by line I just wanted to kind of like, show. What. An. Azure resource manager template looks like, next, to a terraform. Configuration. So, a pretty simple little template here it does two things it. Deploys a resource group. Which. Is a new feature if, you've been keeping track we haven't always been able to deploy resource groups with a resource. Manager templates we used to you know if the creat the Ruiz pre create the resource group and then deploy the template into the resource group so, it's pretty cool we've got that capability now and then, finally we're deploying a storage account and, again, the the, the, goal here is not to go line by line. The. Goal here is in two slides we're gonna see the exact same deployment, using. A terraform, configuration. And. You'll you'll kind of be able to see. Some. Of some of the differences in in kind of the the templating language there, so. That's that's resource manager templates now let's. Dig in to terraform, so what is terraform. It's. An open source project. Cross. Computing. Environment. Templating, language, so. What this means is I can provision update, and delete, resources, not just to measure but. In all major clouds and many. Many many many computing, environments, so. This right here is a huge, differentiator. If, you, know using. Azure resource manager templates I'm kind, of limited deploying, resources. Into. Azure whereas. You know with terraform maybe I do you, know maybe I am in a situation where I need to deploy resources.
To Azure and, then. You know a couple of resources somewhere, else you know maybe another, cloud provider, maybe. On Prem mage maybe Azure stack. Using. Terraform. And we can do that, as. We'll, see you know I can actually write a single terraform, configuration. And deploy. Resources, to Azure and deploy. Resources, somewhere. Else within the same configuration. We're. Gonna get into exactly how that works here in a couple slides. And. Then, you know as far as like how the tenth Leo how we author the template it's authored not. In its, JSON. Like it kind of looks like JSON. But. It's authored in something called Hoshi Corp configuration, language or HCl as. Well, you can author. Terraform. Configurations, in straight JSON but the, recommendation, is HCl, and, we'll see why here basically, it's more, human readable, it, supports comments, it's, got a couple of neat little tricks as well. And. So. Here is the exact same deployment, that we saw with the arm template. Using. A terraform, configuration. So we're deploying a resource group and a, storage account and if you look at that it does look very JSON. Like but. That itself, right there is not not compliant JSON. But, it is there's a lot less going on there to. Achieve kind of the same results, as we. Achieved, with the azure. Resource manager, templates. So. Let's talk about providers, so I mentioned, that you know we're, not limited, to a sure we're not limited to one, computing. Environment, in fact terraform, supports or there. Are, you. Can deploy resources, with terraform, to a lot of different computing environments, and this is done through terraform. Providers, so. You know what exactly is a terraform, provider. You. Know just think of a provider. As an extension. For. A specific, environment. You. Know really like terraform, itself is the engine, and, then. I've got providers, for things like Azure or, you. Know as your stack or I. Think. I may have even seen a provider, for something like exchange, but. They're they're different you know code bases so we've got the engine and the provider and the provider contains. All the logic. Specific. To, the API that it's targeting. I, already mentioned that you know we've got providers available for all major clouds, and other platforms. And. Then. This last bullet point right here. The. Providers, at. Least the Azure provider, I haven't, looked at all the providers and, provider, itself is hand authored and this is kind of going back to that big differentiator. That I called, out. So, what this means is that you. Know as new, features. Light. Up in Azure or, as. New resources, are made, available in Azure the. Terrace somebody needs to actually hand author, the.
Code And get. That committed. Into. The, Azure provider, in. Terraform. So. This is I mean there are there are some limitations here I mean whereas in Azure, resource manager templates that stuff is all kind of like auto-generated. Here, we're kind of like depending on somebody doing the work and I actually ran into this, prepping. For this demo. When, we given our demos I'm gonna be deploying a bunch of Azure container instances, not, because there's any like relationship, between this talk and as your container instances, and just because they deploy quickly. And. Within, azure container instances, we have something called. Secure. Environment variables and what, this does is I can like set environment variables inside. Of my container instance, and the. Values are not visible in the azure portal, so. It's just a way to kind of like give it another layer of abstraction on, top of potentially, sensitive data and I wanted to do that for this session and. The terraform provider for assured has you, know a module, for container instances, it's just that one little feature has not been lit up you know nobody's like authored that and put it into into. The module so there was kind of like this delayed and functionality, there, the. Cool part is is it's an exciting project it's. Open source and you. Know based. On this we now have a PR waiting to be merged that that, adds this functionality. But, this is you know this is one of those things that when you're deciding on like how are we going to proceed with this this, is like one of those big considerations, that I would I would take into consideration, now, that said in in in my experience, I haven't run into many, gaps. In in, the azure provider, the, terraform, provider for. Azure most. Of what I need is there and it's pretty full-featured, but but there is some risk there and, I kind of just see this in like a visual representation. You've. Got terraform, on. The left-hand side we've. Got the providers, and then the endpoints, another, good example of like, an. Example that I've seen several times in my role where. Our multiple, providers, have, come in useful is. Somebody. Wants to deploy kubernetes, cluster. And. Then you, know automate, kind. Of the bootstrapping, of workload, on to that cluster so I I not only want to deploy the cluster I then want to start a bunch of work on it as. A resource manager templates work, great for deploying urban. IKS, esra kubernetes service, cluster, in Azure but. They don't there's not a lot of capability. There in terms of like ok now my cluster is deployed let. Me go ahead and start something on it however. Using, the terraform. Provider for. Azure and, the. Terraform provider for kubernetes, so two different providers, I can. Deploy that cluster, using the azure provider and then. Start workload start. Containers, start pods on that cluster, using, the kubernetes, provider, so just, a practical, example of, using, multiple providers, to, kind of complete something into end. Alright, so we've talked up kind of you know at this point we've talked about what terraform is what a provider is we've. Compared. Methods, for deploying, resources, into. Azure, let's. Kind of get into the meat of how this is done, like what it looks like to author a terraform, configuration. And then, I've got a bunch of demos planned, as well, so, you, know when, we're. Building. A terraform configuration. We. Will create a file with either a dot TF, if we're. Using the HCL language, or dot. TF dot JSON if we're gonna write our terrifying configurations, in JSON. And. Then we basically, we're. Putting a couple things in it the first thing is the resource type, so. You can see here I've got highlighted. Azure underscore, resource, underscore, group so. I'm declaring like hey I am I want, to create, or. Manage or, or do something with a resource group and this relates to the provider. This. Next piece here is a name, now. This is not the name that I'm going to give to the resource group this. Is kind of like an internal name this is name that I'm going to reference this resource. With. Inside. Of my terraform configurations. And we'll see why this is important, and, in how to use it so basically I'm. Just naming this instance, of this thing that I want to manage. And, then. We configure, the resource and. And. And for. The most part generally, this is just going to be providing, a value to a predefined, parameter. And. And I'll show. You and what am I in one of my demos kind, of how to go get the, schema. For the resource that you want to create, which. Then kind of defines like here's all the parameters, that you need to provide values, to and what they are as.
Well As like the outputs from it. And really that's that's. About it. At. A very basic, level. Once. You have your configuration. There. Are a couple commands that you're gonna use quite often, the. First one is terraform, and knit so. What terraform and it does is it it looks at all of, the. Terraform. Configurations. Inside, of the current directory. Pulls. Down the providers, that are needed across all of them, sets. Up a back-end if, a back-end is being used which we're gonna talk about. Yep. So that's terraforming, it we, commend run terraform, plan and this is pretty cool, a terraform, plan can be used to preflight the configuration. It. Basically compares, the configuration. So. Everything. That you've put in the dot TF files. To. The. The target. So. It's it's basically comparing, your intentions. With reality, and then, it spits out kind of a report, like hey you're saying you need nine of these I. Found. Eight of them so. What I'm going to do is create one more I mean, it's a good way to really just kind of like visually, really quickly like hey am I, about to do what I want to do or is there some like consequence. Here that I'm not aware of or is there some sort of configuration, change it's gonna be made that. I don't, intend on making so it's just kind of like a quick plan, we. Then can run terraform, apply which will commit the change infrastructure. And. Then finally if you want to delete. All. Of the resources. Inside. Of the configuration, you can use the tariff and destroy command. And. There's a bunch of other terraform, commands but these are kind of like the daily. Drivers if you will so. I'm, gonna jump into a demo and, this, is gonna be SuperDuper, basic but that's intentional, and. Then. We're gonna get into some, more kind of complex like real world real, real world type of configurations. Cool. All. Right so. I'm. Working in a vs. Code here, I'm. Inside, of this directory, here a 1 - demo, - scratch and I've. Got a file, main, TF, and it's. Empty and. I intentionally left it empty because you, know as I started, working with terraform, one of the things that I was left very impressed, with was.
How Easy it was to write. This stuff from scratch using. The terraform documentation. So we're just gonna actually write. Something here totally from scratch we're just gonna deploy a single resource group just keep it simple for now, so. I'm going to open my browser. I'm. Just gonna do a search for terraform, as your resource group. Here's. The documentation, right now right here within. The site within the documentation. We've each. For, each resource we have example, usage, and then. Argument. Reference, and. Then. Finally attribute, reference. Argument. Reference is like the things that you need to configure so in the case of a resource group a name, location. And tags and attribute. Reference is the output that's. Going to come when we create that resource and we're gonna dig into that after. This demo so we'll see that but. For the sake of this here's kind of the example usage, I'm. Just gonna copy it out. Paste. It in, for. The sake of this demo I'm just gonna remove the tags I don't need those. I'm, gonna give it an internal, name again this is not the name of the resource group but an internal name. I'll. Just do internal, and. I'm. Gonna give the resource group a name and I'll do. Like. Demo, one. So. There is my tariff, on configuration, that's it that's all I needed to do to configure. This. Configuration. No. Down here I'm gonna do. To. Reform in it. So, we can see terraform has been successfully, initialized. It. Validated, it has the providers, locally. That it needs to to, run this configuration and. As. We'll see terraform and it does some other things I mentioned back-end configuration, we're gonna dig into that I. Actually. Have an alias, for, terraform of just T so if you see me just use T the. Actual commands, terraform. So. I'll do terraform, plan alright. So right here we see new no valid, unexpired, as. Your, CLI auth token, so this is pretty cool I can actually I've got the azure CLI. Installed. On this machine I can actually, like. Piggyback, off of the azure CLI, to authenticate, with Azure from. The, terraform configuration, so. I'm just gonna run a Z login. All, right so my address tli has been authenticated. Alright. So terraform, plan. Again. My intentions, here just to deploy a single, resource group and I can see right here in green I've got plus and. Then. As a resource, group dot internal, I can. See the location, in the name now. If like it was going if it was if there was like a resource that, exists. In Azure but. Had been removed from the configuration, like. Right here I would see like a in. Red like a negative, and the resource name, so. It's a pretty again, a you know Terra firme plan is a pretty good, indication, of like hey what's about to happen like oh no. There's some red they're like I don't want to delete anything like let me go examine. What I've done and. Then. Terraform. Apply. And. It's gonna ask me like are, you sure you want to do this click. Yes or type. In yes. Boom. So we can see resources, one added zero change zero destroyed. So. I'll come over here to the. Azure resource as, your portal, and. There. It is right there ignite, demo, one. So. Again, in tensions there were two really, just. Show. How. To use terraform, documentation, and i'm actually going to come back to documentation, in visual, studio code and then, show some kind of cool things around the extension, for terraform, in vs code. But. Simple. Configuration. Pulled, it from documentation. Pretty. Easy peasy so. Let's kind of dig into. You. Know really, there's a whole lot more to terraform, than what we've seen thus far on, the next few slides i'm going to talk about just, a couple kind of configurations, that you're gonna use quite. Often in, most of your terraform, configurations, because obviously not everything is just gonna be like hey not that easy there, are some things that we need to configure we want resources. To communicate, with with each other I've. Been talking about this back-end thing, what. If I want to pull data for from some other places let's talk about that so, first up variables, and output. There's. A couple so terraform, input variables, are defined in a variable block I'm. Sorry I didn't, click Next. So. An input variable is really, kind of like a parameter. Like. We just created a resource group and I kind of like statically, gave it a name but.
I May want to provide that dynamically. Or I may. Want to be able to like change that name without, going through a bunch of my configuration, files and changing. In a bunch of different places so. Particularly, things like a name of a resource I really don't want to hard-code that. So. To get around that I can define. An. Input. Variable so we can see right here on the screen of that variable, resource. Group I've. Got a description and a default value and, I. Can just define this in any one of my terraform, configuration, files, we'll, see when we get into the next demo what I do is I actually create, a variable, ztf, file in the, same directory with. My terraform configurations, and I just define my variables, there. So. It's pretty simple it's just it's just a parameter, that. We want to use throughout our configurations. Additionally. We can use environment, variables, from the system, inside of our terraform configurations. There, are a couple little like caches here so, when, you create the, environment. Variable. You. Need to give it a name of TF, underscore. Var, underscore. And then like. A descriptive, name so. Right here like the first line on this screenshot. Here I've, got TF underscore, var underscore, as your client, ID equals. And then I would have like you know the grid of my, Azure client. Then. With inside of my terraform, configurations. I define, a variable, so we can see here I've got variable. As your, client ID, but. Unlike. Over here where I've given the resource group a description and a default value I can just leave it empty and, as. Long as that environment. Variable is defined in my system or container wherever, it is I'm working and the. Last part so the Azure client, ID of the environment, variable matches the, terraform, variable, with, the name as your client ID it's, just going to use that value, from the environment variable and this is like super duper helpful when. You know you've got sensitive. Data that you don't want to write, to a configuration or, whatnot and. Then. Finally we've got output. So. Most fundamentally. Output. Displays. Something. When. We've run terraform, applies we run terraform, apply our, resources, are deployed, and then it's going to output, to the terminal, whatever we define in. Output block so, you can see here I've got output. Resource. Group and value. Equals. You, know resource group, dot state demo, name. What. We're gonna see and we're gonna dig in the output quite a bit as well I've got a demo just on output it's, not just, as simple as like hey let's output something to the terminal, there are other really cool things that we can do with with output. And. Let's, we'll just save those for that section I'm going to talk about, something. Backend. That I've mentioned a couple times and basically, getting output, data from one deployment, into another, deployment so. You know most buttons to wrap up on that most fundamentally, it displays. Something that is terminal but there's some really cool things that we can do with output data. All. Right so that's variables, environment. Variables and, output. The. Next thing we're going to talk about is string, interpolation so. The. Definition, of interpolation, is the insertion, of something, of a different nature. Into. Something else and it sounds.
Complicated But it's actually pretty simple here, so. We'll. Just start with an example like, let's say, that I've got and we're actually gonna see this demo I've got an azure cosmos database. And. I've. Got an application running inside of a container instance, and I, want to deploy these in a single, terraform. Configuration. However. I need to get the connection string, from, the cosmos, DB, and set. It as an environment variable in, the container instance, so, that the application, in the container instance can write data and read. Data from. From the cosmos DB and this is this is an area where we might use interpolation and we'll see how to do it here most. Fundamentally though when we define a variable. The. Way we get that value, into our configurations. Is through interpolation, so, you can see on this configuration, on the screen we've got resource. Azure. Underscore, resource underscore, group it's. Got an internal name of demo RG and then, instead of hard-coding the, name I'm. Now providing. It a value, that. Is stored in a variable. So. Basically. The way we do this like dollar sign. You know squiggly squiggly. Bracket, and far so, variable. Dot. Resource. Group the name of the variable that we've defined and, it's, gonna suck that value, into my configuration. But. Going back to my cosmos. TV and container instance example. That I just went through. Stringer, interpolation is not just, about getting. The value of a variable into my configuration. It's. Also about you know we looked at the documentation, and I said like you, know it had that that piece that talks, about all the outputs so every time I create a resource, a bunch. Of output is created, I, can, create that Kosmos DB, resource, and some. Of the as part, of the output when I created that is going to be you. Know the endpoint, access. Keys whatever. Information it is I need to, connect. To that cosmos DB instance I can, actually interpolate. That information. Into another. Resource, in kind of the same manner so, we can see here.
I've Got. A, resource, of a. Sure container, group with. An internal name of demo a CI. I've. Got a hard-coded. Name of demo, a CI but. Then I've got a location and I'm, actually using the, location, of a. Resource group, that's. Also, defined in the same configuration. And, we'll we'll, see this in a demo I know the the screen, real estate here is pretty pretty. Small but we'll see how this works in a demonstration but. It's through this method, that we can pass data from one resource to another. There's. A bunch of other ways we can use this as well I'm not gonna demo them or really talk about them too much but we, can terraform. As a bunch of functions think about things like to. Upper like if I have a string, and I want to make sure that all the the, letter is in the string or uppercase you know we've got a function that can do that and those, are also defined inside. Of interpolation blocks, and there's. A bunch of others I put a link in the dot in, my, slide deck to the docs it's got a whole list of functions and things, that you can use. And. Then finally before we jump back into some demos there's dependency. So, terraform. Uses dependency, information, to determine the correct order in which to create resources. This. Is important, you know, for instance if we're gonna create some. Sort of infrastructure. They'll be hosting an app and accessing a database and, we might. Want to make sure that that database is created. Prior. To starting. The application. There's. A couple different ways that. Terraform. Manages. Dependencies. Excuse. Me the first is implicit and this. Is really cool because this is what I mean most most. Of the times I'm writing configurations. I use, this method for determining, dependency, and what, happens is is terraform, actually, like traverses. Or traces, all the, interpolations. Inside, of the configuration. And builds. A dependency, map based on those so. Going back to the example if I've got an azure cosmos database defined. In my configuration and, an, azure container, instance, and I'm, setting an environment, variable in, the azure container, instance, that is you. Know deriving, of value or interpolating, a value. From, the cop output. Of the cosmos DB. Terraform. Like automatically, builds that resource, it's like hey I see, that you want to use information from that resource in that resource so, obviously, that resource, needs to be created before that one. In. I've, done a bunch of these and and usually that's sufficient. However. There may be cases where that's not and so you can hard code an, explicit. Dependency, using. Depends, on and you can see there. Is an example right there so it depends on and I basically give. It the name of or. The path to the resource that I want, to make sure is deployed prior, to deploying the one that I'm working on. Cool. So let's get into something a little more meaty. All. Right. So. I'm gonna walk through this configuration, real, quick and just kind of point out some of the things that we've talked about thus, far so. First thing I'm doing right here is, deploying. A resource group. And. I said I was going to talk about like kind of file configuration. So, within this one folder, here number. To terraform, create I've got a main TF, this is where I'm defining all my configurations. I also. Have a variables, TF this. Is where I'm defining all my variables, this. Could be a single file like there's nothing saying that I need to separate these in different files, this. File name doesn't need to be variables, TF, it could be whatever I want when. I run a, terraform, and NIT it's, going to go through the grid directory, the. Current of the working directory and take all dot TF files and kind, of combine those up it's. Just a neat way to organize, things and you can see in here I've got a variable, name resource group, with. A value, of the demo. Location. With a value of East US and then I've got, a couple others defined. As well, so. Back in my configuration here's, the resource group and unlike. The first demo I'm actually using a variable to provide a name, and a location to, it. Next. I've got this, weird thing resource, random. Integer. So, I'm actually using another. Provider here, that. Just creates, a random a random, integer I, need. This because I want to make sure that my next resource which is an azure Cosmos database account I want.
To Give it a very unique name. So. I'm using that random, random. Value. That's output. From this resource as. An. Input, over here so you can see I've got this resource. Random. Integer, with. The internal, name RI and. Then. Down here for the name of my azure cosmos DB I've got a string. Of vote - cosmos, DB - and then, dollar sign. Random. Integer, so. The, resource type matches, dot, RI. So. There's the internal name dot. Result. Well. How did I know that it was a result that I needed I, went. To the documentation, and found out I was. Going to show you something cool but I'll show I'll click down here and my as your cosmos DB one so. The next resource I have here is resource azure cosmos DB account. There's. A bunch of configurations, in here there's also a bunch of data that comes out how, do I find this stuff out I go to documentation, one. Of the cool things about the vs code plug-in is as I'm working on these resources, straight. From vs code I can click on the resource name or. Type and then. I am. Navigated. Straight to the documentation, so I use this, every. Day it's, so helpful, you. Can see in here I've got an example usage, for. Cosmos, DB I then. Have a definition, of all the stuff that needs to be defined whether, it's required, whether it's optional, and. Then. All the attributes that come out of it so. In the case of this demo we're gonna see endpoint, and. Then. Probably. Primary. Master. Key. So. I've defined a j''r cosmos, DB account I've, configured, it up we can actually see that as far as the location and resource group name I haven't, hard-coded, these I'm using. Output, data from. Up here, and. Then. There's. Some, other configuration, in here that you. Know it's just it's cosmos, TV specific stuff, I won't get into. Notice. Here that I also have some stuff commented. Out I, wanted, to instead of just deploying this well actually the cosmos. TV takes a little while to deploy so, rather than doing it live I did. It just before I came in the room but, there's some cool things about this because we're actually gonna watch not just like deploying something new but, we'll see like an iterative, deployment.
Here. So. What I'm gonna do is. Actually. Let me do it this way I'm gonna go ahead and uncomment, this stuff we'll talk about it and then. We'll run some commands again. So. What, I just uncommented. Was an azure container, group I've. Given it an internal name, I've. Actually hard-coded, a name here vote - a CI, using. The resource group information for its location. You, can see here I've got a DNS, name label, I'm pulling from a variable that's defined in my variables, file. Defining. My container. And then here you can see that I am, setting some environment, variables, inside. Of that container instance, and two, of those are. Well. The first one is the, Kosmos DB endpoint, and the second one is the DB master key so. Here I'm pulling that, output, from the, cosmos DB resource. Into. My Azure container instance resource, you can see here Azure cosmos. DB account, so. That's the type. Dot. Vote. - cosmos, DB is the internal, name and then. The, piece of data that I want to pull in. Now. So if we look over here in. This. Resource group I already have the, azure cosmos, DB created, so, let's actually step through this and see what happens so. I'm gonna do. Terraform. Plan. Oh. Did. I not save my file oh. I. Know what's happening I need. To change directories. And. What we're gonna see is it's gonna come back and stay alright you know I. See, the resource or I see the resource group and I see the cosmos DB account but I don't see the Container instance, so. We're gonna go ahead and and, jam that in and, how. It does this is it actually compares. When. We create. When. We run, a terraform, configuration. It creates. A state, file so, it creates, a representation of what's been deployed let's. Just hold onto that fought we're gonna dig into that in depth. All. Right so do you terraform, apply. And. Yes. And then I'll come over here, refresh, my resource group, there's. My container instance. Then. If I click in on my container instance, and, go. To, containers. And. Properties. You. Can see here that the. Kosmos DB master key and the. Endpoint has been set I could, actually go and compare this to the master key on the cosmos DB instance and you would see that they've matched so.
It Was able to using. That state file you. Know grab, the. Endpoint and, key. For that cosmos DB set, that as environment variables inside, of the container instance. Alright. So. Cool. So something a little more complex and again, here we've seen variables. We've seen interpolation. We've. Seen using multiple, providers. In a single configuration, so, we've got the azure provider, and then just this random integer, provider. We've. Seen variables. And we've, seen taking, output from one resource, and and adding, it to another resource. Let's, now dig, in a, little bit further. And. Talk about state, so. I mentioned. You know when a when, a terraforming configuration, is run it. Writes, a bunch of state to. A state file, and. It's. It's through this state file that when I do something like plan, or apply it compares. The state of the file to. The actual configuration inside. Of you know whatever end point in this case a sure, so. When I when I did a terraform, plan on, the last demo is able to take. The state file compare it to Azure and realize oh wow you know there's a delta here like, I don't see, this container instance, piece so let's go ahead and create that. Init. Stored it locally. Terraform. Includes, the. Ability to store, your state files in a remote back-end. Or a, back-end, and there's, a lot of reasons you want to do this in fact I'm gonna just like be bold and say you always want to do this so. What are the what are the issues with local state. It's. Hard, to work in a. Collaborative, team, environment using, local state like, quite, literally, that file is on my laptop, if, we, are a team working. On something we would then have to come up with a way to keep our laptops, in sync, or to make sure that we're using the, same state, file that's. Not ideal by any means it's. Also easy to delete and lose locally, but, most importantly, and. We've already seen this, these. State files can include, sensitive, data so. When I created that Kosmos DB account it. It, put the endpoint, and the key access keys inside of that state file in fact right now locally, on my laptop there's, a state file with. That access, key and, that's that's not a deal, so. The alternative is to store it in a state back-end there. Are a bunch of different back-end providers, one of them being Azure storage and we'll see that and. Then finally data sources what is a data source so, you. Might. End up in a configuration where. Let's. Just say there's a networking team, deploying. Network infrastructure, and you're on like the line of business app team deploying, your line of business app but. Your line of business app needs to use information from, the.
Network Deployments. Like the B net name, so. A. Data, store is a way to interpolate. Data into, your configurations. From some other source we've, already talked about variables, and other, resources. Within the same configuration, but. In an example I just gave I may want to go get data from somebody, else's configuration. Somebody else's state file and pull, that into my configuration, or maybe even not a state file maybe a database, somewhere and pull it into a configuration file so data. Sources or data source providers, allow us to kind of defined a remote place to. Get data and pull them into our configurations. And we'll, see an example of that. And. There's. Just a we'll actually see this in the demo so in the worries. Alright. So let's take a look at. Remote. State remote, state and data sources. So. Going back to the demo I just gave here's an example of the state file so terraformed, TS state and i, said that they're sensitive information in here and I, won't I won't dig through it but there is sense of information in here so, let's see how to store, this remotely. So. I've got another folder. Here. Three. What. Happened here. Three. - azure - back-end, I've, got a variables, file just like my other deployments, I've. Got a main file and it's the exact same type, of configuration, a resource group a container group oh actually. I don't have a cosmos DB here it's just a resource group and a container instance. But. I've got this additional file, here back-end, and basically. What I'm defining is that I want to configure a back-end of Azure RM, here's. The storage account name. The. Container, name which in this case is just like a folder. Inside of that storage, account and then, the key this, key is like the name of the state file that I want to create. This. Back-end so this is just a provider just like a resource, provider. Terraform. Has a bunch of back-end providers. And just. So happens that we're using the back-end provider for Azure storage here, now. Quickly, I'm going to.
There's. Another configuration, that's needed here which is the storage access key I don't. Want that access, key stored, in a file locally on my disk so. I'm just going to set it as an environment, variable. And. Then. Before I run this let's just jump. Over to my Azure. Portal, here I've. Got this resource group ts state, inside. Of it I have a storage account. Clicking. Blobs and TS, State. And. There, should be one state file in here this is for a demo, that we're gonna I'm. Gonna show in a minute so what's going to happen here is when I run this. It's. Going to come back and say I need to anit the, directory, because I haven't done that yet. Oh, same. Problem I need. To get in the right folder right, directory. All. Right so you can see initializing. The back-end. Go. Ahead and run my configuration. And. One. Of the cool things about state is you, can see there it. Actually said acquiring, state, lock. So. We're actually gonna lock that state, file in the, storage account when, any kind of state operation, is occurring, so, that in that like team collaborative. Environment. You're. Not making, state changes, while I'm making state, changes. And. We're done and what we can see is that I don't have a TF. State. File here if. I go over to my storage account and refresh it boom. There it is I've got my terraformed. TF, state which. Is the state of that configuration, now. As long as I'm you know configuring, that backend every time I run that it's going to pull, state from, either. Storage, in this case. Compare. That state to what's actually been deployed and you know give me the opportunity to make the changes. So. That's pretty cool. Next. Let's look at backend and output. Alright. So inside of this directory here very same a very similar configuration I've, got a bunch of variables defined, I've. Got my main file, defined and I've got a resource group and a container instance, I, then. Have my back-end defined. Exactly like the last configuration I did, change the key name so I'm creating a you. Know an additional, state file for this configuration again. In this case this configuration, is all the dot TF files in this directory and, then. I've got an addition of this output, file, or. Output TF, and again this could be you, know this could be one single TF file if I wanted to do so and basically. I'm outputting the IP address, of the container instance, the resource group name and the. Resource group location. So. We'll go ahead and initialize the directory. Go. Ahead and run, it you, can. See they're acquiring state lock so it's creating that state file and then locking it so, that nobody else can make state changes to it. And there's. The follow right there you can actually see over here lease state it's leased. Alright so there's the output. But. As I stated earlier output is not just about putting you know presenting. That information to the terminal, we. Can do cool things with it particularly when we're using remote, state so. I'm going to jump to my next folder, here. Back. In datastore. So. Here we can see very similar configuration I've, got some variables, but, notice I'm not defining, anything for the resource group here I've, got. My back-end, defined. I've got, main, defined. But. Notice inside of this configuration I only have a container, instance, I don't have a resource, group here, but. I've got this data TF, file and so. What I'm doing here is I'm defining a remote, state, configuration, so, data. Or. I'm sorry I'm defining, a data, configuration, so data. Terraform. Remote state azure. RM. My. Back end is Azure RM and again I've got a storage account name a container, name and the, key of the, remote state, file. That I want to use to. Pull data from. Coming. Back over here in domain if we look at the container. Instance, the location, and resource group name you, can see here now I'm using data, dot, terraform, remote. State, azure. RM resource. Group, so. This is I'm, basically saying hey use this remote provider, or this this data provider, which. Happens to be remote, state and, I've. Got my remote state file in this address storage account. So. If we come over here look. At the resource groups. When. I deployed it this last one. The. Resource group name was. Output. Demo. So. If I come over here I've got one container, instance, an output demo.
Got. To initialize, and. So. Basically what I'm saying is I want to put this container instance in the, same resource group as the last demo and I'm getting the that can I'm getting the resource group name from the state that's stored in Azure. And, again, going back to that kind, of earlier example, things, like my, networking team takes, care of networking resources and I take care of these resources but, I need to get information about that networking stuff and pull it into my configurations. That's kind of the practical use case here. Refresh. This and there is my my, new, container instance, so I dropped that in that resource group without actually defining that resource group inside. Of my configuration I just pulled it I pulled, the name and location, from, remote State. Cool. So I've got one. Or two more slides and then a, final. Cool. Demo. So. You. Know I talked earlier I compared. And contrast you know manual deployments. Scripted. Deployments. And template based deployments, I want. To close out the session really just a really brief, discussion, on, opportunities. For process automation with terraform followed. By a quick demo and it's. Pretty simple you know. Once. We're using things like terraform, backends, environment, variables. Using. Kind of terraform, to its fullest, we. Can then integrate things like you know standard, like collaborative. Tools and other. Process, automation tools like and github or any, kind of get based workflow. Webhooks. Jenkins. Azure. Devops etc. To, really kind of build these. Infrastructure. As code or. Infrastructure. I like to think of them as infrastructure. As configuration. Pipelines. To. Really, attach some of you know what we've been doing here manually like running these commands, into. To, process, automation. So. I'm just gonna close really quick I'm not gonna get kind of into the details but just with a very quick demonstration of. Using, Azure, DevOps. To. Deploy. Or run terraform, configurations. All. Right. Let's. See, time. For us. All, right so let me go to github. So. In github I've just got a simple very simple repository. And I've. Got, just. What. We've been you know we've seen throughout the session I've got a bunch of terraform. Configuration, files and it's. Basically, the, same same. Example, I've got a resource group in an, azure container, instance. If, we look at. The. Variables file. I've. Got a container image, defined, that. Is Neil Peterson, /ne. Peters. Dot, v1 now. I can do it get clone and pull this down to my my system, and and modify. This and get commit and get push it back up I want, to save some time for some questions so, I'm just gonna kind, of cheat and do it all in this UI but. Uh what, we'll see is that I've already got this deployed so, if I come over here to Azure DevOps demo. There's. My container instance. If. I click on containers. You. Can see right here. There's. The name and there's the image being the. Image that's running inside of this container instance, well. Let's say it's time to update like hey we've got a new we've got a new image and we want to update, this, from v1. To v2. So. We go through my you, know. Process. So you know developing the container image pushing it up now, it's time to make the changes, I'm just, gonna keep this very simple. I'll. Go ahead and actually before doing so let's actually get. Azure. DevOps up and running. And. I'll show you exactly like, how simple this actually is, so. Within Azure. DevOps, I'm basically doing two things and it's the things we've already been doing I've basically, basically, essentially running terraform, in it and. Terraform. Apply now, I do have this - Auto approve, which. Removes. The need to type in yes like are you sure you want to do this yes, so. Nothing. You know there's nothing fancy going on here at all. So. I'll go ahead and edit my code. Commit. The changes. Come. Over to Azure DevOps. We. Can see that a new build has started I. Can. Actually trace, that build I've got all the output going on and whatnot and obviously, you know in a production environment you're gonna have pipelines set, up you're gonna have approvals, and you know ways to kind of make, sure that what you're doing is what you actually want to do. Boom. Come. Back over the azure portal. Come. Back into my resource group. There's. My container instance. Click. On containers, and. We. Can see that the, image, has been updated. To version 2, so. You, know everything, we've kind of talked about but in more of an automated process now. Terraform. Isn't really even a thing for me I'm actually just you know modifying.
The, Configuration. Committing, the code and in. Azure DevOps pipelines are taking care of the. Deployment of that for me. So. I've, got, about four minutes left here. That. Is the end of my session please, you know fill out your evaluations. And if. There are any questions you know feel free to walk, up to the mic or crowd. Up here yep go ahead. It's. Oh that's a good question actually so, so. In this can. Configuration. I. There. Are some requirements. Such as terraform, itself so. I had actually built, a container. Image. With. All the the tariffs of the, azure DevOps. Configurations. And the terraform, configurations. That I need to, run terraform, configurations, and so, my build agent is actually running in in this case was running in an azure container, instance itself yep. Yeah. So azure devops basically. Runs that on the container instance, and everything. Happens, from there. Good. Question. Awesome, thanks a lot for coming to my session I'll, be around on, the expo floor if you want to talk Thanks.
2018-10-25 19:20
