Azure DevOps Projects, from the amazing Donovan Brown.

Notes below from “Zero to DevOps” epic presentation by Donovan Brown showing what was behind the scenes: (note the first comment is about adding SSDT/SSIS as part of buildout, I would love this). This is a breakdown minute by minute of key points in his presentation – a little more detail than in my previous post on this. This pulls back the veil on the presentation he gave at PADNUG back in August of 2017 – showing there’s really no magic or months of tinkering happening behind the scenes.

  • If you are using Visual Studio – First go to Tools, Extensions and Updates – select Continuous Delivery Tools for Visual Studio. This allows you to r-click on the project and select Configure Continuous Delivery.
  • Create a new project in VSTS
  • Right click in VSTS and add a new project – in this case an Azure Resource Group.
  • Here you choose from a template – choose Web App.
    • Note all the cool template options. Docker host, Logic Apps, Linux VM’s – it’s all here. No need to reinvent the wheel!
    • VSTS will now generate all the tooling you need to work in Azure.
  • When this is done – check out website.json. These are all the resources you’ll need. On the left is how we’ll navigate. We can easily add or change parameters (see that json file) and destroy and recreate environments at will.
  • R-click in Visual Studio and select Configure Continuous Delivery. You’ll need to know where you are going to deploy to. Note his release project names – BikeSharing360, BikeSharing360D, …P, and ….Q.
  • 8:02 – Release definition – click on Releases tab. Template – click on browse, this is just the output – navigate window on the left. You can override the template parameters here and set it to a variable.
  • ARM templates – your Ops team can be on this in generating templates for Infrastructure as Code. A resource group, inside of it is a web app. Click on it in the portal – Continuous Delivery. This is a lot more mature than just doing a Git repo push. (note the difference between onprem TFS and VSTS – VSTS is updated every 3 wks, TFS every 3 months.)
  • 16:31 – YoTeam, Donovan’s pet project. It’s script based as Jez Humble seems to favor.
  • 20:55 – Handles Grunt, Bower with aplomb. Donovan typically uses Node, Mocha, Signon, Istanbul. This gives him good code coverage. “If you can do it in a CLI, I can put it in your build pipeline. You can even wrap it and add it to the Extensions library for everyone to use.”
  • 21:32 – Approval gates.
    • “note I said – automate everything you can. That’s a big difference from ‘Automate everything!’
  • 26:30 – Build agents – these can be installed behind your firewall. Code, resources never see the public internet – all behind the firewall.
    • Stakeholder licensing – does not count against 5 free teammates. That combined with free licensing for MSDN subscribers makes VSTS deployments a steal.
    • “No more ‘VSTS is in the cloud, it will only deploy to the cloud’ thinking”
    • “No more “We’re not a Microsoft/.NET shop”, or “All or nothing” – you can fold in Jenkins or OctopusDeploy or whatever. Use us to pay the integration tax.”

Website resources:

  1. first official announcement    
  2. Hub on Azure Deploy projects – https://docs.microsoft.com/en-us/vsts/deploy-azure/, and an Ignite overview on the topic.
  3. https://docs.microsoft.com/en-us/vsts/build-release/apps/cd/azure/aspnet-core-to-azure-webapp?tabs=vsts#create-webapp-portal
  4. https://docs.microsoft.com/en-us/vsts/build-release/actions/define-multistage-release-process?toc=/vsts/deploy-azure/toc.json&bc=/vsts/deploy-azure/breadcrumb/toc.json

 

Super cool, need to look into Auto Hotkey for my demos. Thanks Donovan!

Advertisements

Walkthrough notes in creating deployments and the Azure Deploy Project

Recently I’ve been asked to do some complete demos of building out complete release pipelines similar to what Donovan and company have been doing for at least a year now. My craptop has been bottoming out lately and I’ve sworn to “walk the walk” when it comes to making the leap from Visual Studio local on my box to editing/pushing out code using Visual Studio Team Services (VSTS). As VSTS has changed quite a bit since I last looked at this, I thought I’d write up my walkthrough notes so you can do it yourself. Trust me – setting up CI/CD is now LAUGHABLY easy. There’s really no excuse not to try it with your new application.

If you want more information on ARM templates, setting up release definitions, build agents etc – check out the “Zero to DevOps” epic presentation by Donovan Brown showing what was behind the scenes. Note the first comment is about adding SSDT/SSIS as part of the buildout as a suggested feature, I would love this!

In Brief

  • First lets set up some code to import.
  • Create four websites in the Azure portal you want to point to. Let’s create a D, Q, and P set of sites.
  • Now let’s set up a build.
    • Set up build – ASP.NET. Call it “XXX_CI”, select repository
    • Click on the Trigger tab, and select “Enable CI”
    • Click “Save and Queue”
  • Associate this build with a release:
    • Release tab – create a new release definition. When it asks you for the target environment name, give it a name – “Dev” – Azure App Service Deployment template, and select Apply.
    • Click on Artifacts – select your project, and the “XXX_CI” build definition.
    • Enable CI by clicking on the lightning bolt on the artifact, top right.
    • On the Artifacts object – select the “dev” environment. Select the dropdown on the subscription to pull in your Azure subscription. Here we are going to point ot “Phoenix360D”, our destination dev website.
    • Then we edit the index.cshtml file and add some nonsense verbiage. Pull it up in the site – and voila! Any changes we are making flow instantly through to dev where they can be tested.

See below for notes on setting up multiple deployments and creating a DevOps Project.

Full Walkthrough Notes

Note there’s not one original step in here, I just walked thru the steps in this doc like a good, obedient zombie.

Creating Your Environments

First go into the Azure portal     

  1. Log onto portal.azure.com
  2. Create three new websites by clicking New -> Web + Mobile -> Web App.

According to the notes here – https://docs.microsoft.com/en-us/azure/app-service/azure-web-sites-web-hosting-plans-in-depth-overview – you only need to create a new app service plan if a given app is resource intensive or you need to scale it independently. That’s not the case here – we can use one app service plan for all four environments (XXXAPPNAME + D, Q, P). In contrast, the idea behind resource groups are – you update them as a group. They share the same lifecycle, permissions, and policies – you can update security to this batch of resources as a group for example. So we’ll be creating one app service plan, four diff resource groups. We’ll create three websites – see below for “Phoenix360D” – with the appendix -D, Q, P – dev, QA, production.

Depending on current demands Azure should spin each of these up in a few minutes. Now we’re good to go, all 4 environments have been spun up and are running on Azure. And we have a build running successfully.

 

Getting Our Build Started – Single Path

Next, we need some code to work with. If you don’t have your own, no worries, we can give you a very nice working sample complete with test scripts.

  1. Log onto VSTS and click on your Code tab. Import using https://github.com/adventworks/aspnet4-sample – see the nice screenshot below:

  2. Once this is done – you should be on the Code tab – select the “Setup build”button on the right. Select “ASP.NET (PREVIEW)”, and select APPLY.

     

Side note, check out all the steps it stubs out for you below on the left. Whoa!

  1. Give the resulting template a good name– I chose Phoenix360_CI – and select your repository you just created. Here I’m using a Hosted 2017 build agent but you could also use your onprem TFS2017 build agent if you so desire.
  2. And, last, I select the repository we just imported:

  3. From here I can almost see the end of the tunnel – click on Triggers and enable the trigger for continuous integration. (Note you can also set scheduled builds at a particular time of day on this tab.)

  4. Click on Save & Queue, top right. Enter some notes on your commit, and on the popup window click Save & Queue again.

  1. If you’ve done this right – you’ll see the following in your build definition:

 

  1. Who, that build URL there is just begging to be clicked on. Let’s click on this:

  1. To test if this is working – go into Code again and make a hand edit to a web.config file, in the header. If we’ve done this right, we should be seeing a build kick off after our commit of this change:

Click on Build and Release tab. Sure enough, our code commit triggered a build:

  1. This is really quite nice – click on the latest build, and select the Test tab for example. It shows us the tests and the run length:

     

     

     

 

  1. Click on the Release tab and add a new release definition. There’s a few templates to choose from here but it’s definitely easy to start with a precreated template vs rolling your own. Let’s click on “Azure App Service Deployment” and select Apply.

  1. Don’t get overwhelmed – just click on the Add Artifact option. Enter in the following values – the Build Definition you created earlier. Note the different version options as well in the dropdown:

  1. On the Artifacts node you just created – left side – notice that little lightning bolt on the top right? That’s our continuous integration trigger. Let’s click on this and make sure CI is all set up:

     

    And then click again on that Artifacts node on the right, and set up your environment including the destination endpoint:

     

    Create a new release – as you see below – and save it to the default folder. We’re golden!

     

     

    Does it work? Let’s go into the code view and make a change. It should populate out to dev:

     

     

     

     

     

     

     

Setting Up Multiple Deployment Paths

Continuing with this wiki:

 

See below. Clone your dev item – and set it up so the pre-event trigger (lighting bolt, left side) is set to “After environment”. This is also where you can set up approvers and manual stage gates.

 

 

Clicking on the pre deployment conditions lets me set the deployment to trigger after the environment is ready or based on a release trigger (i.e. a simultaneous rollout to DEV/QA). You could also set your production rollouts to a less busy time of day for example.

Then I go into each task for the new cloned environments above and change the deployment pointer to QA (or Prod).

Let’s get fancy and change the deployment to prod to be manual.

Now when I create a release – look how nice this is:

 

And sure enough when it hits prod I get this nice little alert that I need to review the changes and approve a move to prod.

 

Sure enough, now any changes to my source control kicks off a full set of releases out to all 3 environments. Noice!!!!

 

 

 

 

 

DevOps Projects

Log in to VSTS.

Create a new DevOps Project. New (top left), Everything, and filter by “devops”. You should see the DevOps Project below appear.

Let’s select .NET below. But we could import our app or use a ASP.NET site based on PHP, NodeJs, Java, etc.

 

On the next screen choose either ASP.NET or ASP.NET Core. Select Web App in the next window – it’s the only option as yet. Lastly choose your existing MSDN subscription – assuming you have one – and a new project name.

 

I’ll next see a “Deployment in Progress” notice in the taskbar. Super cool!

… and at last I get this shininess.

 

There’s no magic going on here. You can browse to the Build definition and inspect what its doing – and then click on the Dev release and edit the properties. See? It’s exactly what we created before, manually.

Really I’m very glad that I took the time to do this manually first. It really gives me more of a comfort level when it comes to setting up release pipelines manually.

 

Dead ends and miscellany

An annoying issue was in the Artifacts, when I’m trying to point to the correct environment, it kept blowing up with “Couldn’t authorize to AAD. If popup blocker is enabled in the browser, disable it and retry again.” I tried changing adblocker in Chrome but that didn’t fix anything; same with Edge. But, classic IE got me a little further. This doc gave two options – https://docs.microsoft.com/en-us/vsts/build-release/actions/azure-rm-endpoint. I tried to log in but the “User May Add Integrated Applications” under the classic portal was already set to Yes. Tried again in Chrome, went to Release and tried adding a new Azure Resource Manager Service Endpoint. Turns out that wasn’t what it needed anyway.

I also had some subscription issues, where my default directory needed to be changed – that was really killing this walkthrough. Good news was – I submitted a ticket, Sev A, and got a call back from a very competent subscriptions helpdesk person in about two hours. Excellent. Really, I was quite impressed, it totally fixed a very longstanding issue.

 

Helpful sites to do this yourself

Connect KeyNote with Brian Harry. Devops Projects – awesome scaffolding for your release management project!

Still going through videos from Connect, there’s a lot of stuff to wade through! Definitely enjoyed Brian Harry’s keynote address – especially the awesome Abel Wang helping out as copresenter. Here’s my notes.

The takeaway stuff is this –

  1. you can use Azure DevOps Projects to create a fully functional CI/CD pipeline as a starting point to any project, then extend it. This is way cool and I can’t believe it hasn’t existed before. The release visualizations are definitely top knotch.
  2. YAML is now supported. (Several of my customers have been asking about this!)
  3. There’s some real goodies here about how Microsoft handles its releases – hint its not Dev-> Prod with one click, using scale units.
  4. Automated gateways are now possible in VSTS. This is definitely a huge win…

In more detail:

  • (roughly 1 min in) – Food for thought: “DevOps is all about shortening feedback lops… automated deployments are often the first thing we think about.” There’s a lot of plumbing though – it can be daunting.
  • (minutes into broadcast) 4:30 – Azure DevOps Projects- easily create a full end to end RM pipeline, using Node.Js, .Net, Java, PHP, Python.
  • 5:59 – dashboarding – click on links for code / build/dev pipeline. To customize, clone it onto your HD – delete all files, copy in your code, then push it back to VSTS using Git. Easy!

  • 13:44 – YAML support now included in our CI system
  • 14:41 – No one actually pushes a button and code goes out the door to production – “I call that the fastest way of deploying bugs to your customers.” At Microsoft we have 15-20 different scale units (subsets of customers). We use Release Management to gradually roll these out across environments. First we roll out to one scale unit, watch twitter for sentiment downturns, check feedback, use it – etc. Then we wait 24 hours before deploying to the next ring. That’s responsible CI/CD. If we have a blocking bug – we pull the cord and roll back.
  • 23:00 – demo of build agents running natively on all 3 platforms – Win/Linux/Mac. You could use one release to all 3 environments if you wanted to. I thought this was amazing:

  • 26:18 – automated gate creation. These are automatically created post deployment monitors – using Azure monitor, functions, REST API, and service bus to stress test/check your new system’s health.

  • 27:48 – creating a YAML build
  • 32:27 – fork into private repo vs a branch.

 

For a full writeup including a new walkthrough on Azure DevOps Projects, click here. There’s a quickstart here.

Full list of Connect DevOps vids and my writeups: (this will grow)

 

New VSTS features coming up – hawt fresh Agile changes y’all!

Connect() 2017 is all done and wrapped up for the season. If you weren’t able to make it – as I wasn’t (sniffle) – all the content is available on demand. Click here for an overall list of DevOps focused talks.

I wanted to post a little about one of the great webcasts I viewed this morning, Agile Project Management with VSTS, with Aaron Bjork and Sondra Batbold. This is a really great walkthrough of the full capabilities – including some hawt new features – coming up in Visual Studio Team Services (VSTS). Below are the key features I noticed – broken down by where they appear in the webcast so you can skip to the good stuff.

  • 5:09 – Notice the custom Kanban board, with columns for Backlog | Dev Design | Implementing | Code Review and Verify | Closed. There’s a definition of done showing the team’s standards on the info icon – in this case “doing” means fully designed and implementation started; “done” means unit tests written, fx tests updated, and its ready for code review. Nice as well to show the WIP limit on the top right. (Side comment, I love Kanban and how it helps us avoid the myth of multitasking by limiting our Work in Progress. I actually use this at home so I don’t get overwhelmed with my chores around the farm! I do feel, very strongly, that Kanban should be the default starting place and maybe the endpoint for 90% of the teams out there struggling with their Agile implementation.)

  • 6:40 – using swimlanes to separate out important items. (Settings icon, Board > Columns)
  • 8:05 – Setting a styling rule to have high priority bugs turn red (for example). You can also add tags, if the priority is high enough – and highlight in pink.

  • 10:11 – Click on lower left corner of board to add tasks
  • 14:14 – “my activity” query for busy project managers off the Work Items hub.
  • 14:42 – Scrum team setup with 1 week sprints. Notice the division of work here, from New | Next Sprint | Current Sprint | End Game | Closed.


  • 17:02 – Most scrum teams focus on velocity – the forecasting feature.

  • 19:38 – Adding a column to the backlog (customizing display)
  • 20:59 – Capacity planning. Note what it says at 21:34 – “Note this feature is for you and your scrum team, not for management to look down on you. This allows you to make a strong commitment to the upcoming sprint.”

  • 22:15 – task board and burndown chart you can use on a monitor in your daily standups (DSU’s)
  • 23:49 – filter by person (to show your work only for example, I use this all the time)
  • 24:15 – dashboards. Check out the list of widgets in this nice display –
    • current sprint
    • burndown
    • cycle time (closed / new / active) – i.e. “how long it taking us to start working on an item”? this is a key pain point mentioned in the Phoenix Project.
    • Stories by state
    • Team velocity – in this example it shows the team improving in their completion rate by doing better planning.
    • KPI’s – including Open User Stories, Active Bugs, Active Tasks, Ready for Testing, Completed User Stories

  • 25:38 – Very configurable new burndown chart vs the OOTB widget.
  • 28:31 – Delivery Plans – a new feature showing work across all teams. In this case we’ve got three teams working on different schedules. You can expand this to dig into work being done by a specific team, and zoom in/out.
  • 31:29 – Plans – You could put a specific milestone – say a release date – on the chart.

  • 32:19 – How does Microsoft use delivery plans with their product teams? In the VSTS case, the leads for all 4 teams meet regularly. They talk about what’s currently going on, what’s 3 weeks out. There’s a lot of “A-Ha!” moments here as cross dependencies get exposed. (Pro tip – use “T” to show succinct view)
  • 33:32 – new Wiki feature. (Could this take the place of an emailed retrospective?) You could add a new sub page, etc. Very customizable, I like it. Use a pound (#) to add a reference to another work item.

  • 35:53 – Add a new work item type to a custom template inherited from the standard Agile template. In this sample they force people to add a custom color and a icon to a new work item to visually differentiate it from others. (I’m questioning this one, does this really add value?)
  • 38:43 – Adding a “followup owner” so code reviews are enforced.
  • 40:30 – Queries are simplified and redesigned
  • 45:00 – Customizing the dashboard, in this case show a different color if WIP is excessive.
  • 47:15 – I love this part – Extensions. There’s a lot of custom extensions for builds, burndowns, etc. They walk through two paid extensions, one for the Backlog Essentials (quick in-place edits of a work item from the list, why isn’t this standard??!) and TimeTracker (for orgs that want to report/track time on dev hours) These are all available from the shopping cart icon, top right in VSTS. Note you need to add the Analytics extension to really kick up your burndown chart’s capabilities, see Greg Boer’s recent presentation on Channel9 including PowerBI features on Channel9.

 

  • 51:12 – Q&A:
    • Can we display a burndown chart across projects? (not yet, but soon) Note the comment at 54:13 – “I will tell you – we recommend one cadence to rule them all. We run on a 3 week cadence for our 700 people. It adds so much simplicity and clarity when we’re talking about dates.”
    • View Only (vs modify) permissions yet? (that’s coming also, we are working on joining multiple accounts together so we can view on an org level). Note on permissions, MSFT uses Area Path permissions for security to hide work on sensitive projects (a la HoloLens)
    • Hey, there’s lots of clutter on my PBI’s. Can we clean this up? (We’re working on a Personal view so you can pin only the fields a particular person is working on.)

Anyway that’s a lot of content for me to go through and think about. Should keep me busy for the next week or so as work on my book progresses!

Other Connect sessions I will be checking out –

General:

Database

Containers

Release Management

Source Control

Testing

 

The Equifax leak, and what you need to do about it.

Perhaps you are – let’s face it you likely ARE – impacted by the leaked consumer information by Equifax. Or the fact that the contact information they give us goes to a bogus number with no viable help offered. And that if you go to their site for remediation you are opting out of arbitration.

One article said – “In retrospect, we find it surprising that it wasn’t multi-trillion lawsuit in light of the galactic stupidity exhibited by a company whose server apparently had zero firewalls from the internet and where any hacker could get access to the most confidential information available….” One commentator put it, “In retrospect it seems like a really dumb idea to give three random companies access to the entire financial records of every American.”

Let’s forget about what a hellhole these huge companies send you to with their “customer service” lines, which are very well documented, or how evil it is that the executives sold their stock in Equifax before news of this leak – which was deliberately kept secret – got out.

If you want to lock down or freeze your credit – and that seems to be the consensus recommendation out there – you can. Below are the four agencies you’ll need to get in touch with. (I put site links in there even for hollow/404 sites just in case they become available later. Right now at least a few are not functioning well, likely due to high call volume and general jerkiness.)

Mail These Two:

  1. Experian – https://lnkd.in/gxuAehg (note this didn’t work, I had to send it in via mail)
  2. Equifax – https://lnkd.in/gvBWrEq (didn’t work, I tried 1-800-684-1111 and got a busy signal. 1-888-766-0008, “system error”. 866-447-7559 – 1 – this is just a receptionist that will give you the Equifax info. In short, mail it in. )

Call This One:

  1. TransUnion – https://lnkd.in/gCyFZP9 (didn’t work, had to call in to 1-800-680-7289 and go thru steps for credit freeze)

Yay, A Functioning Website!

  1. Innovis – https://lnkd.in/gFmqtVb (worked, free)

 
 

You’ll note, two of the four above required mailings. That’s actually OK for us as we want as much documentation as possible for safety and/or a big ol’ class action lawsuit. Be particularly careful with TransUnion as they’ll want you to sign up for their service which – surprise! – will charge you $19 a month for eternity.

Hope this helped. Now I have to cancel my TransUnion “service” (which I never asked for) thru my credit card, manually. I also went to this site to request a free credit report (you get one annually) from 3 credit agencies.

I need to send another letter to Equifax, on the opt out. I want to sue these guys, VERY badly.

Experian Form Letter – Opt Out

Equifax Consumer Services LLC, Attn.: Arbitration Opt-Out
P.O. Box 105496
Atlanta, GA 30348

The letter needs to include your name, address, and Equifax User ID, as well as “a clear statement that you do not wish to resolve disputes with Equifax through arbitration.”

 

Experian Form Letter – Credit Freeze

Here’s a form letter you can use for Experian:

Experian Security Freeze

P.O. Box 9554

Allen, TX 75013.

 

(date)

 

Please put a security freeze on my information.

 

 

Information:

  • Full Name: ____________________
  • SSN: ________________
  • DOB: ________________
  • Current Address: ________________
  • Previous Address: ________________ (if less than 2 yrs above)

 

Enclosed: $10 payment (if in Oregon), copy of drivers license, copy of utility bill / bank statement

 

Equifax Form Letter – Credit Freeze

Equifax Security Freeze
P.O. Box 105788
Atlanta, Georgia  30348

(date)

Please freeze my credit.

  • Full Name: ____________________
  • SSN: ________________
  • DOB: ________________
  • Current Address: ________________
  • Previous Address: ________________ (if less than 2 yrs above)

 

Enclosed: $10 payment (if in Oregon), copy of drivers license, copy of utility bill / bank statement

For the amount you’ll need to pay, see this site: What are the security freeze fees in my state? 

See “Acceptable Forms of Identification for Verification“.