Azure auditing options for your custom reporting needs

Here’s the five options I’ve been able to find – so far – if you need fine-grained detail on your Azure subscription usage. (i.e. historically showing user access for security audits across multiple resource groups, etc.)

 

If you want a one-sentence recommendation – sorry I have to stick with “It depends”. I think you get great power with the OMS option (#2), but the PowerBI option (#3) is up and coming and very robust.

 
 

  • Option 1: Powershell Client for Azure RM. See the links below for more on this.
  • Option 2: Operational Insights
  • Option 3: Azure built in portal reporting
  • Option 4: PowerBI consuming the REST service. (See the links but this may very well be your best and most powerful option)
  • Option 5: Other tools consuming the ARM auditing APIs/SDK/CLI. There’s lots of log aggregation tool ranging from Excel to very sophisticated third party tooling that consumes the REST interface.

     

    In more detail:

     
     

    Option #1 – Powershell

    This was what we used two years ago. Nowadays, it seems like best practice is log aggregation–using Operations Management Service. That gives you the best level of customization and fine grained detail without having to take on PS scripting or consuming REST endpoints manually.

     
     

    Auditing reports using ARM Powershell, which in turn rests on the REST API we expose as part of the Azure resource manager. A Microsoft walkthrough of setup including deployment is here.

     
     

    There’s a good walkthrough on installing Powershell Client for Azure Resource Manager here. This blog goes through this in detail, including answers like ‘who accessed by subscription in the past 60 days”, “what access does a specific user have”, etc. We could extend this to show more detail points.

     
     

    There’s a walkthu on this blog of building out auditing reports. This blog uses ARM Powershell to come up with user list on subscriptions, modules used etc. And of course there’s third party products offering services in this space as well.

     

    The auditing APIs are evolving fast per my friends on the product team – there are some great third party tools out there that will provide this info. For you script based junkies – PS might be a great option. You can use PowerShell to view the Azure Activity Logs, showing all operations on the subscription and who did what. From here you can consume those API’s – fairly easily – and then you can crunch them into something useful.

     

    Start with the PS Commandlet Get-AzureRmLog:

    https://msdn.microsoft.com/en-us/library/mt603617.aspx

    https://azure.microsoft.com/en-us/documentation/articles/resource-group-audit/

     
     

     

    Option 2 – Operational Insights

    On #2 above, there’s an overview here of Operational Insights. A overview page on Log Analytics is here, documentation and FAQ is here, Not too much deep dive info on Operational Management Service (OMS) within Premier, but if you think this is a worthwhile option we can engage with a PFE and even build you out a pilot on it.  It can also now be connected directly to OMS (as well as Event Hubs and storage accounts). For the type of reporting you are talking about I think OMS would be the answer.

     
     

     
     

    Also worth pointing out that this is only activities carried out though ARM. If you want to see the audit records for changes to RDFE resources i.e. Classic Cloud services etc. then you still need to use the Operation Logs in the classic portal (or API). This caught me out recently trying to help a customer audit config changes to cloud services.

     
     

     
     

    Option #3 – Built in reporting in Azure

    Note that the audit data from Azure (ARM) is now available and searchable in the Azure Portal via the Activity Logs blade.

     
     

     
     

     
     

     
     

  • According to this article, there’s five different types of reporting available to subscription admins OOTB.
    • Anomaly reports – Contain sign in events that we found to be anomalous. Our goal is to make you aware of such activity and enable you to be able to make a determination about whether an event is suspicious.
    • Integrated Application reports – Provides insights into how cloud applications are being used in your organization. Azure Active Directory offers integration with thousands of cloud applications.
    • Error reports – Indicate errors that may occur when provisioning accounts to external applications.
    • User-specific reports – Display device/sign in activity data for a specific user.
    • Activity logs – Contain a record of all audited events within the last 24 hours, last 7 days, or last 30 days, as well as group activity changes, and password reset and registration activity.

     
     

    Option 4 – PowerBI

There’s a couple of slick ways to build out PowerBI reports direct from the REST endpoints. Some great references on this here.

https://azure.microsoft.com/en-us/blog/analyze-azure-audit-logs-in-powerbi-more/ – this goes through the Power BI Content Pack for Azure Audit Logs. There’s a secondary article right here with some snapshots. From this doc:

“In a nutshell, Azure Audit Logs is the go-to place to view all control plane events/logs from all Azure resources. It includes system and user generated events. You can also access this through the Azure Insights SDK, PowerShell, REST API and CLI. The logs are preserved for 90 days in Azure’s Event Logs store.”

Here’s the data you can gather:

  • Events by any particular resource over time
  • Which users perform what actions, how frequently and on what resources
  • Actions and events per subscription, resource group, region etc.
  • Azure Service Health (outages and maintenance) events that potentially impacted your resources
  • Alerts and AutoScale events by resource and time
  • Failures, success of deployments and registrations

 

Microsoft has further documentation explaining how you can access Azure Audit Logs in the Azure Portal.

 

Option 5 – Other options:

  • There’s advanced reporting available in Azure Active Directory as well. Azure Active Directory Premium. Advanced reports help you improve access security, respond to potential threats and get access to analytics on device access and application usage. There’s a walkthrough of this at this page.

 

 

I hope to add to this in the future with some great third party tooling we could recommend. Stay tuned!

 

Portland 2016 DevOps day – wow, thanks!

Guys, had SUCH a blast last Friday at the DevOps roadshow!

Here’s some pix. I really owe Monu Bambroo, Derrick Cawthorn and the amazing Donovan Brown for coming down and buying out their time to spread awareness of DevOps and the answers we have here at Microsoft for this sea change.

If you’re interested in more, give me a holler. We do have that workshop on “DevOps Fundamentals” that in three days goes through setting up a complete release pipeline – way cool!

 

Some link goodness for you:

  1. Donovan’s site: http://donovanbrown.com/ Search for DevOps. There’s something for everybody at this site. For example, here’s a post describing how he went about setting up a demo for a group in New Zealand using Docker, Ubuntu Linux, Visual Studio, Selenium, etc. Way cool! There’s another good link here for how Deployment Slots play into your DevOps pipeline, another on “how many vendors does it take to implement DevOps?”, triggering a rollback based on user feedback during a release, where Powershell DSC fits in.

 
 

  1. Dave Harrison’s site is here. www.driftboatdave.com. I’ve got some links here on “All Happy Families Are Alike“, “Devopoly“, “Cats and Dogs Living Together“, and “The Five Dysfunctions of DevOps“. These are lengthy but put together will give anyone a good overview of the Phoenix Project and Visible Ops.

 
 

  1. Last, may I recommend Channel 9? Here’s a 12 minute intro with Donovan Brown,  and an excellent three part series on Release Management – Part 1 (overview), Part 2 (RM architecture), and Part 3 (release pipelines). Outstanding, and will give you a nice overview of what we covered during DevOps Day in setting up Continuous Integration and build pipelines.

     

 

 

July 15th Premier Roadshow on DevOps – coming your way!

 

Way cool – at long last, we’re hosting a DevOps half day conference right here in Portland on July 15th!

 

The agenda below is set to change and we have some exciting guest speakers on their way. Get in touch with me and we will add you to the list of invitees!

 

Agenda:

  • Building a 3-Phase roadmap to sanity – and getting out of firefighting
  • Defining DevOps For YOUR Organization
  • Release Management Plain and Simple – Which Tool is Best?
  • Metrics Make It Happen – KPI’s You Can Use to Track Progress and Drive Success
  • Puppet, Chef, Octopus and Visual Studio – Better Together

 

As a movement, DevOps has now replaced Agile as the key factor in getting software builds out the door faster and safer. This workshop will help you define conditions of success for your organization and lay out a practical roadmap to change management. We’ll discuss features and advantages of leading DevOps tools and how to make sure your org culture and people can use these to best advantage to drive value and repeatability.

 

How to Register:

Contact:
Dave Harrison

Microsoft Premier

dharriso@microsoft.com

 

Note: This program is in high demand and registration is on a first-come-first-serve basis. You will be placed on a waitlist if the seminar is full.

Snacks and coffee / tea will be provided.

DevOps Roadshow coming to Portland!

Hey folks, more details coming – but we are going to have a half-day roadshow on DevOps coming to Portland on Friday, July 15th! We’re hoping for a few surprise presenters so the agenda is subject to change – but here’s a sneak peek.

Ping me to get an invite!

As a movement, DevOps has now replaced Agile as the key factor in getting software builds out the door faster and safer. This workshop will help you define conditions of success for your organization and lay out a practical roadmap to change management. We’ll discuss features and advantages of leading DevOps tools and how to make sure your org culture and people can use these to best advantage to drive value and repeatability.

Agenda:

  • Building a 3-Phase roadmap to sanity – and getting out of firefighting
  • Defining DevOps For YOUR Organization
  • Release Management Plain and Simple – Which Tool is Best?
  • Metrics Make It Happen – KPI’s You Can Use to Track Progress and Drive Success
  • Puppet, Chef, Octopus, Release Management and Visual Studio – Better Together

 

When/ Where:

Microsoft Pearl Office, 1414 NW Northrup St, Portland, OR 97209

Friday, July 15th, 9 am-Noon

Ok, we all love DevOps. But now what?

We commonly find that everyone – and I mean EVERYONE – is in favor of DevOps once they realize how great it is. Entire teams read through “Continuous Development” by Jez Humble or “The Phoenix Project” by Gene Kim and they are full of enthusiasm, ready to change their deployment processes so changes are safer and more repeatable. But then these teams have a “now what?” moment – we know we want to improve our processes, but where to start?

One of the cool things about DevOps is the lack of fuzziness – it is very, very tangible in terms of measuring ROI and tracking progress. For example, check out the very specific metrics you can use below that the thought leaders above have identified as being common traits of highly effective organizations:

  1. High service levels and availability (as measured by Mean Time To Repair – MTTR, Mean Time Between Failures or MTBF)
  2. High throughput of effective change (change success rate >99%)
  3. Tight collaboration between dev, Ops/IT, QA team, and security auditors
  4. Controls are visible, verifiable, regularly reported
  5. Low amount of unplanned work (<5% of time spent firefighting, compared to the average of 40%)
  6. Systems highly automated and hands-free
  7. Server to System Admins ratio 100:1 or greater (average is 15:1)

Those factors above are beautiful because they’re so specific, not subjective. You could – and should – publish these on a dashboard, showing your current state and tracking your maturity level improvement over time.

So, getting down to brass tacks, once we do a baseline and see where we measure up on those 7 key factors above, how do we get to “Phoenix Project” greatness?

You could tackle this in three stages, as follows:

Phase

Steps

Phase 1 – Assessment

Create a release management team

Institute weekly change management meetings

Begin gathering and publishing “7 power metrics” (above)

Inventory applications and systems, and identify business stakeholders

Phase 2 – Enforcement

Identify fragile artifacts (Martin Fowler’s infamous “snowflake servers“)

Document your policy and change window system by system with stakeholders

Remove access to all but authorized change managers

Electrify the fence with monitoring / active enforcement of policy

Phase 3 – Stabilization

Build a library of repeatable builds

Feed change info to first responders and trouble ticket system

Kaizen (improve and expand metrics gathering, feedback to stakeholders and management)

 

These phases aren’t strictly done in a series – there’ll be overlap, and its definitely a monumental undertaking. But, if you love the idea of change management and reducing all the wasted time and stress you spend in firefighting in your company, rest assured – DevOps isn’t just buzz and fluff, it’s tangible and measurable. And it’s a journey that – while it has no true ending – you’ll be very glad that you took. It’ll mean a happier relationship with your business partners and customers, less time tied down in reactive troubleshooting, and more time with your loved ones and families. What’s not to love?

P.S. if you enjoyed The Phoenix Project and want to read up more on your next steps, check out “Visible Ops“. This tiny little book is 100 pages of very specific, tangible steps you can take to inject some DevOps goodness into your own IT organization.