AWS re:Invent Day 4

Poolside lunch at Mirage

Yet another incredibly long day - breakfast @7:00am, keynote by Andy Jassy, a breakout at 11:00am at the Venetian, a very fast pit stop to try to shovel some lunch before another 1:00pm breakout session at the Mirage, followed by sessions until 6:30pm.  Pub Crawl with 30,000 other re:Invent thirsty geeks to 7:30pm.

Keynote - Andy Jassy 

10,000 people jammed the Sands Expo to see AWS President Andy Jassy give Wednesday's Keynote.  Along with several announcements of new features a few AWS customers (FINRA, McDonalds, Italian electric company, ENEL, Workday) told their cloud migration stories.  Anyone that thinks that data centers aren't dead isn't paying attention.

Some announcements:

• New instance types across all families 
• IoT Platform (Greengrass) being incorporated into IoT devices
• Aurora for Postgres!
• Snowmobile
• ...and a lot more

Andy Jassy's job is to promote AWS and make the case for the cloud by talking about the AWS roadmap, presenting customer stories and defending AWS from Larry Ellison's attacks (a must read).  He's pretty good at it!

Other than the McDonalds's story, which sounds impressive - but really they should spend more time making their food better, not their technology -  the keynote kept my attention for the entire 2+ hours.  Especially when they drove the new Amazon Snowmobile Exabtye transport system onto the stage.  Snowmobile allows really, really, really big data producers to send exabytes of data to the cloud, literally via a truck.

Become an AWS IAM Policy Ninja in 60 Minutes or Less

I won't spend too much time on this talk.  It was ok and had some interesting tips for policy wonks.

Takeaways:

• Policies are powerful and you need to be pretty good at them if you want to be a cloud sysop or at least aware of them if you are a developer because they will control what you can and can't do if you are not an administrator
• Policy syntax is not always intuitive and clearly has an air of hackery around it - looks like AWS has added additional arms, legs and tentacles to overcome shortcomings (NotAction, StringEqualsIfExists). Honestly, anyone ever hear of PCRE?
• Version key required if using variables (${username} - Read More
• JSON is unwieldy for this purpose and someone must have a really cool tool by now to stop this madness, if not there is a market for one
• Speaker knew him some policy but was a Windoze guy.  He could have made his life a lot easier by using jq or some other JSON CLI parser.  His automated PowerShell script that automatically brings up Notepad was sad.  I felt bad for him, but not really.  Here's a nickel kid, get yourself a better computer.

Getting to Ground Truth with Amazon Mechanical Turk

Very interesting talk about using MT for creating training sets for machine learning.  Specifically the speaker was part of the team that created the new AWS service called Rekognition that does image recognition.

Considerations for creating training sets for Machine Learning applications:

• Design your "golden set" carefully 
• Use majority based voting - you may have to alter the quorom, i.e. 3 of 5, 2 of 3, etc.
• Worker evaluation and training are important
• Present hints for workers
• Question design is important - segmentation - more choices though can obfuscate and confuse workers
• Workers do not fatigue over time

I think MT is going to have some interesting contributions to AI moving forward.

Cloud Monitoring - Understanding Preparing and Troubleshooting Dynamic Apps on AWS

Did I mention serverless yet? This conference always exposes me to new terms, TLAs, geek phraseology and of course ideas.  Apparently we now have Dynamic Cloud which is code for serverless.

Dynamic Cloud being, I guess the extension of the metaphor to treat servers as cattle not pets - so we should now treat cattle like ground beef?  Someone will steal that from me I'm sure!

Update: The new metaphor is "..there are no cattle, just the herd..", although  I think though we might want to just say there are no cattle, just hamburger meat.

The title of this talk was a real tease, but really offered no insight other than to define the terms and the problems without really offering any patterns or best practices.

Monitoring serverless apps is more akin to monitoring infrastructure because of the ephemeral nature of event processing...which is all I got out of the guy's talk.  The rest of the audience seemed stunned when it ended abruptly, with a thud.  No amazing insights, kind of a waste of a mile long walk back from the Mirage.

Develop Your Migration Toolkit

Mostly a demo that showed the migration of a supposed on-prem Gogs (self-hosted Git server) to AWS using various migration tools.  The end result was a highly available stack in the cloud.  That is, the Gogs server was replicated across two AZs and used a Multi-AZ RDS instance.  Standard pattern.

Automation scripts were written using CloudFormation, Ansible and some COTS migration tools.  While I tried to keep an open mind and understood the demo was showing a pipeline that could potentially be replicated and tweaked to migrate other server configurations, it begged the question of whether anyone would really take the approach of automating a potentially one-time migration activity unless is was part of a larger automation project and even then it seemed pretty specific.

If you are viewing your target servers as black boxes to begin with in terms of your applications, a server migration isn't really relevant - you're really migrating applications to a new platform and that task may be infinitely easier to accomplish, especially if you applications are decoupled from a lot of server (instance) dependencies.  They also discussed migration tools that allow you to replicate your VMware instances to AMIs.  Again, if you're already able to re-platform your apps with some automation on new server instances in some automated or semi-automated fashion it might make more sense to re-platform, not migrate.  Seems to me that some of these tools are going to have a short shelf life.

They also discussed Amazon's Cloud Adoption Framework which is absolutely worth a read if you are part of a team or leading a cloud migration team.

Architecting Security and Governance Across a Multi-Account Strategy

This talk highlights where the industry is on the cloud migration trail.  Talks are no longer about pilots or cloud migration dreams.  AWS and customers are driving new patterns and exposing well trodden paths.  The pioneers have laid down the trails.

Thomas-Reuters shared how they rationalized their AWS cloud usage with a multi-account strategy.  In other words, big companies, sometimes in conjunction with AWS teams, are helping to create new frameworks, blueprints, patterns and best practices.  We are learning about the pioneers journey and it is just a matter of a few years before we will look back and realize that 2016 is when the wagon trains started headed west in earnest along these trails.

So, how many accounts should you be using?

Talk presented a blue print for creating an account footprint and a action plan with a check list of things to consider and perform.

• An AWS account is a security boundary, that's a good thing!  You can protect environments, business units and dev teams from one another.
• Highest level of resource isolation - same as two distinct Amazon customers.
• Billing separation (or consolidation)

OTOH, a single account easy to manage - single policies (IAM), single billing.

Why one isn't enough?

Some use cases:

• If you have more than 1 development team and only had 1 account and you want user isolation you will need complex IAM policies.
• You may want resource isolation - prevent networking access, resource access/creation or even VPC peering require IAM policies.
• Teams may have different development processes and deployment pipelines?
• Billing - often driving force for a multi-account setup  Need extensive tagging and religious adherence to tagging (enforced by automation scripts that terminate instances or create notification when instances are found that are not tagged according to standards).  Even with multiple accounts tagging is a good thing.
• Forecasting spend - having billing separation allows IT managers to ingest historical information on a more granular level in order to forecast future spend along line of business or IT functional (dev/test/ops) lines.

Multiple Accounts (Pros)

• Complete isolation
• Smaller blast radius
• Simplified billing per account

Multiple Accounts (Cons)

• Setup/management overhead
• cross account security policies more complex
• federation required between accounts or between accounts and corporate identity provider
• VPC peering can be complex, but powerfully used can create isolation and cooperation zones

Some additional thoughts:

• You'll want automated setup of accounts
• You'll want your process of setup to scale
• Self service for teams to whatever extent your policies can allow 
• Need to provide guardrails while enabling innovation
• Accounts need to be auditable (CloudTrail across all accounts and regions!)

Blueprint:

• 1 Master billing account (no resources)
• Security account
• Shared Services account
• Sandbox
• Non-prod
• Prod
• Other

Other Notes:

• Root account  - requires MFA
• Turn on CloudTrail in all regions
• Map current enterprise roles - federation (LDAP)
• E-Mail account names alias to 1 actual e-mail

In a nutshell, multi-account setup is essential for any company bigger than 1 person running a WordPress site if you want to provide a sane, secure, manageable IT infrastructure in the cloud.