I recently used AWS DataSync as part of a lab I was building. These are my notes for using DataSync to replicate an Amazon Elastic File System (EFS) share from one region to another.
AWS DataSync is a managed service that enables replication of data between AWS services and from on-prem to AWS. It automates the scheduling of transfer activities, validates copied data, and uses a purpose-built network protocol and multi-threaded architecture to achieve very high efficiency on the wire.
The use case I needed to tackle was replicating an Amazon EFS share in one region to an EFS share in a different region (a one-way replication). (DataSync can also connect to Amazon S3 and Amazon FSx for Windows File Server)
Consider for a moment that you have an application running on a server that needs to push some data out to multiple consumers and that every consumer needs the same copy of the data at the same time. The canonical example is live video. Live audio and stock market data are also common examples. At the re:Invent conference in 2019, AWS announced support for multicast routing in AWS Virtual Private Cloud (VPC). This blog post will provide a walkthrough of configuring and verifying multicast routing in a VPC.
There can be times when you're working on the AWS Cloud where you need to grant limited access to your account to a third-party. For example:
- A contractor or a specialist needs to perform some work on your behalf
- You're having AWS Professional Services or a partner from the Amazon Partner Network do some work in your account
- You're conducting a pilot with AWS and you want your friendly neighborhood Solutions Architect to review something
In each of these cases you likely want to grant the permissions the third-party needs but no more. In other words, no granting of
AdministratorAccess policies because it's easy and just works. Instead, adherence to the principle of least privilege.
This post will describe two methods—IAM users and IAM roles—for proving limited access to third-parties.
Here's a simple scenario: you have some Virtual Machines (VMs) in your on-premises environment, likely in VMware vSphere or Microsoft Hyper-V. You want to either fully migrate some or all of those VMs to the AWS Cloud or you want to copy a gold image to the AWS Cloud so you can launch compute instances from that image. Simple enough.
Now, how do you do it?
Can you just export an OVA of the VM, copy it up, and then boot it? Can you somehow import the VMDK files that hold the VM's virtual drive contents? Regardless the eventual method, how do you do it at scale for dozens or hundreds of VMs? And lastly, how do you orchestrate the process so that VMs belonging to an application stack are brought over together, as a unit?
Following on the heels of my previous post, Five Functional Facts about AWS Identity and Access Management, I wanted to dive into a separate, yet related way of enforcing access policies in AWS: Service Control Policies (SCPs).
SCPs and IAM policies look very similar—both being JSON documents with the same sort of syntax—and it would be easy to mistake one for the other. However, they are used in different contexts and for different purposes. In this post, I'll explain the context where SCPs are used and why they are used (and even why you'd use SCPs and IAM policies together).
Read on, dear reader!
There are roughly a GAJILLION articles, blogs, and documents out there that explain how to setup Amazon CloudFront to work with WordPress.
Most of them are wrong in one or more ways.
This post is part of an open-ended series I'm writing where I take a specific protocol, app, or whatever-I-feel-like and focus on five functional aspects of that thing in order to expose some of how that thing really works.
The topic in this post is the AWS Identity and Access Management (IAM) service. The IAM service holds a unique position within AWS: it doesn't get the attention that the machine learning or AI services get, and doesn't come to mind when buzzwords like "serverless" or "containers" are brought up, yet it's used by-or should be used by-every single AWS customer (and if you're not using it, you're not following best practice, tsk, tsk) so it's worthwhile to take the time to really get to know this service.
Given that my technical background is largely in the networking space (exhibit A, exhibit B, exhibit C (CIE)), one of the first things I tried to wrap my head around when being introduced to AWS is how networking works in the AWS cloud.
What I attempted to do was build a mental model by relating cloud networking constructs such as Virtual Private Cloud (VPC), subnets, and routing tables to on-prem, physical networking constructs. This worked pretty well but I did get tripped up at times because some of these constructs don't map exactly one-for-one.
This post will explain the mental model I used while also calling attention to the elements or behaviors that don't map exactly between on-prem and AWS.
In a previous post, I reviewed what a public subnet and Internet Gateway (IGW) are and that they allowed outbound and _in_bound connectivity to instances (ie, virtual machines) running in the AWS cloud.
If you're the least bit security conscious, your reaction might be, "No way! I can't have my instances sitting right on the Internet without any protection".
Fear not, reader. This post will explain the mechanisms that the Amazon Virtual Private Cloud (VPC) affords you to protect your instances.