Quickwork – SSM Patch Manager

How QUICKWORK uses AWS Systems Manager for instance patching in AWS Cloud

Quickwork provides an all-in-one platform with the tools and services you need to build powerful & scalable workflows, integrations, serverless APIs, conversational experiences, and a lot more!

Quickwork supports all types of events used in automated workflows, such as real-time, polling, scheduled, and manual. This unique capability enables building a wide range of workflows across industry verticals.

THE CLIENT’S CHALLENGE

The clients infrastructure and the operating systems used need constant up-gradation and patching for better security and make the infrastructure compliant with the certifications. The client is looking for a system to check the availability of patches, and then apply them on a regular basis, for the servers and compliance of the instances.

ARCHITECTURE

SSM Deployment

The multiple components of the infrastructure need to be patched for security and compliance. Following process is used for the implementation:

Pre-requisites:
1) System manager should be able to manage the instance. This can be seen on the fleet manager. If our instance ping status is showing “online” in the fleet manager of ssm, it means that the instance is managed by the system manager. To make the instances being managed in the fleet manager follow the below steps.

1. We need to have an instance role which provides ssm service access to aws ec2 instances, this role does nothing but provides permissions for communication between instances and the Systems Manager API. (This role needs to have “AmazonSSMManagedInstanceCore” policy attached to it)
2. All the instances that we want to manage using aws system manager, should have aws ssm agent installed in it.
3. Since this agent needs to talk to aws ssm api we need to make sure the ssm api is accessible from the instances, the recommended approach for this would be to setup an aws ssm endpoint (interface), allow port 443 on the endpoint security group from all the desired instances.
4. The above  points will ensure that instances are managed by aws systems manager, the same can be verified in the “Fleet Manager” option in aws system manager it will list down all the instances managed by aws system manager.

2) Instances should be able to reach the internet for checking the packages availability.

Scanning schedule for vulnerability : (Once a week)

• We will create a schedule in the system manager patch manager where we can define the schedule on which scanning should take place on our instances.
• We can also define the action like “scan” or “scan and install”. Since we do not have the lower environment where we can first test the implementation for prod instances, we suggest to use action “scan” which will only scan and provide us the instances compliant status.
• Once we get the compliant status we can decide whether the patch needs to be installed or not.
• Refer to the below screenshot which shows the scanning schedule. We can add the target i.e instances based on our requirement using tags or we can manually select them as shown below in the screenshot.
• Patching reports can be generated using patch manager, reports will be published to defined s3 buckets, users are expected to pull the report from the s3 bucket.

Instance selection process in the patching configuration screenshot. We can do this manually or tag based as per  the feasibility. Since the instances are very less, we have gone with the manual process for now.

Patch scanning schedule will only provide us with the compliance reporting status of the instances as shown below.

Once the vulnerability is detected, we can manually go onto the server and patch the required vulnerability accordingly.

INSIGHT TO ACTION

The TECHPARTNER team worked with the Enkash management team and tech leads to understand the project needs. Together we chalked down the plan and finalized the setup. Our focus was to leverage AWS services and utilize the open source tools to achieve the required output.

• A patch policy configured in Quick Setup
• A Host Management option configured in Quick Setup
• A maintenance window to run a patch Scan or Install task
• An on-demand Patch now operation

APPLYING PATCHES (Every 15 days or as per the requirement )

Note: Take Ami backup before doing patching of the critical prod servers.

Installing vulnerability to the instances. (Manually)

• Hold the packages on the server which are application dependent using the below command.

sudo apt-mark hold <package-name>

• To check the packages which are held, use the following command.

sudo apt-mark showhold

• To get the latest updates, use the following command.

sudo apt update

• To install the updates, use the following command.

apt-get install –only-upgrade <package name>

• Once the packages are installed, we might have to reboot the server based on the installation. Application sanity must be followed by the dev team after the patches are installed.
• If we face any application, revert the changes by restoring the application using the backup ami.
• In the next scanning from our schedule ssm patch it will show our instances as Compliant.

BENEFITS

AWS STACK