AWS11: Installing AWS CLI on Windows 10

AWS Command line interface allows you to access your AWS resources directly from powershell console.

As of 2020/03/22, you can download msi installer for AWS CLI from here.

After installation of AWS CLI, you have to proceed basic configuration as it is required to access to aws environment.

# Configure your aws credential
> aws configure

AWS Access Key ID [None]: AK*************

AWS Secret Access Key [None]: ViC****************

Default region name [None]: ap-northeast-1

Default output format [None]: json

These information will be stored on .aws folder on your user directory with files "config" and "credentials".

After setting up, you can check connectivity and resources on you aws environment with following commands.

# Run aws commands
> aws ec2 describe-instances

> aws lambda list-functions

AWS10: Cloudformation (Implement loadbalancer for EC2 instances)

Tutorial:

We will set up application loadbalancer for provisioned two EC2 instances.
The stack consists of resource for application loadbalancer which allows HTTP access to public and forward connection to attached EC2 instances.



In below template, "VPC", "SUBNETS", "SecurityGroup" and "WebServers" are defined as parameters which we can specify and refer provisioned stacks.
On outputs, these loadbalancer resource is exported for future usage.



Key resource types are below:

AWS::ElasticLoadBalancingV2::LoadBalancer	Loadbalancer which is required to specified subnets and security groups
AWS::ElasticLoadBalancingV2::Listener	Listener mainly defines listening port of loadbalancer
AWS::ElasticLoadBalancingV2::TargetGroup	It defines specific targets to forward the request which is received by loadbalancer.

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Description": "Application Loadbalancer.",
    "Parameters": {
        "VPC": {
            "Type": "String"
        },
        "SUBNETS": {
            "Type": "String"
        },
        "SecurityGroup": {
            "Type": "String"
        },
        "WebServers": {
            "Type": "String"
        }
    },
    "Resources": {
        "ApplicationLoadBalancer": {
            "Type": "AWS::ElasticLoadBalancingV2::LoadBalancer",
            "Properties": {
                "Subnets": [
                    {
                        "Fn::ImportValue": {
                            "Fn::Sub": "${SUBNETS}-Subnet01"
                        }
                    },
                    {
                        "Fn::ImportValue": {
                            "Fn::Sub": "${SUBNETS}-Subnet02"
                        }
                    }
                ],
                "Name": "ALB01",
                "SecurityGroups": [{ "Fn::ImportValue": {
                        "Fn::Sub": "${SecurityGroup}-WebServerSecurityGroup"
                    }
                }]
            }
        },
        "ALBListener": {
            "Type": "AWS::ElasticLoadBalancingV2::Listener",
            "Properties": {
                "DefaultActions": [
                    {
                        "Type": "forward",
                        "TargetGroupArn": {
                            "Ref": "ALBTargetGroup"
                        }
                    }
                ],
                "LoadBalancerArn": {
                    "Ref": "ApplicationLoadBalancer"
                },
                "Port": "80",
                "Protocol": "HTTP"
            }
        },
        "ALBTargetGroup": {
            "Type": "AWS::ElasticLoadBalancingV2::TargetGroup",
            "Properties": {
                "HealthCheckIntervalSeconds": 10,
                "HealthCheckTimeoutSeconds": 5,
                "HealthyThresholdCount": 2,
                "Port": 80,
                "Protocol": "HTTP",
                "UnhealthyThresholdCount": 5,
                "VpcId": {
                    "Fn::ImportValue": {
                        "Fn::Sub": "${VPC}-VPCID"
                    }
                },
                "TargetGroupAttributes": [
                    {
                        "Key": "stickiness.enabled",
                        "Value": "true"
                    },
                    {
                        "Key": "stickiness.type",
                        "Value": "lb_cookie"
                    },
                    {
                        "Key": "stickiness.lb_cookie.duration_seconds",
                        "Value": "30"
                    }
                ],
                "Targets": [
                    {
                        "Id": {
                            "Fn::ImportValue": {
                                "Fn::Sub": "${WebServers}-WebServer01"
                            }
                        },
                        "Port": 80
                    },
                    {
                        "Id": {
                            "Fn::ImportValue": {
                                "Fn::Sub": "${WebServers}-WebServer02"
                            }
                        },
                        "Port": 80
                    }
                ]
            }
        }
    },
    "Outputs": {
        "ApplicationLoadBalancer": {
            "Description": "ALB output value",
            "Value": {
                "Ref": "ApplicationLoadBalancer"
            },
            "Export": {
                "Name": {
                    "Fn::Sub": "${AWS::StackName}-ApplicationLoadBalancer"
                }
            }
        }
    }
}

See more detailed information of each element on here.

Here's a video tutorial for this.

AWS09: Cloudformation (Provision EC2 linked to subnets and security groups stack)

Tutorial:

We will set up EC2 which is linked to provisioned subnets and security groups.
The stack contains 2 web server and 1 bastion server which are associated with provisioned security groups, and deployed on provisioned subnets.

In below template, "SUBNETS" and "SecurityGroup" are defined as we can specify and refer provisioned stacks.
On outputs, these EC2 resources are exported for future usage.

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Description": "EC2 instances.",
    "Parameters": {
        "SUBNETS": {
            "Type": "String"
        },
        "SecurityGroup": {
            "Type": "String"
        }
    },
    "Resources": {
        "WebServer01": {
            "Type": "AWS::EC2::Instance",
            "Properties": {
                "InstanceType" : "t2.micro",
                "ImageId": "ami-0c00780768a0dad61",
                "KeyName": "webserver_key",
                "Tags": [
                    {
                        "Key": "Name",
                        "Value": "WebServer01"
                    }
                ],
                "NetworkInterfaces": [
                    {
                        "DeviceIndex": "0",
                        "GroupSet": [
                            {
                                "Fn::ImportValue": {
                                    "Fn::Sub": "${SecurityGroup}-WebServerSecurityGroup"
                                }
                            }
                        ],
                        "SubnetId": {
                            "Fn::ImportValue": {
                                "Fn::Sub": "${SUBNETS}-Subnet01"
                            }
                        }
                    }
                ]
            },
            "Metadata": {
                "AWS::CloudFormation::Designer": {
                    "id": "fc33911e-cd3b-496b-9975-a77684964fee"
                }
            }
        },
        "WebServer02": {
            "Type": "AWS::EC2::Instance",
            "Properties": {
                "InstanceType" : "t2.micro",
                "ImageId": "ami-0c00780768a0dad61",
                "KeyName": "webserver_key",
                "Tags": [
                    {
                        "Key": "Name",
                        "Value": "WebServer02"
                    }
                ],
                "NetworkInterfaces": [
                    {
                        "DeviceIndex": "0",
                        "GroupSet": [
                            {
                                "Fn::ImportValue": {
                                    "Fn::Sub": "${SecurityGroup}-WebServerSecurityGroup"
                                }
                            }
                        ],
                        "SubnetId": {
                            "Fn::ImportValue": {
                                "Fn::Sub": "${SUBNETS}-Subnet02"
                            }
                        }
                    }
                ]
            },
            "Metadata": {
                "AWS::CloudFormation::Designer": {
                    "id": "e0d44d1a-18b1-47db-92b2-71a9376bce33"
                }
            }
        },
        "Bastion": {
            "Type": "AWS::EC2::Instance",
            "Properties": {
                "InstanceType" : "t2.micro",
                "ImageId": "ami-0c00780768a0dad61",
                "KeyName": "webserver_key",
                "Tags": [
                    {
                        "Key": "Name",
                        "Value": "Bastion"
                    }
                ],
                "NetworkInterfaces": [
                    {
                        "AssociatePublicIpAddress": "true",
                        "DeviceIndex": "0",
                        "GroupSet": [
                            {
                                "Fn::ImportValue": {
                                    "Fn::Sub": "${SecurityGroup}-WebServerSecurityGroup"
                                }
                            }
                        ],
                        "SubnetId": {
                            "Fn::ImportValue": {
                                "Fn::Sub": "${SUBNETS}-Subnet01"
                            }
                        }
                    }
                ]
            },
            "Metadata": {
                "AWS::CloudFormation::Designer": {
                    "id": "f646646c-aa3f-4fec-9252-9eb16f19266a"
                }
            }
        }
    },

    "Outputs" : {
      "Bastion" : {
        "Description" : "EC2 instance exported values",
        "Value" :  { "Ref" : "Bastion" },
        "Export" : { "Name" : {"Fn::Sub": "${AWS::StackName}-Bastion" }}
      },
      "WebServer01" : {
        "Description" : "EC2 instance exported values",
        "Value" :  { "Ref" : "WebServer01" },
        "Export" : { "Name" : {"Fn::Sub": "${AWS::StackName}-WebServer01" }}
      },
      "WebServer02" : {
        "Description" : "EC2 instance exported values",
        "Value" :  { "Ref" : "WebServer02" },
        "Export" : { "Name" : {"Fn::Sub": "${AWS::StackName}-WebServer02" }}
      }
    }    
}  

See more detailed information of each element on here.

Here's a video tutorial for this.

AWS08: Cloudformation (Provision security groups linked to VPC stack)

Tutorial:

We will set up security group which is linked to provisioned VPC exactly same way of previous post.
The security group would be for ssh and web connection with port 22 and 80 respectively.

In below template, "NetworkStackNameParameter" is defined as we can specify provisioned VPC stack "cf-vpc-igw" during subnets stack creation.
On outputs, the security group is exported as well.

{

    "AWSTemplateFormatVersion": "2010-09-09",
    "Description" : "Subnets.",

    "Parameters" : {
        "NetworkStackNameParameter": {
            "Type" : "String"
        }  
    },

    "Resources" : {
      "WebServerSecurityGroup" : {
        "Type" : "AWS::EC2::SecurityGroup",
        "Properties" : {
          "GroupDescription" : "Enable HTTP ingress",
          "VpcId" : { "Fn::ImportValue" : {"Fn::Sub": "${NetworkStackNameParameter}-VPCID" } },
          "SecurityGroupIngress" : [ { 
            "IpProtocol" : "tcp",
            "FromPort" : "80",  
            "ToPort" : "80",
            "CidrIp" : "0.0.0.0/0"
          }, {
            "IpProtocol" : "tcp",
            "FromPort" : "22",  
            "ToPort" : "22",
            "CidrIp" : "0.0.0.0/0"
          } ],
          "Tags" : [ { "Key" : "Name", "Value" : "WebSecurityGroup"} ] 

        }
      }
    },
    
    "Outputs" : {
      "WebServerSecurityGroup" : {
        "Description" : "The subnet ID to use for public web servers",
        "Value" :  { "Ref" : "WebServerSecurityGroup" },
        "Export" : { "Name" : {"Fn::Sub": "${AWS::StackName}-WebServerSecurityGroup" }}
      }
    }
    
}

See more detailed information of each element on here.

Here's a video tutorial for this.

AWS07: Cloudformation (Provision subnets linked to VPC stack)

Tutorial:

We will set up 2 subnets on provisioned VPC exactly same way of previous post.

In below template, "NetworkStackNameParameter" is defined as we can specify provisioned VPC stack "cf-vpc-igw" during subnets stack creation.
Public route is added to "Subnet01" as we need to communicate bastion server via ssh.
On outputs, 2 subnets value are exported as we will refer them another stacks we will create later.

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Description" : "Subnets.",

    "Parameters" : {
        "NetworkStackNameParameter": {
            "Type" : "String"
        }  
    },

    "Resources" : {
      "Subnet01" : {
        "Type" : "AWS::EC2::Subnet",
        "DeletionPolicy" : "Delete",
        "Properties" : {
          "VpcId" : { "Fn::ImportValue" : {"Fn::Sub": "${NetworkStackNameParameter}-VPCID" } },
          "CidrBlock" : "10.0.0.0/25",
          "AvailabilityZone" : "ap-northeast-1a",
          "Tags" : [{ "Key" : "Name", "Value" : "Subnet-01" }]
        }
      },

      "Subnet02" : {
        "Type" : "AWS::EC2::Subnet",
        "DeletionPolicy" : "Delete",
        "Properties" : {
          "VpcId" : { "Fn::ImportValue" : {"Fn::Sub": "${NetworkStackNameParameter}-VPCID" } },
          "CidrBlock" : "10.0.0.128/25",
          "AvailabilityZone" : "ap-northeast-1c",
          "Tags" : [{ "Key" : "Name", "Value" : "Subnet-02" }]
        }
      },
    
      "PublicRouteTable" : {
        "Type" : "AWS::EC2::RouteTable",
        "Properties" : {
          "VpcId" : { "Fn::ImportValue" : {"Fn::Sub": "${NetworkStackNameParameter}-VPCID" } } 
        }
      },
      "PublicRoute" : {
        "Type" : "AWS::EC2::Route",
        "Properties" : {
          "RouteTableId" : { "Ref" : "PublicRouteTable" },
          "DestinationCidrBlock" : "0.0.0.0/0",
          "GatewayId" : { "Fn::ImportValue" : {"Fn::Sub": "${NetworkStackNameParameter}-InternetGateway" } } 
        }
      },
      "PublicSubnetRouteTableAssociation" : {
        "Type" : "AWS::EC2::SubnetRouteTableAssociation",
        "Properties" : {
          "SubnetId" : { "Ref" : "Subnet01" },
          "RouteTableId" : { "Ref" : "PublicRouteTable" }
        }
      }
    },
    
    "Outputs" : {
      "Subnet01" : {
        "Description" : "The subnet ID to use for public web servers",
        "Value" :  { "Ref" : "Subnet01" },
        "Export" : { "Name" : {"Fn::Sub": "${AWS::StackName}-Subnet01" }}
      },
      "Subnet02" : {
        "Description" : "The subnet ID to use for public web servers",
        "Value" :  { "Ref" : "Subnet02" },
        "Export" : { "Name" : {"Fn::Sub": "${AWS::StackName}-Subnet02" }}
      }
    }
    
}

See more detailed information of each element on here.

Here's a video tutorial for this.

AWS06: CloudFormation (Provision VPC and Internet gateway)

Description: 

CloudFormation is aws feature that helps you create, setup and manage infrastructure resources on aws more efficiently.
You compose template, and CloudFormation handles it for provisioning resources as described in the template.
Construct infrastructure with CloudFormation can be done either by design console on aws or uploading prepared JSON/YAML template.

Tutorial:

Let's check how it works step by step.
We will setup webserver infrastructure which consists of 1 vpc, 2 subnets and webservers within it.
And templates would be following:

1. cf-vpc -------------------- Template defines VPC and Internet gateway.
2. cf-subnets --------------- Template defines subnets and route table.
3. cf-securitygroups ------- Template defines security groups.
4. cf-webserver ------------ Template defines EC2 instances, including bastion host.
5. cf-loadbalancer --------- Template defines application load balancer.

In below template, VPC and Internet gateway are defined with "Resources" section.
At the same time, Internet gateway is associated with VPC.
"Outputs" defines the resources which will be referred by another template.

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Resources" : {
      "VPC" : {
        "Type" : "AWS::EC2::VPC",
        "Properties" : {
          "EnableDnsSupport" : "true",
          "EnableDnsHostnames" : "true",
          "CidrBlock" : "10.0.0.0/24",
          "Tags" : [ { "Key" : "Name", "Value" : "VPC01"} ]
        }
      },
      "InternetGateway" : {
        "Type" : "AWS::EC2::InternetGateway",
        "Properties" : {
          "Tags" : [ { "Key" : "Name", "Value" : "IntGW01" } ]
        }
      },
      "VPCGatewayAttachment" : {
         "Type" : "AWS::EC2::VPCGatewayAttachment",
         "Properties" : {
           "VpcId" : { "Ref" : "VPC" },
           "InternetGatewayId" : { "Ref" : "InternetGateway" }
         }
      }
    },
    "Outputs" : {
      "VPCId" : {
        "Description" : "VPC ID",
        "Value" :  { "Ref" : "VPC" },
        "Export" : { "Name" : {"Fn::Sub": "${AWS::StackName}-VPCID" }}
      },
      "InternetGateway" : {
        "Description" : "InternetGateway",
        "Value" :  { "Ref" : "InternetGateway" },
        "Export" : { "Name" : {"Fn::Sub": "${AWS::StackName}-InternetGateway" }}
      }
    }
}

See more detailed information of each element on here.

Here's a video tutorial for this.

AWS 05: Accessing S3 bucket from EC2 instance

Let's see how we can access AWS S3 (Simple Storage Services) bucket, and save file on it with aws command.
Also we will check that we can display saved image on S3 bucket referred by web page.

On this tutorial, we use below  environment:

FYI:
S3 offers a robust and scalable storage service, and you can check further details of S3 on here.

Let's take a look each steps.

1. Check s3 bucket is being created and make it public.
2. Create s3 access role and attache it to EC2 instances.
3. Put an image file to the bucket.
4. Make the image file public on s3 console.
5. Get object URL of the image file.
6. Edit html file and refer the image by img tag.
7. Access to the web server, and check if the image is displayed.  

Please take a look an actual operation on following video.

AWS 04: Construct HA web server environment (Classic Loadbalancer)

Let's see how we can construct high availability web server environment with the use of Loadbalancer on AWS.

The environment would be like below:

The users http requests is to be handled by Loadbalancer (WebLB01) which is exposed to public internet, and the handled requests passed to member servers under the Loadbalancer.
As two web servers are assigned, any visitors will not be affected even if one of them goes down.



First of all, two web servers must be prepared, it would be achieved by creating instance from the AMI image of previous session.




Let's take a look each steps of provisioning of Loadbalancer.

1. Check required instances and RDS are ready for this tutorial.
2. Navigate your self to "Load Balancers" section on left pane of EC2, and hit create "Load Balancer".
3. Select Classic Loadbalancer for this time.
4. Define Loadbalancer name, open HTTP port and assign subnets.
5. Assign security group.
6. Health check configuration uses default setting.
7. Assign EC2 instances to the Loadbalancer.
8. Make sure the Loadbalancer is facing to the internet.
9. After provisioned Loadbalancer, it would take a while the status of assigned instances to be availabled. Be patient.
10. After the instances get inservice status, you can check how the Loadbalancer allocate requests to each servers.
11. One important thing is that classic Loadbalancer do not keep the session on default. So that you need to set "stickiness" to be enabled.
12. Navigate yourself to Loadbalancer page and find "Port Configuration" on "Description" tab.
13. Hit "Edit stickness", then tick "Enable load balancer generated cookie stickness.
14. You finally see the Loadbalancer handles web session.
15. And also try to shutdown one of EC2 instance, and see how the requests to be handled by Loadbalancer.

Please take a look an actual operation on following video.

AWS 03: Create EC2 instance snapshot and launch another instance from it

Let's see how we can create a spanshot of particular EC2 instance, and launch EC2 instance from create image.
We will take the snapshot of WebServer_01, and launch an EC2 instance as WebServer_02.

1. Navigate your self to Snapshots page on EC2, and create Snapshot.
   
2. It's going to take a while to create snapshot, please be patient.
   
3. After take the snapshot, you need to create an image for launching EC2 instance.
   
4. Now you can find the image is regisitered on AMIs page.
   
5. Let's launch another web server EC2 instance from created image.
    
6. You can access to another launched web server via assigned IPv4 address.
   
7. Let's also check if the web server is getting requests from outside.

Please take a look an actual operation on following video.

AWS 02: Launch mySQL service with Amazon RDS

This time I will launch first Amazon RDS to replace existing mySQLserver on Subnet_02 which was created on previous session. RDS is a database service available in AWS, and you can choose various types of database platform, such as mySQL, MS-SQL, Oracle or DynamoDB. Using RDS means that you are free from launching EC2 instance, such as Linux / Windows for setting up database.

I will launch free version of RDS and choose mySQL as database platform, and try to connect from web server sitting on Subnet_01.

Followings are brief description of each steps.

1. Navigate to RDS console.
2. Hit "Create Database" and make basic settings, such as choose database or set master user and password etc.
3. It will take a while to boot up the database.
4. Confirm DNS name assigned and login from each EC2 instances.
5. By default, security groups for your RDS needs to be corrected, otherwise you will not be able to connect and login to the database from remote servers.
6. After you confirm the connectivity has no problem, then try to login from phpMyAdmin via internet access. Don't forget to update your config.inc.php file with correct host name.

Please watch a video for more detail.

AWS 01: First step to launch VPC and EC2 instances

Let's create simple AWS VPC with two subnets and two instances.
The structure is that an web server is exposed to public, and the server is connected to mySQL server which resides on another network segment within VPC.

Here is an image:

And followings are brief glossary for each AWS resources.

VPC	Virtual Private Cloud is an on-demand configurable pool of shared computing resources allocated within a public cloud environment. You define / design network segments, and deploy EC2 instances on VPC.
Subnets	Segmented network within VPC. Appropriate route table setting is needed for connecting with other subnets or gateways.
Internet gateway	The gateway which your deployed servers access to the internet.
NAT gateway	The gateway provides the internet access to your servers resides on private subnet.
EC2 instance	Servers which are created from particular images.
Security group	Firewall setting associated with EC2 instance. SG's are stateful. Example: If you allow an incoming port 80, the outgoing port 80 will be automatically opened.
Network ACL	Firewall setting associated to with subnets. Unlike SG, network ACL is stateless.

1. I am showing a video how we can setup VPC and two subnets.

2. Secondary, we provision an EC2 instance for web server.

I used following commands for setting up web service on provisioned EC2 instance.

sudo yum update -y

sudo yum install -y httpd24 php72 php72-mysqlnd

sudo service httpd start

sudo chkconfig httpd on

sudo usermod -a -G apache ec2-user

sudo chown -R ec2-user:apache /var/www

sudo chmod 2775 /var/www

find /var/www -type d -exec sudo chmod 2775 {} \;

find /var/www -type f -exec sudo chmod 0664 {} \;

sudo yum install php72-mbstring.x86_64 -y

sudo service httpd restart

cd /var/www/html

wget https://www.phpmyadmin.net/downloads/phpMyAdmin-latest-all-languages.tar.gz

mkdir phpMyAdmin && tar -xvzf phpMyAdmin-latest-all-languages.tar.gz -C phpMyAdmin --strip-components 1

3. Next step is to provision EC2 instance for mySQL.
Since the instance is sitting on private subnet, NAT gateway and route table setting should be implemented for providing subnet_02 internet connection.

Here is a set of commands for initial setup for mySQL.

sudo yum install -y mysql57-server

sudo service mysqld start

sudo mysql_secure_installation

sudo chkconfig mysqld on

4. Finally, we add required security group settings on subnet_02, and we can access to phpMyAdmin page via web browser with assigned global IP address.