If you try to install Virtualbox on top of a RHEL instance you will likely see an error

When you try to install a virtualization layer on top of an EC2 instance, you will likely see an error like below:-

As per Amazon Support, "It's not possible to run VirtualBox within an EC2 instance since we trying to run a hypervisor within Xen based hypervisor which is unsupported. The required VT-x CPU extensions simply aren't exposed at the guest layer either. You can try to convert your appliance (into a format supported by Xen which is very similar to EC2) and then to import the VM per the instructions provided at instances of your vm

Simple grep command to output lines before and after the search string

$grep -C 3 "NullPointerException" *.log --color

The option "-C 3" will output 3 lines before and 3 lines after the occurrence of NPE in the logs. 

Using jmap utility to take heap dump on an application running in VPC

If you have secure environments in which there are strict ingress/egress rules and cannot open up jmx ports for tools like VisualVM or set JVM parameters like -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/heapdump

In such cases you can use jmap utility such as

$ jmap -dump:format=b,file=/tmp/heapdump <pid>
Dumping heap to /tmp/heapdump ...
Heap dump file created

Once the dump has been generated, you can analyze the heap using Eclipse memory analyzer (MAT)

AWS ELB cannot redirect users from http port to https port automatically

In AWS ELB there is no option to automatically redirect http requests back to https unlike apache httpd, so we will have to allow both ports (http and https) and then use reverse proxy like httpd to have a rewrite rule such as below

1. Have your ELB pass both HTTP and HTTPS traffic on to your backend server as HTTP traffic on port 80. ELB Backend Http -> http Https -> http 2. Create a rewrite rule on your Back end Web-Server For Apache: <VirtualHost *:80> ... RewriteEngine On RewriteCond %{HTTP:X-Forwarded-Proto} =http RewriteRule https://%{HTTP:Host}%{REQUEST_URI} [L,R=permanent] ... </VirtualHost> In the above re-write rule we are utilizing the X-Forwarded-Proto header from the request to do the redirection. The X-Forwarded-Proto request header helps you identify the protocol (HTTP or HTTPS) that a client used to connect to your server. Your server access logs contain only the protocol used between the server and the load balancer; they contain no information about the protocol used between the client and the load balancer. To determine the protocol used between the client and the load balancer, use the X-Forwarded-Proto request header. Elastic Load Balancing stores the protocol used between the client and the load balancer in the X-Forwarded-Proto request header and passes the header along to your server. Your application or website can use the protocol stored in the X-Forwarded-Proto request header to render a response that redirects to the appropriate URL. More information on the X-Forwarded headers http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/TerminologyandKeyConcepts.html#x-forwarded-headers

Uploading SSL certificates on ELB through AWS CLI without providing console access

In certain situations (like when you are hosting sites for your customer) you may not want your customers to send you their SSL certificates and private keys for security reasons. Best practices dictate that private keys reside with the customers. In other cases, you could be hosting a site that your customers wants to CNAME to on their DNS server. Under those conditions, you don't want to handle customer's SSL certificates and private keys. You could follow the steps below to have the upload the certs

• Create a temporary IAM user in your customer called "temp-cert-user" and assign the below custom IAM policy:-

{
 "Version": "2012-10-17",
 "Statement": [
  {
   "Effect": "Allow",
   "Action": ["iam:*AccessKey*"],
   "Resource": "arn:aws:iam::<AWS_ACCOUNT_ID>:*"
  },
  {
   "Effect": "Allow",
   "Action": ["iam:UploadServerCertificate"],
   "Resource": "arn:aws:iam::*"
  }
 ]
}

• Next, ask your customer to install AWS CLI 
• From the customer's machine, ask them to add the AWS_ACCESS_KEY and AWS_SECRET_KEY in $HOME/.aws/config file
• Next, they can execute the below command

$aws iam --profile <profile>  upload-server-certificate --server-certificate-name test-cert --certificate-body file://testcert.pem --private-key file://testcertkey.pem --certificate-chain file://testcert.pem

{

    "ServerCertificateMetadata": {

        "Path": "/",

        "Arn": "arn:aws:iam::<AWS_ACCOUNT_ID>:server-certificate/test-cert",

        "ServerCertificateId": "ASC....QN7B4",

        "ServerCertificateName": "test-cert",

        "UploadDate": "2014-06-18T17:34:16.567Z"

    }

}

• Now they can check if the certificate got upload by using get-server-certificate command

C:\certs>aws iam --profile tempcert get-server-certificate --server-certificate-name test-cert

{

    "ServerCertificate": {

        "CertificateChain": "-----BEGIN CERTIFICATE-----

                                        -----END CERTIFICATE-----",

        "CertificateBody": "-----BEGIN CERTIFICATE-----

                                        -----END CERTIFICATE-----",

        "ServerCertificateMetadata": {

            "Path": "/",

            "Arn": "arn:aws:iam::<AWS_ACCOUNT_ID:server-certificate/test-cert",

            "ServerCertificateId": "ASC...QN7B4",

            "ServerCertificateName": "test-cert",

            "UploadDate": "2014-06-18T17:34:16Z"

        }

    }

}

NOTE - For allowing temp-cert-user to have get-server-certificate and delete-server-certificate authorization, you have to modify the IAM policy to include the below:-

{
   "Effect": "Allow",
   "Action": ["iam:GetServerCertificate"],
   "Resource": "arn:aws:iam::*"
  },
  {
   "Effect": "Allow",
   "Action": ["iam:DeleteServerCertificate"],
   "Resource": "arn:aws:iam::*"
  }

Key ELB metrics to watch for in a load spike

When experiencing a surge in inbound requests, we have to watch the ELB's cloudwatch metrics closely. The metrics are documented in the MonitoringLoadBalancerWithCW

The key metrics are

• RequestCount
• Latency
• HTTPCode_ELB_5XX
• HTTPCode_Backend_5XX
• SurgeQueueLength

Typically, you will see a linear relationship between 'RequestCount' and 'Latency' metrics. When the load increases, the latency will also increase correspondingly. With default settings you will see a ELB timeout of 60 secs getting invoked if latency intervals are greater than ELB timeout. Latency metric is indicative of the time duration that ELB has to wait for a response from the instance to which it has handed the request. If the instances takes longer to respond (HTTP 200, 4xx or 5xx codes) then we will see increased latency. HTTPCode_ELB_5xx metric indicates the no. of occurrences of ELB failing to handle the incoming request and ELB directly send a http 5xx error code back to the client. HTTPCode_Backend_5xx indicates the occurrences of errors in the backend instances failing to get a valid response from end service. SurgeQueueLength metric indicates the no. of requests that have been queued up by the ELB waiting for a healthy instance to become available. 

The CloudWatch metrics of RequestCount and Latency showing linear relationship will look like

NOTE - If your instances behind the ELB are in the same zone, then you may want to disable the "cross zone load balancing" feature of the ELB so reduce the performance overhead by a small amount. 

you can also enable access log and access collection as per the AWS documentation links below:-

1. http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/enable-access-logs.html
2. http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/access-log-collection.html

Amazon questionnaire to request ELB pre-warming through AWS support

Amazon suggests informing them of any load tests in your infrastructure that is front ended by ELB's. If any flash traffic is expected, then it would make sense to have the ELB's pre-warmed. You can read more about pre-warming in the AWS documentation - pre-warming

Questionnaire from AWS support which we need to answer for requesting ELB pre-warming:-

******************

1. What is the DNS name for the ELB(s) that require manual scaling?
2. An approximate increase percentage in traffic, or expected requests/sec that will go through the load balancer (whichever is easier to answer).
3. The average amount of data passing through the ELB per request/response pair (size of packets in KB).
4. The rate of traffic increase expected. This can be a qualitative value such as "this should increase steadily over the span of an hour," or "we expect traffic to increase suddenly once our sale/event is announced."
5. Expected percent of traffic going through the ELB that will be using SSL termination.
6. The number of Availability Zones added to the load balancer, or that will be added.
7. Is the back-end currently scaled to the level it will be during the event?
8. If not, when do you expect to add the required back-end instance count? Also what type of instance and how many ?
9. When will your event start?
10. When do you expect the event will end?
11. A brief description of your use case, with a detailed explanation of your expected (if you're going to deviate from the normal request pattern) and normal traffic patterns.
12. We're looking to glean details about what kinds of requests you expect. For example:

• Are they long running requests?
• Do you need connections to be open for an extended period of time?
• Are these basic GET requests? POSTs? PUTs? Large file uploads/downloads?
• How frequently do you expect surges in traffic, and how quickly traffic will ramp up to peak during these surges.
• Are the back-end instances using persistent connections (aka: keep-alive)
• Is your production application currently down due to this, or is your production application traffic severely impacted?.
• Is this an expected surge in traffic?

******************