aws

http://status.aws.amazon.com/ https://www.expeditedssl.com/aws-in-plain-english
Hlodowig 2020-07-21T16:31:52.372100Z

Hi everybody, off-topic AWS question. Any of you have experienced terrible response times after putting your EC2 through an application LB? We had this thing running fine in a t2 micro. As soon as we put it behind the LB performance dropped. We had another case were we just removed availability zones from the LB that had no instances on them. But this time I just can't get what's going on.

ghadi 2020-07-21T16:33:13.372300Z

it could be anything

ghadi 2020-07-21T16:33:31.372700Z

you could have run out of cpu credits

ghadi 2020-07-21T16:34:01.373100Z

you need some data to know definitively

1
ghadi 2020-07-21T16:35:17.374700Z

profiling the application/JVM profiling the machine monitor the load balancer (in rough priority order of things to investigate)

Hlodowig 2020-07-21T18:15:48.375300Z

What's weird is this happens as soon as we put this thing behind the LB. Otherwise things run just fine.

jumar 2020-07-21T20:32:25.376500Z

"performance dropped" means "latency increased"? How much?

Hlodowig 2020-07-21T20:39:56.377900Z

Yup, from 1-3 seconds to up to a minute. Seems like the first requests you make "warm up" something, after that the application works fine for a while and then takes a dive again.

ghadi 2020-07-21T20:44:18.379200Z

my $0.02; it is too early to speculate -- find some data about what is going on, then make a hypothesis

ghadi 2020-07-21T20:44:26.379500Z

CPU normal?

ghadi 2020-07-21T20:44:41.380Z

Loadbalancer target group instances healthy?

ghadi 2020-07-21T20:44:58.380400Z

t2/t3 instances use CPU credits -- are you still running with t2 instances?

ghadi 2020-07-21T20:45:55.381200Z

starting up the application could have used up all the cpu credits -- load balancer might be a total red herring

Hlodowig 2020-07-21T20:53:37.382900Z

CPU normal. Instances show as healthy. No CPU credits since these are being paid on demand. Everything looks fine until I plug this thing to the LB. Direct access to the instances works just fine.