Scaling should occur on actual traffic/user throughput statistics not on response time. Response times can increase for a variety of reasons, of which only one of is increased traffic load. For example, response times can increase based on back-end database contention or thread contention.
Of course whichever configuration is chosen, it is important to load test your application so you understand its performance profile and that all contention points are understood.
Comments
Performance engineer here.
Scaling should occur on actual traffic/user throughput statistics not on response time. Response times can increase for a variety of reasons, of which only one of is increased traffic load. For example, response times can increase based on back-end database contention or thread contention.
Of course whichever configuration is chosen, it is important to load test your application so you understand its performance profile and that all contention points are understood.