Default is 5. Should be just fine if you don't have a specific use case that would require higher limits.
If so, just crank it up, should be safe unless you assign Infinity or something like that and push it too much (then you have another problem though). We use 15 in production where our server parses a lot of external web pages.
The new default in master is Infinity. (Also, there's opt-in KeepAlive that actually keeps sockets alive even if there are no pending requests.)
The ulimit will prevent you from opening up too many anyway. The HTTP Client is not the correct place to implement job control and queueing with such a low limit by default.
First things first, IsaacSchlueter in the thread, wow :)
So, I'm stress testing our company Node app to find where we can go with it's performance. First problem was file descriptor, which I fixed with your "graceful-fs" module.
But now, I'm reaching some "invisible" limit that I can't identify. My app doesn't return any error in the log.
Does "maxSockets" will help to receive more requests also or is just to make requests?
What performance problem with FD? How many requests exactly are you handling per second with what code or processing? Are you sure its not what he just mentioned, ulimit? The docs say client requests so yes its just to make them.
Why do you need to handle 5k requests simultaneously? Maybe you mean per second? Maybe you can add a server and do round-robin DNS? Then you will he able to do double unless there is a database bottleneck or something. But you said concurrent which 4k really concurrent is asking a lot.
Currently, we are with a C# application and in the process to migrate to Node.js. I wan't to prove that Node.js can handle huge concurrency, so I wan't to benchmark the highest possible numbers I can. We are currently usign AWS to load balace.
And it's being a great exercise anyway.
We are using Redis for session store. Do you thing raising this parameter can influence on Redis performance? Because in the Redis server, the service is only using 5% in process, when Node.js is 99%.
Comments
I didn't know about this maxSockets limitation.
Is this something safe to raise?
Default is 5. Should be just fine if you don't have a specific use case that would require higher limits.
If so, just crank it up, should be safe unless you assign Infinity or something like that and push it too much (then you have another problem though). We use 15 in production where our server parses a lot of external web pages.
The new default in master is Infinity. (Also, there's opt-in KeepAlive that actually keeps sockets alive even if there are no pending requests.)
The ulimit will prevent you from opening up too many anyway. The HTTP Client is not the correct place to implement job control and queueing with such a low limit by default.
First things first, IsaacSchlueter in the thread, wow :)
So, I'm stress testing our company Node app to find where we can go with it's performance. First problem was file descriptor, which I fixed with your "graceful-fs" module.
But now, I'm reaching some "invisible" limit that I can't identify. My app doesn't return any error in the log.
Does "maxSockets" will help to receive more requests also or is just to make requests?
What performance problem with FD? How many requests exactly are you handling per second with what code or processing? Are you sure its not what he just mentioned, ulimit? The docs say client requests so yes its just to make them.
I'm handling from 4k to 10k simultaneous requests.
And about the client requests, hmm ok, no problem. I will keep looking. Thanks!
Why do you need to handle 5k requests simultaneously? Maybe you mean per second? Maybe you can add a server and do round-robin DNS? Then you will he able to do double unless there is a database bottleneck or something. But you said concurrent which 4k really concurrent is asking a lot.
Currently, we are with a C# application and in the process to migrate to Node.js. I wan't to prove that Node.js can handle huge concurrency, so I wan't to benchmark the highest possible numbers I can. We are currently usign AWS to load balace.
And it's being a great exercise anyway.
We are using Redis for session store. Do you thing raising this parameter can influence on Redis performance? Because in the Redis server, the service is only using 5% in process, when Node.js is 99%.
http://nodejs.org/docs/latest/api/http.html#http_agent_maxso...
If you use it smart its safe enought
It's pretty much always safe to set to Infinity or to turn the agent off. This is an anti-feature.
I'd call it a bug. It's fixed in master.