https://www.thegeekdiary.com/high-number-of-connections-is-close_wait-state-in-netstat-command-output/
BOUND | Bound, ready to connect or listen. |
CLOSED | Closed. The socket is not being used. |
CLOSING | Closed, then remote shutdown; awaiting acknowledgment. |
CLOSE_WAIT | Remote shutdown; waiting for the socket to close. |
ESTABLISHED | Connection has been established. |
FIN_WAIT_1 | Socket closed; shutting down connection. |
FIN_WAIT_2 | Socket closed; waiting for shutdown from remote. |
IDLE | Idle, opened but not bound. |
LAST_ACK | Remote shutdown, then closed; awaiting acknowledgment. |
LISTEN | Listening for incoming connections. |
SYN_RECEIVED | Active/initiate synchronization received and the connection under way |
SYN_SENT | Actively trying to establish connection. |
TIME_WAIT | Wait after close for remote shutdown retransmission. |
'CLOSE_WAIT' state means the other end of the connection has been closed while the local end is still waiting for the application to close. The local end-point has received a connection termination request and acknowledged it e.g. a passive close has been performed and the local end-point needs to perform an active close to leave this state.
CLOSE_WAIT indicates that the remote endpoint (other side of the connection) has closed the connection.
TIME_WAIT indicates that local endpoint (this side) has closed the connection.
Details
‘CLOSE_WAIT’ state on tcp connections occurs if the system has not received a close system call from the application, after having received notification (‘FIN’ packet) from the other system that it has closed its endpoint. In other words it means that the local end of the connection has received ‘FIN’ from the other end, but the OS is waiting for the program at the local end to actually close its connection.
The problem is that a program running on the local machine is not closing the socket. It is not a TCP tuning issue. A connection can stay in ‘CLOSE_WAIT’ forever while the program holds the connection open. So mostly this issue happens due to an application bug. However if TCP/IP parameters are not properly set, closed TCP/IP connections will stay for a very long time in various ‘CLOSE’ states which take file descriptors from the process. In order to solve this problem, sometimes it is necessary to tune TCP/IP parameters, so that TCP/IP connections are closed in a short amount of time.
修改Time_Wait参数的方法 (在服务端修改,需要重启机器)
reg add "HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters" /v "TcpTimedWaitDelay" /t REG_DWORD /d 30 /f
<2012R2和Win8.1的系统,执行:
reg add "HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters" /v "TcpTimedWaitDelay" /t REG_DWORD /d 30 /f
≥2012R2和Win8.1的系统,执行:
reg add "HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters" /v "TcpTimedWaitDelay" /t REG_DWORD /d 2 /f
Key: | HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters |
---|---|
Value: | TcpTimedWaitDelay |
Data Type: | REG_DWORD |
Range: | Windows Server 2012 and earlier: 30-300 (decimal)Windows 8 and earlier: 30-300 (decimal)Windows Server 2012 R2 and later: 2-300 (decimal)Windows 8.1 and later: 2-300 (decimal) |
Default value: | 0x78 (120 decimal) |
Recommended value: | 30 |
Value exists by default? | No, needs to be added. |
Close_Wait引发的问题:
Close_Wait数量过多会引起网络可用连接少、网络性能下降,并占用系统非换页内存。 尤其是在有连接池的情况下(比如HttpRequest)会耗尽连接池的网络连接数,导致无法建立网络连接。
解决CLOSE_WAIT的方法(在客户端执行,需要重启机器):
https://blog.csdn.net/lgxzzz/article/details/124551824
如果一直保持在CLOSE_WAIT状态,那么只有一种情况,就是在对方关闭连接之后服务器程序自己没有进一步发出ack信号。换句话说,就是在对方连接关闭之后,程序里没有检测到,或者程序压根就忘记了这个时候需要关闭连接,于是这个资源就一直 被程序占着。个人觉得这种情况,通过服务器内核参数也没办法解决,服务器对于程序抢占的资源没有主动回收的权利,除非终止程序运行。
1、一般原因都是TCP连接没有调用关闭方法。需要应用来处理网络链接关闭。
2、对于Web请求出现这个原因,经常是因为Response的BodyStream没有调用Close
比如Widnows下:
使用HttpWebRequest 一定要保证GetRequestStream和GetResponse对象关闭,否则容易造成连接处于CLOSE_WAIT状态。
3、TCP的KeepLive功能,可以让操作系统替我们自动清理掉CLOSE_WAIT的连接。
但是KeepAliveTime在Windows操作系统下默认是7200秒,也就是2个小时才清理一次,往往满足不了需求,可以调小该数值。
Windows下的调整方法为
[HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters]下的以下三个参数:
KeepAliveTime,设置其值为300000(单位为毫秒,300000代表5分钟)
KeepAliveInterval,设置其值为1000(单位为毫秒,1000代表1秒)
TcpMaxDataRetransmissions,设置其值为4
具体执行:管理员身份运行如下命令(重启生效)
reg add "HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters" /v "KeepAliveTime" /d 300000 /t REG_DWORD /f
reg add "HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters" /v "KeepAliveInterval" /d 1000 /t REG_DWORD /f
reg add "HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters" /v "TcpMaxDataRetransmissions" /d 4 /t REG_DWORD /f
reg add "HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters" /v "TcpTimedWaitDelay" /t REG_DWORD /d 2 /f
reg add "HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters" /v "TcpMaxConnectRetransmissions" /t REG_DWORD /d 2 /f
或者以下另存为close_wait.reg导入系统即可(重启生效)
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters]
"KeepAliveTime"=dword:000493e0
"KeepAliveInterval"=dword:000003e8
"TcpMaxDataRetransmissions"=dword:00000004
"TcpTimedWaitDelay"=dword:00000002
"TcpMaxConnectRetransmissions"=dword:00000002
像一些暴力破解占用服务端资源导致服务端出现很多close_wait的,客户端不可控,只能从服务端限制IP了,例如下图,就从服务端限制195.201.59.45这个bad ip访问。
参考资料:
KeepAliveInterval
https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-2000-server/cc957548(v=technet.10)
KeepAliveTime
https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-2000-server/cc957549(v=technet.10)
TcpMaxConnectRetransmissions
https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-2000-server/cc938209(v=technet.10)
TcpMaxDataRetransmissions
https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-2000-server/cc938210(v=technet.10)
TcpTimedWaitDelay
https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-2000-server/cc938217(v=technet.10)
EnableDeadGWDetect
https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-2000-server/cc960464(v=technet.10)
https://www.ibm.com/support/pages/enabling-tcpip-keepalive-windows
https://community.progress.com/s/article/Avoiding-TCP-IP-Port-Exhaustion
统计closewait的powershell命令
(get-nettcpconnection -state closewait).count
(netstat -ano).count
cmd下netstat -ano|find /c "CLOSE_WAIT"