Server stops receiving data
Of course, the UI continues to work.
Traccar client shows an error: "sending failed".
When running, Traccar Client sends data every 5 minutes. Then something happens, the server stops responding and every minute there is a failed attempt to connect to the server. Below are logs from my traefik, which balances HTTPS traffic for osmand protocol. I will add that I have the port for gt06 (5023) exposed directly from docker and forwarded on the firewall and traefik is not involved. Besides, such a configuration worked fine for a very long time.
Of course, the UI continues to work.
Traccar client shows an error: "sending failed".
When running, Traccar Client sends data every 5 minutes. Then something happens, the server stops responding and every minute there is a failed attempt to connect to the server. Below are logs from my traefik, which balances HTTPS traffic for osmand protocol. I will add that I have the port for gt06 (5023) exposed directly from docker and forwarded on the firewall and traefik is not involved. Besides, such a configuration worked fine for a very long time.
I can't paste here logs (get forbidden), so I pasted them here: https://pastebin.com/raw/GMMZDcru
As you can see, the client closes the connection after about 15 seconds (HTTP status code 499), due to lack of response from the server.
The server also continues to listen on TCP/5023 port, I can perform telnet to it.
$ telnet XXXXX 5023
Trying XXXXX...
Connected to XXXXXX.
Escape character is '^]'.
sadfa
asdf
asdf
asdf
^]
telnet>
Connection closed.Nothing in the logs appears.
I will add that as Traccar works normally and responds to telnet, everything I type in telnet appears in the logs.
$ telnet XXXXX 5023
Trying XXXXX...
Connected to XXXXXX.
Escape character is '^]'.
asd
asd
asd
asd
asd
^]
telnet>
Connection closed.Traccar logs:
2024-04-02 11:04:00 INFO: [T7968fbdf] connected
2024-04-02 11:04:01 INFO: [T7968fbdf: gt06 < YYYY] 6173640d0a
2024-04-02 11:04:01 INFO: [T7968fbdf: gt06 < YYYY] 6173640d0a
2024-04-02 11:04:02 INFO: [T7968fbdf: gt06 < YYYY] 6173640d0a
2024-04-02 11:04:02 INFO: [T7968fbdf: gt06 < YYYY] 6173640d0a
2024-04-02 11:04:03 INFO: [T7968fbdf: gt06 < YYYY] 6173640d0a
2024-04-02 11:04:05 INFO: [T7968fbdf: gt06 < YYYY] 04
2024-04-02 11:04:07 INFO: [T7968fbdf] disconnectedWhat could be the cause?
Can increasing the logging level give any clues?
How to approach the topic.
I am having the same issue in version 5.6 with 200 devices. The server stops receiving the data from the device and I have to restart the traccar service.
If server stops accepting new data, you have to check the server status:
- Check the memory limit and see if you're hitting it
- Check number of open connections
- Get jstack output to see thread status
I increased the memory limit. Previously I had 4GB, but now I saw, the default limit is 1GB.
Add this to the docker compose file:
entrypoint:
- java
- -Xms4g
- -Xmx4g
- -Djava.net.preferIPv4Stack=true
command:
- -jar
- tracker-server.jar
- conf/traccar.xml
ulimits:
nproc: 65535
nofile:
soft: 50000
hard: 50000Check number of open connections
Before change:
root 529185 529161 0 kwi02 ? 00:04:13 java -Xms1g -Xmx1g -Djava.net.preferIPv4Stack=true -jar tracker-server.jar conf/traccar.xml
$ cat /proc/529185/limits | grep "open files"
Max open files 1048576 1048576 files After change:
root 1171680 1171660 8 14:17 ? 00:00:49 java -Xms4g -Xmx4g -Djava.net.preferIPv4Stack=true -jar tracker-server.jar conf/traccar.xml
$ cat /proc/1171680/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size unlimited unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 65535 65535 processes
Max open files 50000 50000 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 63029 63029 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us Lets wait.
Nothing has changed. After some time, the data is not received.
I don't think it's related to MEM or limits of connection. I have connected only 5 devices.
I've switched logs to debug, will upload them when when the situation repeats itself.
Hi,
I am writing because I have a problem with my traccar server instance.
Version 5.12-debian running in docker along with PostgreSQL database
For a very long time everything worked great. In February the server stopped receiving data after some time. Restarting the server helps, but usually within 24 hours the situation repeats.
For example, yesterday I restarted traccar around noon (14:03 CEDT):
It was running today until 10:56 CEDT:
Nothing else appears in the logs after that.
I am using two clients: