503 Service Unavailable - after upgrade to 4.6

Glenn Miler5 years ago

Hi Guys

After upgrading i simply can't connect to the server, i know it is working and i have been looking apache logs and traccar logs but nothing seems to help i even tried to reinstall everything, but nothing helps.

Traccar log

2019-10-27 17:39:11  INFO: HikariPool-1 - Starting...
2019-10-27 17:39:11  INFO: HikariPool-1 - Start completed.
2019-10-27 17:39:12  INFO: Clearing database change log checksums
2019-10-27 17:39:12  INFO: SELECT COUNT(*) FROM db1002847.DATABASECHANGELOGLOCK
2019-10-27 17:39:12  INFO: SELECT COUNT(*) FROM db1002847.DATABASECHANGELOGLOCK
2019-10-27 17:39:12  INFO: SELECT `LOCKED` FROM db1002847.DATABASECHANGELOGLOCK WHERE ID=1
2019-10-27 17:42:12  INFO: Waiting for changelog lock....
2019-10-27 17:42:22  INFO: SELECT `LOCKED` FROM db1002847.DATABASECHANGELOGLOCK WHERE ID=1
2019-10-27 17:42:22  INFO: Waiting for changelog lock....
2019-10-27 17:42:32  INFO: SELECT `LOCKED` FROM db1002847.DATABASECHANGELOGLOCK WHERE ID=1
2019-10-27 17:42:32  INFO: Waiting for changelog lock....
2019-10-27 17:42:42  INFO: SELECT `LOCKED` FROM db1002847.DATABASECHANGELOGLOCK WHERE ID=1
2019-10-27 17:42:42  INFO: Waiting for changelog lock....
2019-10-27 17:42:52  INFO: SELECT `LOCKED` FROM db1002847.DATABASECHANGELOGLOCK WHERE ID=1
2019-10-27 17:42:52  INFO: Waiting for changelog lock....
2019-10-27 17:43:02  INFO: SELECT `LOCKED` FROM db1002847.DATABASECHANGELOGLOCK WHERE ID=1
2019-10-27 17:43:02  INFO: Waiting for changelog lock....
2019-10-27 17:43:12  INFO: SELECT `LOCKED` FROM db1002847.DATABASECHANGELOGLOCK WHERE ID=1
2019-10-27 17:43:12  INFO: Waiting for changelog lock....
2019-10-27 17:43:22  INFO: SELECT `LOCKED` FROM db1002847.DATABASECHANGELOGLOCK WHERE ID=1
2019-10-27 17:43:22  INFO: Waiting for changelog lock....
2019-10-27 17:43:32  INFO: SELECT `LOCKED` FROM db1002847.DATABASECHANGELOGLOCK WHERE ID=1
2019-10-27 17:43:32  INFO: Waiting for changelog lock....
2019-10-27 17:43:42  INFO: SELECT `LOCKED` FROM db1002847.DATABASECHANGELOGLOCK WHERE ID=1
2019-10-27 17:43:42  INFO: Waiting for changelog lock....
2019-10-27 17:43:52  INFO: SELECT `LOCKED` FROM db1002847.DATABASECHANGELOGLOCK WHERE ID=1
2019-10-27 17:43:52  INFO: Waiting for changelog lock....
2019-10-27 17:44:02  INFO: SELECT `LOCKED` FROM db1002847.DATABASECHANGELOGLOCK WHERE ID=1
2019-10-27 17:44:02  INFO: Waiting for changelog lock....
2019-10-27 17:44:12  INFO: SELECT ID,`LOCKED`,LOCKGRANTED,LOCKEDBY FROM db1002847.DATABASECHANGELOGLOCK WHERE ID=1
2019-10-27 17:44:12 ERROR: Main method error - Could not acquire change log lock.  Currently locked by 2a01:4f8:c2c:6fe5:0:0:0:1%eth0 (2a01:4f8:c2c:6fe5:0:0:0:1%eth0) since 10/27/19, 9:34 AM - LockException (... < DataManager:310 < *:90 < Context:292 < Main:110 < ...)

Apache Log

[Sun Oct 27 17:37:37.407080 2019] [proxy:error] [pid 942:tid 139780658497280] (111)Connection refused: AH00957: HTTP: attempt to connect to 127.0.0.1:8082 (localhost) failed
[Sun Oct 27 17:37:37.407178 2019] [proxy_http:error] [pid 942:tid 139780658497280] [client 82.103.130.178:8751] AH01114: HTTP: failed to make connection to backend: localhost
[Sun Oct 27 17:37:39.522094 2019] [proxy:error] [pid 942:tid 139780641711872] (111)Connection refused: AH00957: HTTP: attempt to connect to 127.0.0.1:8082 (localhost) failed
[Sun Oct 27 17:37:39.522161 2019] [proxy_http:error] [pid 942:tid 139780641711872] [client 82.103.130.178:18111] AH01114: HTTP: failed to make connection to backend: localhost

Traccar Status

root@track:/opt/traccar/logs# sudo systemctl status traccar.service
● traccar.service - traccar
   Loaded: loaded (/etc/systemd/system/traccar.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sun 2019-10-27 17:54:01 CET; 41s ago
  Process: 1785 ExecStart=/opt/traccar/jre/bin/java -jar tracker-server.jar conf/traccar.xml (code=exited, status=1/FAILURE)
 Main PID: 1785 (code=exited, status=1/FAILURE)

Oct 27 17:54:01 track.seginova.com traccar[1785]:         at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:236)
Oct 27 17:54:01 track.seginova.com traccar[1785]:         at liquibase.Liquibase.clearCheckSums(Liquibase.java:1455)
Oct 27 17:54:01 track.seginova.com traccar[1785]:         at org.traccar.database.DataManager.initDatabaseSchema(DataManager.java:310)
Oct 27 17:54:01 track.seginova.com traccar[1785]:         at org.traccar.database.DataManager.<init>(DataManager.java:90)
Oct 27 17:54:01 track.seginova.com traccar[1785]:         at org.traccar.Context.init(Context.java:292)
Oct 27 17:54:01 track.seginova.com traccar[1785]:         at org.traccar.Main.run(Main.java:110)
Oct 27 17:54:01 track.seginova.com traccar[1785]:         ... 1 more
Oct 27 17:54:01 track.seginova.com systemd[1]: traccar.service: Main process exited, code=exited, status=1/FAILURE
Oct 27 17:54:01 track.seginova.com systemd[1]: traccar.service: Unit entered failed state.
Oct 27 17:54:01 track.seginova.com systemd[1]: traccar.service: Failed with result 'exit-code'.

Can any help where i need to look, it has worked before this with out any problems.

Anton Tananaev5 years ago

Your database is locked. Usually it means that you restarted in the middle of the migration. Search forum. It has been discussed many times.

Glenn Miler5 years ago

Hi Anton

Thanks for answering me back it was also more like pointing in my right direction instead of looking 1000 places for fixes :)

And you where right.

I open phpmyadmin and in the table DATABASECHANGELOGLOCK i changed LOCKED from 1 to 0 and now everything is working again. thanks.

Hussah Saleh4 years ago

Hi All,
Well, I am pointing two traccar server to the same database. Old one working fine while the new traccar server always get 503 even if I update the locked to 0 on DATABASECHANGELOGLOCK.

any help how can I make those two traccar servers pointing to the same DB without any issues.

Thank you,

Hussah Saleh4 years ago

This is my log file:

2020-08-04 02:40:55  INFO: UPDATE traccar.DATABASECHANGELOG SET MD5SUM = '8:805ef202791dcc246373ca16f7c19c4f' WHERE ID = 'changelog-4.9' AND AUTHOR = 'author' AND FILENAME = 'changelog-4.9'
2020-08-04 02:40:56  INFO: SELECT COUNT(*) FROM traccar.DATABASECHANGELOG
2020-08-04 02:40:56  INFO: Reading from traccar.DATABASECHANGELOG
2020-08-04 02:40:56  INFO: SELECT * FROM traccar.DATABASECHANGELOG ORDER BY DATEEXECUTED ASC, ORDEREXECUTED ASC
2020-08-04 02:40:56  INFO: Successfully released change log lock
2020-08-04 02:40:57  INFO: Logging initialized @22431ms to org.eclipse.jetty.util.log.Slf4jLog
2020-08-04 02:40:59  INFO: Your platform does not provide complete low-level API for accessing direct buffers reliably. Unless explicitly requested, heap buffer will always be preferred to avoid potential system instability.
2020-08-04 02:41:01  INFO: Operating system name: Linux version: 4.15.0-112-generic architecture: amd64
2020-08-04 02:41:01  INFO: Java runtime name: OpenJDK 64-Bit Server VM vendor: Oracle Corporation version: 11.0.5+10-LTS
Hussah Saleh4 years ago

Sorry for non clear log file, here is my log file:

This is my log file:

2020-08-04 02:48:08  INFO: SELECT MD5SUM FROM traccar.DATABASECHANGELOG WHERE MD5SUM IS NOT NULL LIMIT 1
2020-08-04 02:48:08  INFO: SELECT COUNT(*) FROM traccar.DATABASECHANGELOG
2020-08-04 02:48:08  INFO: Reading from traccar.DATABASECHANGELOG
2020-08-04 02:48:08  INFO: SELECT * FROM traccar.DATABASECHANGELOG ORDER BY DATEEXECUTED ASC, ORDEREXECUTED ASC
2020-08-04 02:48:08  INFO: UPDATE traccar.DATABASECHANGELOG SET MD5SUM = '8:ea6f202534dc7845fd1a49e1d8384d9c' WHERE ID = 'changelog-3.3' AND AUTHOR = 'author' AND FILENAME = 'changelog-3.3'
2020-08-04 02:48:08  INFO: UPDATE traccar.DATABASECHANGELOG SET MD5SUM = '8:194e3ae7ec6d201917136e04a26e237b' WHERE ID = 'changelog-3.5' AND AUTHOR = 'author' AND FILENAME = 'changelog-3.5'
.
.
.
2020-08-04 02:48:17  INFO: UPDATE traccar.DATABASECHANGELOG SET MD5SUM = '8:a23040cfc84d4b8021f40ee214a9d78b' WHERE ID = 'changelog-4.7' AND AUTHOR = 'author' AND FILENAME = 'changelog-4.7'
2020-08-04 02:48:17  INFO: UPDATE traccar.DATABASECHANGELOG SET MD5SUM = '8:805ef202791dcc246373ca16f7c19c4f' WHERE ID = 'changelog-4.9' AND AUTHOR = 'author' AND FILENAME = 'changelog-4.9'
2020-08-04 02:48:17  INFO: SELECT COUNT(*) FROM traccar.DATABASECHANGELOG
2020-08-04 02:48:17  INFO: Reading from traccar.DATABASECHANGELOG
2020-08-04 02:48:17  INFO: SELECT * FROM traccar.DATABASECHANGELOG ORDER BY DATEEXECUTED ASC, ORDEREXECUTED ASC
2020-08-04 02:48:18  INFO: Successfully released change log lock
2020-08-04 02:48:19  INFO: Logging initialized @22201ms to org.eclipse.jetty.util.log.Slf4jLog
2020-08-04 02:48:21  INFO: Your platform does not provide complete low-level API for accessing direct buffers reliably. Unless explicitly requested, heap buffer will always be preferred to avoid potential system instability.
2020-08-04 02:48:23  INFO: Operating system name: Linux version: 4.15.0-112-generic architecture: amd64
2020-08-04 02:48:23  INFO: Java runtime name: OpenJDK 64-Bit Server VM vendor: Oracle Corporation version: 11.0.5+10-LTS