MySQL stop suddenly

Rafael Amores 5 years ago

Hi,
Mysql stop suddenly check the log and I have the following:

WARN: HikariPool-1 - Connection com.mysql.cj.jdbc.ConnectionImpl@37fd8833 marked as broken because of SQLSTATE(08S01), ErrorCode(0) - Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost. - EOFException (... < QueryBuilder:446 < DataManager:425 < DefaultDataHandler:40 < ...)
2021-03-31 22:03:18  WARN: Failed to store position - Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost. - EOFException (... < QueryBuilder:446 < DataManager:425 < DefaultDataHandler:40 < ...)
2021-03-31 22:03:18  WARN: HikariPool-1 - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@630021bd (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.
2021-03-31 22:03:18  WARN: HikariPool-1 - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@92298a9 (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.
2021-03-31 22:03:18  WARN: HikariPool-1 - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@524535ae (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.
2021-03-31 22:03:18  WARN: HikariPool-1 - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@2922fc2a (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.
2021-03-31 22:03:18  WARN: HikariPool-1 - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@4bcd3d7a (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.
2021-03-31 22:03:18  WARN: HikariPool-1 - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@5637137a (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.
2021-03-31 22:03:18  WARN: HikariPool-1 - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@2d24a457 (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.
2021-03-31 22:03:18  WARN: HikariPool-1 - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@13d9ce75 (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.
2021-03-31 22:03:18  WARN: HikariPool-1 - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@57917fb3 (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.
2021-03-31 22:03:48  WARN: Failed to update device - Connection refused (Connection refused) - ConnectException (...)

Any idea.

Anton Tananaev 5 years ago

Sounds like MySQL issue. Is there a reason you are asking on this forum instead of MySQL related resources?

Avant 3 years ago

Probably because like me, he set up Traccar with the recommended MySql database with the supplied settings and it fails just like that.. I used the suggested Docker setup.
I've added "spring.datasource.hikari.max-lifetime=500000" to the url connection string from this page https://www.traccar.org/mysql/ and so far seems to be OK.
Source: https://stackoverflow.com/questions/60310858/possibly-consider-using-a-shorter-maxlifetime-value-hikari-connection-pool-spr/60380157#60380157

Avant 2 years ago

fyi, for the record, this didn't fix the issue.. I have since removed the spring.datasource.hikari.max-lifetime..

so still an issue..

Avant 2 years ago

So, today's test is I have added to new .cnf file to the docker volume that maps to /etc/mysql/conf.d

and added this (which is double the default time which I confirmed was set to 28800):

[mysqld]
wait_timeout=57600

I restarted MySql and confirmed this has taken effect:
SHOW global VARIABLES LIKE 'wait_timeout';

And then restarted Traccar..

Avant 2 years ago

One possible reason why I (alone?) might be getting this, is I'm running in a Docker Swarm.. this is mentioned in this thread due to a TCP idle timeout:
https://github.com/MariaDB/mariadb-docker/issues/113

Changing the MySql timeout from 8 hours to 16 hours is unlikely to fix this as the TCP idle timeout is supposedly 15 mins..

Avant 2 years ago

So also referring to this https://flavio.aiello.ch/mariadb-automatic-docker-limits/, the max timeout should be less than 15 mins (900 seconds) to cater for the network timeout..

I have changed the timeout to 750 seconds as per last article and re-started.. I'll check over the coming days to see if this is now resolved.

If you need alternative ideas, check this thread: https://github.com/moby/moby/issues/37822

Logically this makes sense now.. stay tuned.

Avant 2 years ago

ps - is there a post size limit?? I had to split above to 3 messages to get them to save.. otherwise I got 'forbidden'.

Anton Tananaev 2 years ago

There's no post limit, but there's a limit on number of URLs in a comment.

Avant 2 years ago

that's handy..

Avant 2 years ago

Sadly, this is still unresolved.. I also have added
spring.datasource.hikari.maxLifetime=750000
to the database.url in the traccar.xml

Avant 2 years ago

I don't get an error though.. just this warning.. is it losing data?

2023-07-13 13:42:37  WARN: HikariPool-1 - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@66e48648 (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.
2023-07-13 13:42:37  WARN: HikariPool-1 - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@682364e4 (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.
2023-07-13 13:42:37  WARN: HikariPool-1 - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@6c1ced38 (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.
Anton Tananaev 2 years ago

We don't use spring, so I'm not sure why you keep using that config parameter. It won't do anything at all.