Ubuntu / MySQL Database Crash

Niko Lync 2 years ago

Hi

My database working for motnhs and crashed itself today... with error "Data Dictionary initialization failed." I can't start mysql.service anymore - anybody help please ?

There are logs from mysql after trying to start service:

2023-08-16T09:02:52.032346Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.34-0ubuntu0.22.04.1) starting as process 7397
2023-08-16T09:02:52.046210Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2023-08-16T09:02:52.084885Z 1 [ERROR] [MY-012209] [InnoDB] Multiple files found for the same tablespace ID:
2023-08-16T09:02:52.084960Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 193 = ['sys/traccar/DATABASECHANGELOG.ibd', 'traccar/DATABASECHANGELOG.ibd'] 
2023-08-16T09:02:52.084992Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 194 = ['sys/traccar/DATABASECHANGELOGLOCK.ibd', 'traccar/DATABASECHANGELOGLOCK.ibd'] 
2023-08-16T09:02:52.085008Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 195 = ['sys/traccar/tc_attributes.ibd', 'traccar/tc_attributes.ibd'] 
2023-08-16T09:02:52.085022Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 196 = ['sys/traccar/tc_calendars.ibd', 'traccar/tc_calendars.ibd'] 
2023-08-16T09:02:52.085036Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 197 = ['sys/traccar/tc_commands.ibd', 'traccar/tc_commands.ibd'] 
2023-08-16T09:02:52.085050Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 198 = ['sys/traccar/tc_device_attribute.ibd', 'traccar/tc_device_attribute.ibd'] 
2023-08-16T09:02:52.085064Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 199 = ['sys/traccar/tc_device_command.ibd', 'traccar/tc_device_command.ibd'] 
2023-08-16T09:02:52.085077Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 200 = ['sys/traccar/tc_device_driver.ibd', 'traccar/tc_device_driver.ibd'] 
2023-08-16T09:02:52.085091Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 201 = ['sys/traccar/tc_device_geofence.ibd', 'traccar/tc_device_geofence.ibd'] 
2023-08-16T09:02:52.085104Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 202 = ['sys/traccar/tc_device_maintenance.ibd', 'traccar/tc_device_maintenance.ibd'] 
2023-08-16T09:02:52.085118Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 203 = ['sys/traccar/tc_device_notification.ibd', 'traccar/tc_device_notification.ibd'] 
2023-08-16T09:02:52.085132Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 204 = ['sys/traccar/tc_device_order.ibd', 'traccar/tc_device_order.ibd'] 
2023-08-16T09:02:52.085151Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 205 = ['sys/traccar/tc_devices.ibd', 'traccar/tc_devices.ibd'] 
2023-08-16T09:02:52.085165Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 206 = ['sys/traccar/tc_drivers.ibd', 'traccar/tc_drivers.ibd'] 
2023-08-16T09:02:52.085178Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 207 = ['sys/traccar/tc_events.ibd', 'traccar/tc_events.ibd'] 
2023-08-16T09:02:52.085207Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 208 = ['sys/traccar/tc_geofences.ibd', 'traccar/tc_geofences.ibd'] 
2023-08-16T09:02:52.085221Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 209 = ['sys/traccar/tc_group_attribute.ibd', 'traccar/tc_group_attribute.ibd'] 
2023-08-16T09:02:52.085234Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 210 = ['sys/traccar/tc_group_command.ibd', 'traccar/tc_group_command.ibd'] 
2023-08-16T09:02:52.085248Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 211 = ['sys/traccar/tc_group_driver.ibd', 'traccar/tc_group_driver.ibd'] 
2023-08-16T09:02:52.085261Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 212 = ['sys/traccar/tc_group_geofence.ibd', 'traccar/tc_group_geofence.ibd'] 
2023-08-16T09:02:52.085274Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 213 = ['sys/traccar/tc_group_maintenance.ibd', 'traccar/tc_group_maintenance.ibd'] 
2023-08-16T09:02:52.085288Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 214 = ['sys/traccar/tc_group_notification.ibd', 'traccar/tc_group_notification.ibd'] 
2023-08-16T09:02:52.085301Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 215 = ['sys/traccar/tc_group_order.ibd', 'traccar/tc_group_order.ibd'] 
2023-08-16T09:02:52.085314Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 216 = ['sys/traccar/tc_groups.ibd', 'traccar/tc_groups.ibd'] 
2023-08-16T09:02:52.085327Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 217 = ['sys/traccar/tc_maintenances.ibd', 'traccar/tc_maintenances.ibd'] 
2023-08-16T09:02:52.085340Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 218 = ['sys/traccar/tc_notifications.ibd', 'traccar/tc_notifications.ibd'] 
2023-08-16T09:02:52.085354Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 219 = ['sys/traccar/tc_orders.ibd', 'traccar/tc_orders.ibd'] 
2023-08-16T09:02:52.085367Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 220 = ['sys/traccar/tc_positions.ibd', 'traccar/tc_positions.ibd'] 
2023-08-16T09:02:52.085402Z 1 [ERROR] [MY-012930] [InnoDB] Plugin initialization aborted with error Failed, retry may succeed.
2023-08-16T09:02:52.085441Z 1 [ERROR] [MY-010334] [Server] Failed to initialize DD Storage Engine
2023-08-16T09:02:52.085625Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
2023-08-16T09:02:52.085687Z 0 [ERROR] [MY-010119] [Server] Aborting
2023-08-16T09:02:52.086807Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.34-0ubuntu0.22.04.1)  (Ubuntu).
2023-08-16T09:02:52.987000Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.34-0ubuntu0.22.04.1) starting as process 7408
2023-08-16T09:02:53.000447Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2023-08-16T09:02:53.039920Z 1 [ERROR] [MY-012209] [InnoDB] Multiple files found for the same tablespace ID:
2023-08-16T09:02:53.040005Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 193 = ['sys/traccar/DATABASECHANGELOG.ibd', 'traccar/DATABASECHANGELOG.ibd'] 
...
Niko Lync 2 years ago

Traccar Version: 5.6

David Cole 2 years ago

Restore from backup?

Niko Lync 2 years ago

Backup is the last resort. I solved the problem by deleting files from the sys/traccar/ location - but the question?

Where did they come from? and why did the problem occur suddenly when no work was done on the server???

Anton Tananaev 2 years ago

Someone must have copied those files, but it's definitely not Traccar.