Got fatal error 1236 from master – MySQL – Broken Master – Master Replication

Scenario

• You have a MySQL Master-Master replication setup, Lets say Master1 (LIVE) and Master2 (DR Warm Standby)
• Master2 server has been restarted for some reason.
• Now Master1 LIVE (the slave of DR) slave status is stopped with the below error.

Last_IO_Errno: 1236
Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'

Solution

For the safer side, Backup your database used by the application with mysqldump and keep it in a safe place. The command usually wont cause any data corruption on downtime.

Master1: stop slave;

Master2: flush logs;
Master2: show master status; — take note of the master log file and master log position

mysql> show master status;
+-------------------+----------+--------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+-------------------+----------+--------------+------------------+
| master-bin.000006 | 106 | | |
+-------------------+----------+--------------+------------------+

Master1: CHANGE MASTER TO MASTER_LOG_FILE = 'master-bin.000006', MASTER_LOG_POS = 106;
Master1: start slave;

Now run show slave status \G; and verify everything is up and running.