This information is faulty. Fine for leaving Azerbaijan after visa expiration is 300-400 AZN which is around 200 USD.
Foreign nationals have to leave Azerbaijan within 48 hours from the expiry date of the visa or extend the visa before it expires at the Consular Department of the Ministry of Foreign Affairs (Address: 4 Shikhali Gurbanov Street, Baku AZ 1009, Republic of Azerbaijan, tel: + 994 12 4929692).
If foreign nationals fail to extend their visas before expiry date, they will be liable to a fine from US $ 20 to 30. It is not possible to renew a visa after it has expired. Those foreign citizens who stay over 30 days since entering Azerbaijan and fail to register are subject to arrest by authorities with investigation to follow. There are facts when foreigners extend their visas but forget to get registration certificate. In this case the person is fined and gets a warning to obtain certificate. These certificates differ from those that Azerbaijan citizens have by color, those ones for foreigners are of pink color and bear information about a foreign citizen. If you leave the country before termination of 30 days you are free of registration.
There is an opportunity to extend your visa through Migration Service once you are in Azerbaijan. Process takes 4 days and costs 30 AZN (roughly 30 Euros). Make sure you apply for visa extension before your visa expires, otherwise they would be able to issue you only exit papers to leave a country.
The immigration office is located near Azedliq Prospekti metro station - this is a new station at end of the line after Nasimi and not on older metro maps. You come out in a park in the middle of two roads. Walk slightly uphill to the big main road, turn right past the building construction works and the next impressive building is the immigration office. It opens at 9am but get there much earlier to avoid long waits. Saturdays they are apparently open at 9am for training and open proper from 11am (for a short time).