The town draws its name from the fjord. With a population of about 2,600 Ísafjörður is the largest town in the peninsula of Vestfirðir (Westfjords) and the seat of the Ísafjarðarbær municipality, which includes the nearby Hnífsdalur, Flateyri, Suðureyri, and Þingeyri. It is located on a spit of sand, or eyri, in the Skutulsfjörður fjord which meets the waters of the larger Isafjarðardjúp fjord.

According to the Landnámabók (the book of settlement), Skutulsfjördur was first settled by Helgi Magri Hrólfsson in the 9th century. In the 16th century the town grew as it became a trading post for foreign merchants. The town of Ísafjörður was granted municipal status in 1786. The local folk museum contains the oldest house in Iceland, built in 1734. The largest collection of old timber frame houses in Iceland is in this area. The houses were mostly constructed by foreign traders in the late 18th century. These include Tjöruhús (completed in 1742), Krambúð (1761), and Turnhús (1744) which now contains a maritime museum.

The town is connected by road and a recent 5.4 km road tunnel to Bolungarvík which lies 15 km to the northwest, and to the small town of Súðavík to the east. The partly one-lane Vestfjarðagöng (Vestfirðir Tunnel), completed in 1996, leads to the small towns of Flateyri and Suðureyri, and to the western parts of the Westfjords. Ísafjörður has an airport with regular flights to Reykjavík.

Fishing has been the main industry in Ísafjörður, and the town has one of the largest fisheries in Iceland. A severe decline in the fishing industry, has led the inhabitants to seek work elsewhere, leading to a decline in the town's population. The harbor also serves ferries to nearby settlements as well as larger cruise ships for tourists visiting the area.

Ísafjörður has a school of music as well as a hospital. The older former hospital building now accommodates a cultural center. A university center, Háskólasetur Vestfjarða, which acts as a distance learning center for the 7000 residents of the Westfjords, was established in March 2005.