Fort William, originally named Fort Anomabo, was built by the British around 1753–1757 after thwarting a French attempt to erect a fort in Anomabu. It succeeded earlier forts from 1640 (by the Dutch) and 1674 (Fort Charles by the English), both of which were later destroyed or abandoned.

Designed by military engineer John Apperly and completed under Richard Brew in 1760, the fort was renamed Fort William in the 19th century by its commander Brodie Cruickshank, who added another storey and named it after King William IV.

It became a central British stronghold in the transatlantic slave trade until abolition in 1807, extremely notable for having a large prison built specifically for detaining enslaved Africans.

Post-independence, Fort William served varied roles as a rest house, post office, and later a state prison until 2001. Today, it functions as a community library, preserving its historical significance.

In 1979, it was inscribed on the UNESCO World Heritage List, recognizing its crucial role in Atlantic history.