James Carroll was born at Wairoa, in northern Hawke's Bay, in 1857. His father was European. His mother was of Ngāti Kahungunu. In 1870, although only 13, he joined the campaign against Te Kooti and was mentioned in despatches. He later became a cadet in the Native Department, and took part in selling land on the East Coast.