Alexander was born in 1804 in Louisa County, Virginia, the eldest son of Rev. Archibald Alexander and his wife Janetta Waddel. He was born on the Hopewell estate near present-day Gordonsville at the residence of his maternal grandfather after whom he was named, the blind Presbyterian preacher James Waddel. His younger brothers included William Cowper Alexander (1806-1874), president of the New Jersey State Senate and first president of the Equitable Life Assurance Society, and Joseph Addison Alexander (1809-1860), a biblical scholar.
At the time of Alexander's birth, his father was president of Hampden-Sydney College in Virginia. He attended his first schools in Philadelphia after his father was called to serve as minister of the Third Presbyterian Church in 1807. The family then moved to Princeton, New Jersey when Archibald Alexander was named the first professor of the Princeton Theological Seminary in 1812. Alexander entered the College of New Jersey (now Princeton University) in 1817 and graduated in 1820. In 1824, he helped to create the Chi Phi Society, a semi-religious, semi-literary organization, which ceased activity the following year when it merged with the Philadelphian Society.
After graduation, Alexander studied theology at the Princeton Seminary. In 1824 he was appointed a tutor, and during the same year he was licensed to preach by the Presbytery of New Brunswick, New Jersey. He was pastor of a Presbyterian church in Charlotte County, Virginia from 1826 to 1828, and of the First Presbyterian Church of Trenton, New Jersey from 1829 to 1832.
In 1833, he was appointed Professor of Rhetoric and Belles-Letters in the College of New Jersey. He served in this position until 1844, when he became pastor of New York City's Duane Street Presbyterian Church. He served as professor of ecclesiastical history and church government at Princeton Theological Seminary from 1849 to 1851. He then returned to the New York church, which in its new location was known as the Fifth Avenue Presbyterian Church. He served as minister there until his death.
Alexander became a patron of Henry Baldwin Hyde, who founded the Equitable Life Assurance Society in 1859. Many of the company's original directors were members of the Fifth Avenue Presbyterian Church recruited by Alexander. Alexander's brother, William Cowper Alexander, was named the first president of the company. His son, James Waddell Alexander, would also later serve as president of the company, while another son, William C. Alexander, served as company secretary.
On June 18, 1830, Alexander married Elizabeth Clarentine Cabell (1809-1885), daughter of George Cabell and Susannah Wyatt. Her paternal great-grandfather, William Cabell (1699-1774), was the patriarch of the prestigious Cabell family of Virginia. They had seven children:
- George Cabell Alexander (1831-1839)
- Archibald Alexander (1832-1834)
- Henry Carrington Alexander (1835-1894), author of The Life of Joseph Addison Alexander (1870)
- James Waddell Alexander (1839-1915), president of the Equitable Life Assurance Society, 1899-1905; father-in-law of portrait painter John White Alexander and grandfather of mathematician James Waddell Alexander II
- John Alexander (1845-1847)
- William C. Alexander (1848-1937), cofounder of Pi Kappa Alpha and secretary of the Equitable Life Assurance Society
- Janetta Alexander (1850-1851)
Alexander died of dysentery in Red Sweet Springs, Alleghany County, Virginia in 1859 at the age of 55. He had visited the springs due to his feeble health. He was buried in the family plot at Princeton Cemetery.
His published works include his sermons and a book on the life of his father. Alexander's English translation of the hymn "O Sacred Head, Now Wounded," became the most widely used version in 19th and 20th century hymnals. His books, and many juvenile books for Sunday-school libraries, include:
- A Gift to the Afflicted (1835)
- The American Mechanic and Workingman (2 vols., 1847, a collection of papers to mechanics first printed under the pseudonym of "Charles Quill")
- Thoughts on Family Worship (1847)
- Sacramental Addresses (1854)
- The Revival and its Lessons (1859)
- Thoughts on Preaching (1861)
- Faith (1862)
His correspondence is collected in Forty Years' Familiar Letters of James W. Alexander (2 vols., New York, 1860), edited by Dr John Hall.