A native of Nevada, Rizzi migrated to New York City following trouble with the law and became a friend of Sonny Corleone, through whom he met Sonny's sister Connie in 1941 at a surprise birthday party for Sonny's father Vito (depicted in a flashback in The Godfather Part II). They were married in 1945. Vito didn't like the idea of Connie marrying a small-time criminal. He also wasn't enamored with the fact that Carlo wasn't a full-blooded Sicilian; his mother was from northern Italy. He only granted Connie's hand in marriage on condition that they hold an old-style Sicilian wedding.
Rizzi was thrilled at the prestige of being a member of the Corleone crime family, but Vito instructed Tom Hagen not to allow Carlo significant knowledge of the Family's workings, but to "give him a living". Described in the novel as "a punk sore at the world" and bitter at being relegated to a bookmaking job, he regularly beat up and cheated on Connie as a means of exerting his own power over the mighty Corleones. When Connie complained to her parents, Vito coldly refused to help, presumably to punish her for making such a poor choice. In truth, Vito was angered at how Carlo treated his daughter. However, he felt that couldn't do anything because Italian tradition of the time forbade a father from interfering with a daughter's marriage. However, Sonny had to be forcibly restrained and kept away from Rizzi to be prevented from taking action.
Sonny visited Connie one day, and discovered his sister covered in bruises after a particularly bad beating. She begged him not to do anything about it, and he gave her his word. However, without the Don's calming influence, he lost his temper and beat Rizzi mercilessly in the street and threatened to kill him if he ever hit Connie again. Afterward, Rizzi sought revenge by making a deal with the Corleones' chief rival, Emilio Barzini to kill Sonny.
Rizzi set the plan in motion by setting up a call from one of his girlfriends, provoking a pregnant Connie into an argument in which he beat her senseless, beating her with a belt and calling her a "spoiled guinea brat." Connie called Sonny, who flew into a rage and set out to confront Rizzi. En route, Sonny was killed by Barzini's men in a hail of gunfire on the causeway.
Vito forbade any investigations into his son's death, and concluded on his own that the Barzini family was responsible. After his retirement, his youngest son Michael took over the family and brought Rizzi in as his 'right-hand man' for the planned family move to Nevada, treating him as a lieutenant for several years. In truth, however, Michael had learned sometime earlier that Rizzi had plotted with Barzini to have Sonny killed. While Rizzi thought that he was ascending to the top of the family, it was merely a ploy to make him vulnerable. Michael even stood as godfather to Rizzi and Connie's second child, at the same time as his massacre of the heads of the Five Families.
When Michael finally confronted Rizzi, he assured Rizzi that, while he would be exiled from the family, his life would be spared, which satisfied Rizzi enough to confess his involvement with Barzini. When Rizzi got into his car to leave, however, he was violently garroted by Peter Clemenza. It appeared Clemenza handled the job personally since he had stood as godfather to Sonny several years earlier. Despite how badly Rizzi had treated her and the fact that he had had her eldest brother murdered, Connie was enraged upon learning what Michael had done, and resented him for many years afterward.
In Puzo's original novel, Connie initially berates Michael for arranging the murder of her husband, but a week later apologizes for her outburst and assures Michael's wife, Kay, that Michael had nothing to do with Carlo's death. Connie marries again less than a year later.
In the 2006 book "Supermob" by investigative reporter Gus Russo (no relation mentioned), Russo states that the actor who portrayed Carlo, Gianni Russo secured the role by acting as an intermediary between Paramount Studios and New York City Colombo crime family mob boss and patriarch Joseph Colombo, whose Italian-American Civil Rights League had shut down early production of the film in Little Italy, Manhattan over protests. The mob boss Colombo met with the film executives who then hired Russo to play Carlo.