The musical begins on the morning of July Fourth in 1912. A railroad conductor announces the next stop, River City, Iowa, to a coach filled with traveling salesmen. Speaking rhythmically, the salesmen begin a conversation about the merits of cash versus credit and the ways their products and lives have changed as the result of "modren" merchandising ("Rock Island").

One of the salesmen, Charlie Cowell, asks if anyone has heard of Professor Harold Hill, a salesman who is ruining the reputation of all traveling salesmen. Cowell explains that Hill moves from town to town, selling musical instruments, uniforms and the promise of lessons for a boy's band, leaving town with the collected money before anyone has discovered that he is musically illiterate.

As the train stops in River City, Cowell, who has been trying to find and expose Hill, mentions that Hill wouldn't get far with the stubborn Iowans. Before the train begins to move again, a salesman who has quietly been playing cards grabs his suitcase and announces that the conversation has prompted him to give Iowa a try. When asked his name, the stranger flashes his suitcase, bearing the name "Prof. Harold Hill," and he quickly exits the train as it starts to move off. He finds himself facing River City's Main Street, decorated with Fourth of July bunting and crowded with townspeople.

As workers move a pool table into the River City Billiard Parlor that is owned by Mayor Shinn, the townspeople greet the mayor and each other. They sing with pride of their contrariness ("Iowa Stubborn"). As they disperse, Hill enters the scene and tries to rent a horse and buggy at the livery stable. There, he meets his old friend and one-time partner, Marcellus Washburn. Washburn, who knows Harold's real first name is Greg, remembers Hill's last sales gimmick was selling steam-powered automobiles. Hill tells Marcellus he'd be selling them still if somebody hadn't actually invented such a vehicle.

Marcellus has given up his old ways and settled down in River City to work in the livery stable. After Harold explains his plans, Marcellus warns him to watch out for the town's music teacher and librarian, Marian Paroo, as she'd expose Harold's con on the spot. Harold asks him to point her out and then he sets about thinking of a way to convince the parents of River City of the necessity of a boy's band.

When Marcellus tells him about the new pool table in town, Harold recognizes his chance. He begins talking about the trouble that has entered River City in the shape of a pool table. To the fast-growing crowd, Harold delivers a rapid-fire sales pitch about the corrupting influence of a pool table on the boys of the town ("Trouble"); as the townspeople join him, Marcellus signals that Marian Paroo is passing by.

Harold follows Marian home, but she rejects his attempts to start a conversation with her on the street.

As Marian enters the house, Amaryllis, her young piano student, is playing an exercise while Mrs. Paroo, Marian's mother, continues with her household chores. Marian tells her mother about the strange man (Harold) who has been following her and trying to speak with her. While Amaryllis plays arpeggios, Mrs. Paroo scolds Marian for not speaking to the man, criticizing Marian's high expectations, both for the townspeople and for men ("Piano Lesson/If You Don't Mind My Saying So").

Winthrop, Marian's little brother, enters the house, and Amaryllis invites him to a party. Winthrop, who has a lisp and doesn't like to speak, mispronounces Amaryllis's name. When she giggles, he runs from the room. Amaryllis, upset that Winthrop never talks to her, starts crying and tells Marian she is worried she'll never find a sweetheart to wish about on the evening star. Marian tells her to go on wishing, using the word "someone" until the right person comes along. As Amaryllis plays her crossed-hands piece, Marian gazes at the evening star and wishes her unnamed "someone" goodnight ("Goodnight, My Someone").

Inside the high school gymnasium, Mayor Shinn is presiding over the Fourth of July celebrations. His wife, Eulalie Mackecknie Shinn leads a group of Wa Tan Ye girls and then, dressed as Columbia, leads the town in singing "Columbia, Gem of the Ocean." As the mayor begins his recitation of the Gettysburg Address, he is stopped by the constantly bickering members of the school board, who remind him that the next presentation is a Native American costume spectacle. The spectacle concludes with his wife counting to 20 in the "Indian tongue," but before she can finish, young Tommy Djilas lights a firecracker behind her.

The four school board members begin arguing as the mayor again tries his Gettysburg recitation. The mayor is foiled once more, this time by Harold, who steals the crowd's attention, continuing his earlier sermon about the pool table. He tells the crowd that he has come to River City to organize a boy's band as the solution to the corrupting influence of the pool table. He then entrances them with a story of when six of the greatest marching bands in America came to town on the very same day ("Seventy-Six Trombones"). The townspeople join in, dancing and parading around the gymnasium.

The mayor, alarmed at seeing the Iowans so excited, orders the school board to get Harold's credentials. As Tommy is being led out of the gymnasium by the constable, he is warned by the mayor to stay away from Zaneeta, the Shinn's oldest daughter. Harold realizes that, if he can make an ally of Tommy, he'd have the town's youth on his side, too. He quickly intercedes on Tommy's behalf and agrees to take responsibility for the boy. Harold points out a passing girl and gives Tommy money to take her to the candy shop. After the teenagers leave, the constable tells Harold he's made a couple of mistakes: the girl is the mayor's oldest daughter, Zaneeta Shinn, and Mayor Shinn owns the Billiard Parlor.

The school board approaches Harold and demands his credentials; Harold, stalling because he has no credentials, asks them each to sing the words "ice cream," which they do in perfect barbershop quartet harmony. Finding music more interesting than Harold, the quartet sings "Sincere" as Harold sneaks away to look for Marian.

Harold follows Marian to the library where, before slamming the door in his face, she warns him she will check his credentials in the reference books. The ladies of the town surround Harold, buzzing with excitement over the band. Mrs. Shinn, however, is still withholding her judgment until her husband receives Harold's credentials. When she moves her foot to relieve the pain of her bunions, Harold comments on her grace and insists she lead the Ladies Auxiliary for the Classic Dance, with the other ladies as members. Mrs. Shinn immediately falls under Harold's spell. She consents to head the committee and she, too, is now an ally.

When Harold asks about Marian, the ladies huddle together like hens and begin to gossip. They accuse her of promoting Balzac, Chaucer and other authors of "dirty books" ("Pick-a-Little"). They also suggest that she had been involved with "Miser" Madison, a late River City resident who donated the gymnasium, picnic park, hospital and library to the town. The school board appears, again demanding Harold's credentials, and again he deftly distracts them by saying goodnight to the ladies, prompting a song from the quartet ("Goodnight Ladies").

Harold arrives at the Paroo house. He flatters Mrs. Paroo on her facial muscles, suggesting this means Winthrop will be a great cornet player. After Winthrop asks if the uniform will have a stripe, Harold tries to engage him in a conversation, but the boy runs off. Mrs. Paroo explains that Winthrop hardly speaks at all. Thinking Harold's gift of gab might mean he's Irish, she asks Harold where he is from. As Harold tells her his alma mater is the Gary Conservatory of Gary, Indiana, Marian returns home and tries to dissuade her mother from ordering an instrument. Marian gets angry when Harold asks to speak to Winthrop's father, who is dead. When she enters the house, Mrs. Paroo apologizes for Marian's outburst.

After Harold leaves, Marian sends Winthrop to the library to get the reference book she needs to check on Harold's credentials. Mrs. Paroo, who likes Harold, accuses Marian of not thinking of the future.

Tommy is making a date with Zaneeta as Mayor Shinn enters, complaining to his wife that Harold has mesmerized the entire town. Marian appears with the reference book, but before she can hand it to the mayor, Gracie, his youngest daughter, excitedly announces the arrival of the Wells Fargo Wagon. The townspeople line the street to greet it ("Wells Fargo Wagon"). Winthrop breaks through the crowd to express his hope that the wagon is bringing his band instrument.

Harold hands Winthrop his cornet. Winthrop, now seemingly unashamed of his speech impediment, turns and excitedly tells Marian how happy he is. Harold hands out the rest of the instruments to the boys. He tells them lessons will follow, but they should first get acquainted with their instruments and think about the Minuet in G. The mayor concedes that Harold has won the day, but he threatens Harold with a grand jury appearance if the boys aren't soon playing. The mayor then turns his attentions to Marian and he asks her for the book. Marian, grateful to Harold for Winthrop's new found joy and confidence, secretly rips out the relevant page of the book before handing it to Mayor Shinn.

At the young people's insistence, Marcellus leads the crowd in a new dance Harold has taught them ("Shipoopi"); even Harold and Marian join in the fun. The dance ends when Mayor Shinn objects to Tommy dancing with Zaneeta. He turns to Harold and again demands his credentials. Marian, who has now warmed to Harold, thanks him for defending Tommy. Marian invites Harold to call on her to explain the Think System. The ladies, impressed with Marian after seeing her dance with Harold, ask her to join their committee. They also mention that, at Harold's suggestion, they've read Chaucer, Rabelais and Balzac and adored them all ("Pickalittle – Reprise").

Winthrop returns home from fishing and sings for his mother and sister the song Harold has just taught him ("Gary, Indiana"). He happily runs into the house, singing the Minuet in G, followed by Mrs. Paroo. Charlie Cowell, the traveling salesman, arrives and asks Marian for directions to the mayor's house. He mentions he has information about Harold Hill's dishonest past, but only has a few minutes in town to deliver that information before his train leaves. To protect Harold, Marian tries to delay Cowell by flirting with him. She kisses him just as the train whistle begins to blow. As he realizes what she's done, he angrily runs off to catch the train, telling her she is but one of a long line of women who have fallen for Harold.

After Cowell leaves, Harold arrives; he begins to talk about the Think System, but Marian asks him to explain what Cowell has said. Harold tells her not to believe rumors about traveling salesmen because they are the product of jealousy. Marian agrees, telling him the rumors about her and Mr. Madison are also the product of jealousy. Harold then asks Marian to meet him at the Footbridge, a favorite lover's meeting place. She accepts. After Harold leaves, she tells her mother she has accepted his invitation; Mrs. Paroo remarks that the Think System, which she's been using on Harold and Marian, really works.

The Ladies Auxiliary Committee is finishing its Grecian Urn tableau as the mayor enters with Charlie Cowell. Cowell tells the townspeople about Harold's plan to leave town with their money without providing lessons for the boy's band. The mayor sends the townspeople off to find Harold. Winthrop runs away, stunned by the news that Harold Hill cannot lead a band.

Marcellus shows up looking for Harold at the Footbridge and he suggests Harold catch the last freight train, which leaves town in a little over an hour. Marian meets Harold and, when they are alone, she confesses her love for him ("Till There Was You"). She also tells him she has known all about his phony credentials for weeks. And as a final loving gesture, she gives Harold the page she removed from the reference book. Marcellus rushes in, holding Harold's suitcase in one hand. Marcellus pleads with Harold to hurry to the waiting horse and buggy, but Harold doesn't move.

Winthrop angrily asks if Harold can lead a band. Harold truthfully tells him he can't. He explains he wanted Winthrop in the band because it was a way to get Winthrop to stop feeling sorry for himself. Marian tells Winthrop that Harold has offered the town a reason to be happy. She also tells the boy she's glad Harold came to River City as the constable and the townspeople arrive and Harold is put in handcuffs.

The mayor suggests tarring and feathering, but Marian defends Harold, reminding the crowd of the excitement and joy Harold has brought to River City. The mayor then asks if anyone objects to tarring and feathering Harold; the constable, the Ladies Auxiliary Committee (including the mayor's wife), the school board, the mayor's daughter and Mrs. Paroo all step forward. The mayor reminds the crowd of Harold's promise to teach the boys to play and, as he demands to know where the band is, the boys all enter in uniform and line up in band formation with their instruments. Harold pleads with the boys to think and gives the upbeat. Miraculously, they are able to play a barely recognizable Minuet in G. The townspeople, including the mayor, are all thrilled; all the parents proudly call to their sons. The mayor shakes Harold's hand and the crowd cheers; the play ends as Marian and Harold embrace.