A rhyme scheme is the pattern of rhymes at the end of each line of a poem or song. It is usually referred to by using letters to indicate which lines rhyme; lines designated with the same letter all rhyme with each other.
An example of the ABAB rhyming scheme, from "To Anthea, who may Command him Anything", by Robert Herrick:
Bid me to weep, and I will weep | A | |
While I have eyes to see | B | |
And having none, yet I will keep | A | |
A heart to weep for thee | B |