The five user number comes from the number of users you would need to detect approximately 85% of the problems in an interface, given that the probability a user would encounter a problem is about 31%. Most people either leave off the last part or are not sure what it means.  This does not apply to all testing situations such as comparing two products or when trying to get a precise measure of task times or completion rates but to discovering problems with an interface. Where does 31% come from?  It was found as an average problem frequency from several studies (more on this below).

via Why you only need to test with five users (explained): Measuring Usability.