No, it is not necessary to avoid dairy if you are lactose intolerant. International authorities such as the National Institutes of Health agree that most people who have lactose intolerance can eat small amounts of dairy foods without it causing discomfort.