No, milk is not bad for athletes unless an athlete has a milk protein allergy or suffers from lactose intolerance. Milk contains high-quality protein, carbohydrate calcium, magnesium, phosphorous, and vitamin B12, which are involved in bone health, recovery from exercise, metabolism, and various other critical physiological functions .
Specifically, milk is rich in leucine, an amino acid that appears to stimulate muscle protein synthesis and enhance recovery after a workout . In addition, milk also offers readily-absorbed carbohydrate to help replenish muscle glycogen stores after exercise. Some research suggests that chocolate milk is especially beneficial for recovery because it offers a unique combination of carbohydrate and protein, as well as fluid .
That said, milk is not critical for athletes to perform at their best, especially if an athlete has a milk protein allergy or lactose intolerance. For these athletes, consuming milk products can be not just uncomfortable but also, dangerous.
 Thomas, D. T., Erdman, K. A., & Burke, L. M. (2016). Position of the Academy of Nutrition and Dietetics, Dietitians of Canada, and the American College of Sports Medicine: Nutrition and Athletic Performance. Journal of the Academy of Nutrition and Dietetics, 116(3), 501–528. https://doi.org/10.1016/j.jand.2015.12.006
 Ferguson-Stegall, L., McCleave, E., Ding, Z., Doerner Iii, P. G., Liu, Y., Wang, B., Healy, M., Kleinert, M., Dessard, B., Lassiter, D. G., Kammer, L., & Ivy, J. L. (2011). Aerobic exercise training adaptations are increased by postexercise carbohydrate-protein supplementation. Journal of nutrition and metabolism, 2011, 623182. https://doi.org/10.1155/2011/623182