Controlled environment agriculture, particularly hydroponic cultivation, has emerged as a critical solution to global food security challenges, enabling year-round crop production with reduced water consumption and land requirements. However, traditional hydroponic systems rely on static nutrient schedules and rule-based environmental controls that fail to account for dynamic plant responses and environmental variability. Reinforcement learning algorithms offer a paradigm shift by enabling adaptive, data-driven decision-making that optimizes nutrient delivery, climate control, and resource allocation in real-time. This review examines the application of reinforcement learning methodologies, including Q-learning, Deep Q-Networks, policy gradient methods, and actor-critic algorithms, in developing intelligent hydroponic systems that integrate sensor networks, actuator control, and closed-loop optimization frameworks. Key applications include adaptive nutrient dosing, dynamic pH and electrical conductivity regulation, light spectrum optimization, and multi-objective yield enhancement. The synthesis of experimental validations demonstrates significant improvements in crop biomass, resource efficiency, and operational cost reduction compared to conventional approaches. Critical challenges encompassing computational complexity, hardware integration, reward function design, and scalability for commercial deployment are discussed. The review concludes that reinforcement learning represents a transformative technology for next-generation precision agriculture, with ongoing advances in edge computing, transfer learning, and hybrid control architectures poised to accelerate widespread adoption in commercial hydroponic operations.