Classical optimization models are applicable for portfolio optimization when capital market constraints and portfolio construction criteria are limited. However, given the existing complexities in capital markets, intricate relationships, and non-linear issues, classical models exhibit significant weaknesses and limitations. To address and solve such problems, computers and Machine Learning (ML)-based models have emerged to assist human decision-makers. Recently, techniques based on hybrid ML models have been designed, demonstrating robust capabilities in handling non-linear problems. This paper proposes a hybrid ML model for portfolio optimization based on specific historical data in the capital market. For this purpose, daily returns from 20 high-liquidity companies listed on the Tehran Stock Exchange (TSE) over a specific timeframe were utilized. The proposed framework integrates Random Forest (RF) for feature importance analysis and Extreme Gradient Boosting (XGBoost) for return prediction, combined with a Deep Reinforcement Learning (DRL) agent using Proximal Policy Optimization (PPO) for dynamic asset allocation. Portfolio performance is evaluated across four distinct risk metrics: Variance, Mean Absolute Deviation (MAD), Semi-Variance, and Conditional Value at Risk (CVaR). Based on evaluations of portfolio return, risk indices, and other key indicators, the results of this research indicate that the performance of the proposed ML model is at an optimal and desirable level.