Hybrid portfolio optimization with pca, clustering, and the barzilai-borwein method

Portfolio optimization aims to balance risk and return by identifying an effective mix of assets. In this study, we integrate principal component analysis (PCA) and hierarchical clustering for stock selection with the Barzilai–Borwein (BB) gradient method for portfolio optimization. Forty-eight U.S. stocks from the Kaggle fundamental stock dataset were initially collected, and 42 stocks were retained after preprocessing. Financial ratios from 2006 and adjusted closing prices from 2016–2017 were analysed, with one representative stock from each cluster selected using the Sharpe ratio. The BB method was then applied to determine optimal weights, ensuring full capital allocation without short selling. Among the tested approaches, the Barzilai–Borwein gradient method 1 (BB1) step size achieved strong performance, producing an annual return of 25.6% while maintaining relatively low volatility. The portfolio also generated a Jensen’s alpha of 1.55, confirming the presence of positive abnormal returns beyond market expectations. These results suggest that combining PCA-based clustering with the BB optimization method offers a practical and efficient way to construct diversified portfolios. The study highlights the BB algorithm’s potential as a lightweight yet effective alternative to more complex optimization techniques in financial decision-making.