Semi-Supervised Metric Learning in Stratified Space by Accurate Exploiting of Prior Knowledge
Subject Areas : electrical and computer engineeringZ. Karimi 1 , S. Shiry Ghidary 2 * , R. Ramezani 3
1 -
2 -
3 -
Keywords: Semi-supervised metric learningstratified spaceLaplacianconstraintsmoothness assumption,
Abstract :
Semi-supervised metric learning has attracted increasing interest in recent years. They enforce smoothness label assumption on the manifold. However, they suffer from two challenges: (1) since data in each class lies on one manifold and the similarity between classes leads the intersection between manifolds, the smoothness assumption on the manifold is violated in intersecting regions. (2) 1NN classifier, which is applied for predicting the label of classes in metric learning methods, is suffered from the rare of labeled data and has not suitable accuracy. In this paper, a novel method for learning semi-supervised metric in the stratified space has been proposed that exploit the prior knowledge, which is the smoothness assumption on each manifold, more accurate than existing methods. In the metric learning stage, it doesn’t apply smoothness assumption on the intersecting regions and in the classification stage, labeled data in the interior regions of manifolds are extended based on the smoothness assumption. The different behavior of the Laplacian of piecewise smooth function on stratified space is exploited for the distinction of the intersecting regions from interior regions of manifolds. The results of experiments verify the improvement of the classification accuracy of the proposed method in the comparison with other methods.
[1] J. Lu, X. Zhou, Y. P. Tan, Y. Shang, and J. Zhou, "Neighborhood repulsed metric learning for kinship verification," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 36, no. 2, pp. 331-345, Feb. 2014.
[2] J. Yu, M. Wang, and D. Tao, "Semisupervised multiview distance metric learning for cartoon synthesis," IEEE Trans. on Image Processing, vol. 21, no. 11, pp. 4636-4648, Nov. 2012.
[3] B. Kulis, "Metric learning: a survey," Foundations and Trends in Machine Learning, vol. 5, no. 4, pp. 287-364, 2012.
[4] Y. Ying and P. Li, "Distance metric learning with eigenvalue optimization," J. of Machine Learning Research, vol. 13, no. 1, pp. 1-26, Jan. 2012.
[5] L. K. Saul and S. T. Roweis, "Think globally, fit locally: unsuper-learning of low dimensional manifolds," the J. of Machine Learning Research, vol. 4, pp. 119-155, 12 Jan. 2003.
[6] J. B. Tenenbaum, V. De Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science, vol. 290, no. 5500, pp. 2319-2323, 22 Dec. 2000.
[7] Z. Zhang and H. Zha, "Principal manifolds and nonlinear dimensionality reduction via tangent space alignment," SIAM J. on Scientific Computing, vol. 26, no. 1, pp. 313-338, 2004.
[8] M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Computation, vol. 15, no. 6, pp. 1373-1396, 1 Jun. 2003.
[9] J. Wang, X. Sun, and J. Du, "Local tangent space alignment via nuclear norm regularization for incomplete data," Neurocomputing, vol. 273, pp. 141-151, 17 Jan. 2018.
[10] M. A. Carreira-Perpin and Z. Lu, "Manifold learning and missing data recovery through unsupervised regression," in Proc. IEEE 11th Int. Conf. on Data Mining, ICDM’11, pp. 1014-1019, Vancouver, Canada, 11-14 Dec. 2011.
[11] N. Pitelis, C. Russell, and L. Agapito, "Learning a manifold as an atlas," in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, CVPR’13, pp. 1642-1649, Portland, OR, USA, 23-28 Jun. 2013.
[12] X. Wang, T. Peter, and A. F. Mark, "Multiple manifolds learning framework based on hierarchical mixture density model," in Proc. Joint European Conf. on Machine Learning and Knowledge Discovery in Databases, pp. 566-581, Antwerp, Belgium, 15-19 Sept. 2008.
[13] K. Q. Weinberger and L. K. Saul, "Distance metric learning for large margin nearest neighbor classification," The J. of Machine Learning Research, vol. 10, pp. 207-244, 12 Jan. 2009.
[14] M. Der and L. K. Saul, "Latent coincidence analysis: a hidden variable model for distance metric learning," in Proc. 25th Int. Conf. on Neural Information Processing Systems, vol. 2, pp. 3230-3238, Lake Tahoe, NV, US, 3-6 Dec. 2012.
[15] S. C. Hoi, W. Liu, and S. F. Chang, "Semi-supervised distance metric learning for collaborative image retrieval and clustering," ACM Trans. on Multimedia Computing, Communications, and Applications, vol. 6, no. 3, pp. 18-43, Aug. 2010.
[16] W. Liu, S. Ma, D. Tao, J. Liu, and P. Liu, "Semi-supervised sparse metric learning using alternating linearization optimization," in Proc. Int. 16th Conf. on Knowledge Discovery and Data Mining, ACMSIGKDD’10, pp. 1139-1148, Washington, DC, US, 25-28 Jul. 2010.
[17] M. Baghshah and S. Shouraki, "Semi-supervised metric learning using pairwise constraints," in Proc. of the 21st Int. Joint Conf. on Artificial Intelligence, pp. 1217-1222, Pasadena, CA, US, 11-17 Jul. 2009.
[18] G. Niu, B. Dai, M. Yamada, and M. Sugiyama, "Information-theoretic semisupervised metric learning via entropy regularization," Neural Computation, vol. 26, no. 8, pp. 1717-1762, Aug. 2012.
[19] Q. Wang, P. C. Yuen, and G. Feng, "Semi-supervised metric learning via topology preserving multiple semi-supervised assumptions," Pattern Recognition, vol. 46, no. 9, pp. 2576-2587, Sept. 2013.
[20] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, John Wiley and Sons, 2001.
[21] L. Yang and R. Jin, Distance Metric Learning: A Comprehensive Survey, Michigan State Universiy, 2006.
[22] M. Belkin, Q. Que, Y. Wang, and X. Zhou, "Toward understanding complex spaces: graph laplacians on manifolds with singularities and boundaries," in Proc. 25th Annual Conf. on Learning Theory, COLT’12, pp. 1-26, Edinburgh, Scotland, 2012.
[23] Z. Karimi and S. Shiry Ghidary, "Semi-Supervised Classification in Stratified Spaces by Considering Non-interior Points Using Laplacian Behavior," Neurocomputing, vol. 239, pp. 223-231, 24 May 2017.
[24] L. Vandenberghe and S. Boyd, "Semidefinite programming," SIAM Review, vol. 38, no. 1, pp. 49-95, Mar. 1996.
[25] M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold regularization: a geometric framework for learning from labeled and unlabeled examples," J. of Machine Learning Research, vol. 7, pp. 2399-2434, Nov. 2006.