Cuijiao Zhang


In modern society, clothing has gradually become an important beauty-enhancing product, playing an important role in human’s social life. Nevertheless, each outfit is composed of items of many different types (e.g. top, bottom, shoes) that share some complicated relationship with one another, how to make a suitable outfit has become a daily headache for many people, especially those who do not have much sense of aesthetics. In recent years, the booming fashion sector and its huge potential benefits have attracted tremendous attention from many research communities. Moreover, deep learning has garnered considerable recognition by researchers in many research fields such as computer vision and natural language processing, owing not only to stellar performance but also the attractive property of learning feature representations from scratch. The influence of deep learning is also pervasive, recently demonstrating its effectiveness when applied to information retrieval and outfit recommender systems research. In the light of this, many research efforts have been dedicated to the task of complementary clothing matching and have achieved great success relying on the advanced data-driven neural networks. However, finding suitable architectures, the selection and extraction of clothing features is a challenge for researchers. This article aims to provide a comprehensive review of recent research efforts on deep learning based clothing matching. More concretely, we first review the research on the clothing matching system and categorized deep learning-based clothing matching methods into two categories, namely collaborative method and content-based method. We then carry out a detailed introduction to this relevant research on deep learning based clothing matching.


Clothing Matching; Deep Learning; Survey

Full Text:



Liu, S., Feng, J., Song, Z., Zhang, T., Lu, H., Xu, C., & Yan, S. (2012). Hi, magic closet, tell me what to wear!. acm multimedia.

Sha, D., Wang, D., Zhou, X., Feng, S., Zhang, Y., & Yu, G. (2016). An Approach for Clothing Recommendation Based on Multiple Image Attributes. web age information management.

Liu, Q., Wu, S., & Wang, L. (2017). DeepStyle: Learning User Preferences for Visual Recommendation. international acm sigir conference on research and development in information retrieval.

Lu, Z., Hu, Y., Jiang, Y., Chen, Y., & Zeng, B. (2019). Learning Binary Code for Personalized Fashion Recommendation. Computer Vision and Pattern Recognition.

Heinz, S., Bracher, C., & Vollgraf, R. (2017). An LSTM-Based Dynamic Customer Model for Fashion Recommendation. arXiv: Information Retrieval,.

Covington, P., Adams, J., & Sargin, E. (2016). Deep Neural Networks for YouTube Recommendations. conference on recommender systems.

Zhang, S., Yao, L., Sun, A., & Tay, Y. (2019). Deep Learning Based Recommender System: A Survey and New Perspectives. ACM Computing Surveys, 52(1).

Jagadeesh, V., Piramuthu, R., Bhardwaj, A., Di, W., & Sundaresan, N. (2014). Large scale visual recommendations from street fashion images. knowledge discovery and data mining.

Mcauley, J., Targett, C., Shi, Q., & Den Hengel, A. V. (2015). Image-Based Recommendations on Styles and Substitutes. international acm sigir conference on research and development in information retrieval.

Sonie, O., Chelliah, M., & Sural, S. (2019). Concept to Code: Deep Learning for Fashion Recommendation. the web conference.

Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780.

Wu, C., Ahmed, A., Beutel, A., Smola, A. J., & Jing, H. (2017). Recurrent Recommender Networks. web search and data mining.

Li, Y., Cao, L., Zhu, J., & Luo, J. (2017). Mining Fashion Outfit Composition Using an End-to-End Deep Learning Approach on Set Data. IEEE Transactions on Multimedia, 19(8), 1946-1955.

Hu, Y., Yi, X., & Davis, L. S. (2015). Collaborative Fashion Recommendation: A Functional Tensor Factorization Approach. acm multimedia.

Iwata, T., Watanabe, S., & Sawada, H. (2011). Fashion coordinates recommender system using photographs from fashion magazines. international joint conference on artificial intelligence.

Han, X., Wu, Z., Jiang, Y., & Davis, L. S. (2017). Learning Fashion Compatibility with Bidirectional LSTMs. acm multimedia.

He, R., Packer, C., & Mcauley, J. (2016). Learning Compatibility Across Categories for Heterogeneous Item Recommendation. international conference on data mining.

Veit, A., Kovacs, B., Bell, S., Mcauley, J., Bala, K., & Belongie, S. (2015). Learning Visual Clothing Style with Heterogeneous Dyadic Co-Occurrences. international conference on computer vision.

Rendle, S., Freudenthaler, C., Gantner, Z., & Schmidtthieme, L. (2009). BPR: Bayesian personalized ranking from implicit feedback. uncertainty in artificial intelligence.

Gao, G., Liu, L., Wang, L., & Zhang, Y. (2019). Fashion clothes matching scheme based on Siamese Network and AutoEncoder. Multimedia Systems, 25(6), 593-602.

Song, X., Han, X., Li, Y., Chen, J., Xu, X., & Nie, L. (2019). GP-BPR: Personalized Compatibility Modeling for Clothing Matching. acm multimedia.

Liu, J., Song, X., Chen, Z., & Ma, J. (2019). Neural fashion experts: I know how to make the complementary clothing matching. Neurocomputing,, 249-263.

Han, X., Song, X., Yao, Y., Xu, X., & Nie, L. (2020). Neural Compatibility Modeling With Probabilistic Knowledge Distillation. IEEE Transactions on Image Processing,, 871-882.

Song, X., Feng, F., Liu, J., Li, Z., Nie, L., & Ma, J. (2017). NeuroStylist: Neural Compatibility Modeling for Clothing Matching. acm multimedia.

Kipf, T., & Welling, M. (2016). Variational Graph Auto-Encoders.. arXiv: Machine Learning,.

Den Berg, R. V., Kipf, T., & Welling, M. (2017). Graph Convolutional Matrix Completion. arXiv: Machine Learning,.

Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2019). A Comprehensive Survey on Graph Neural Networks. arXiv: Learning,.

Hamilton, W. L., Ying, R., & Leskovec, J. (2017). Representation Learning on Graphs: Methods and Applications. IEEE Data(base) Engineering Bulletin,, 52-74.

Cai, H., Zheng, V. W., & Chang, K. C. (2018). A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications. IEEE Transactions on Knowledge and Data Engineering, 30(9), 1616-1637.

Cui, P., Wang, X., Pei, J., & Zhu, W. (2019). A Survey on Network Embedding. IEEE Transactions on Knowledge and Data Engineering, 31(5), 833-852.

Pan, S., Wu, J., Zhu, X., Zhang, C., & Wang, Y. (2016). Tri-party deep network representation. International Joint Conference on Artificial Intelligence.

Bromley, J., Guyon, I., Lecun, Y., Sackinger, E., & Shah, R. (1993). Signature Verification using a "Siamese" Time Delay Neural Network. neural information processing systems.

Zagoruyko, S., & Komodakis, N. (2015). Learning to compare image patches via convolutional neural networks. computer vision and pattern recognition.

Yue, Y., Wang, C., Elarini, K., & Guestrin, C. (2014). Personalized collaborative clustering. the web conference.

Shen, E. Y., Lieberman, H., & Lam, F. (2007). What am I gonna wear?: scenario-oriented recommendation. intelligent user interfaces.

Weston, J., Wang, C., Weiss, R., & Berenzweig, A. Latent Collaborative Retrieval, available at : , accessed 2012.

Adomavicius, G., & Tuzhilin, A. (2005). Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6), 734-749.

Han, X., Song, X., Yin, J., Wang, Y., & Nie, L. (2019). Prototype-guided Attribute-wise Interpretable Scheme for Clothing Matching. international acm sigir conference on research and development in information retrieval.

Yang, X., He, X., Wang, X., Ma, Y., Feng, F., Wang, M., & Chua, T. (2019). Interpretable Fashion Matching with Rich Attributes. international acm sigir conference on research and development in information retrieval.

Feng, Z., Yu, Z., Yang, Y., Jing, Y., Jiang, J., & Song, M. (2018). Interpretable Partitioned Embedding for Customized Multi-item Fashion Outfit Composition. international conference on multimedia retrieval.

Song, X., Feng, F., Han, X., Yang, X., Liu, W., & Nie, L. (2018). Neural Compatibility Modeling with Attentive Knowledge Distillation. international acm sigir conference on research and development in information retrieval.

Cucurull, G., Taslakian, P., & Vazquez, D. (2019). Context-Aware Visual Compatibility Prediction. computer vision and pattern recognition.

Gori, M., Monfardini, G., & Scarselli, F. (2005). A new model for learning in graph domains. international joint conference on neural network.

Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009). The Graph Neural Network Model. IEEE Transactions on Neural Networks, 20(1), 61-80.

Bruna, J., Zaremba, W., Szlam, A., & Lecun, Y. (2013). Spectral Networks and Locally Connected Networks on Graphs. arXiv: Learning,.

Defferrard, M., Bresson, X., & Vandergheynst, P. (2016). Convolutional neural networks on graphs with fast localized spectral filtering. neural information processing systems.

Kipf, T., & Welling, M. (2016). Semi-Supervised Classification with Graph Convolutional Networks. arXiv: Learning,.

Monti, F., Bronstein, M. M., & Bresson, X. (2017). Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks. neural information processing systems.

Kipf, T., & Welling, M. (2016). Variational Graph Auto-Encoders.. arXiv: Machine Learning,.

Yu, W., Zhang, H., He, X., Chen, X., Xiong, L., & Qin, Z. (2018). Aesthetic-based Clothing Recommendation. the web conference.

Dunlavy, D. M. (2011). All-at-once Optimization for Coupled Matrix and Tensor Factorizations.. arXiv: Numerical Analysis,.

Vasileva, M. I., Plummer, B. A., Dusad, K., Rajpal, S., Kumar, R., & Forsyth, D. (2018). Learning Type-Aware Embeddings for Fashion Compatibility. european conference on computer vision.

Liu, Z., Luo, P., Qiu, S., Wang, X., & Tang, X. (2016). DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations. Computer Vision and Pattern Recognition.

Liu, S., Song, Z., Wang, M., Xu, C., Lu, H., & Yan, S. (2012). Street-to-shop: cross-scenario clothing retrieval via parts alignment and auxiliary set. ACM Multimedia.

Kalantidis, Y., Kennedy, L., & Li, L. (2013). Getting the look: clothing recognition and segmentation for automatic product suggestions in everyday photos. International Conference on Multimedia Retrieval.

Lin, Y., Ren, P., Chen, Z., Ren, Z., Ma, J., & De Rijke, M. (2019). Improving Outfit Recommendation with Co-supervision of Fashion Generation. The Web Conference.

Huynh, C. P., Ciptadi, A., Tyagi, A., & Agrawal, A. (2018). CRAFT: Complementary Recommendations Using Adversarial Feature Transformer. arXiv: Computer Vision and Pattern Recognition,.

Polania, L. F., & Gupte, S. (2019). Learning fashion compatibility across apparel categories for outfit recommendation. arXiv: Computer Vision and Pattern Recognition,.

Simoserra, E., Fidler, S., Morenonoguer, F., & Urtasun, R. (2015). Neuroaesthetics in fashion: Modeling the perception of fashionability. computer vision and pattern recognition.

Wang, X., Wu, B., & Zhong, Y. (2019). Outfit Compatibility Prediction and Diagnosis with Multi-Layered Comparison Network. acm multimedia.

Yamaguchi, K., Kiapour, M. H., Ortiz, L. E., & Berg, T. L. (2012). Parsing clothing in fashion photographs. computer vision and pattern recognition.

Yamaguchi, K. , Kiapour, M. H. , & Berg, T. L. . (2013). Paper doll parsing: retrieving similar styles to parse clothing items. IEEE, 3519-3526.

Liu, S., Feng, J., Domokos, C., Xu, H., Huang, J., Hu, Z., & Yan, S. (2014). Fashion Parsing With Weak Color-Category Labels. IEEE Transactions on Multimedia, 16(1), 253-265.

Khurana, T. , Mahajan, K. , & Arora, C. . (2018). Exploiting Texture Cues for Clothing Parsing in Fashion Images. IEEE International Conference on Image Processing. IEEE.

Ihsan, A. M. , Loo, C. K. , Naji, S. A. , & Seera, M. . (2020). Superpixels features extractor network (sp-fen) for clothing parsing enhancement. Neural Processing Letters(11).


  • There are currently no refbacks.

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright © 2019 International Educational Applied Scientific Research Journal