The man who was China’s AI Prometheus fell

The passing of Tang Xiaoou on December 16, 2023, marks the end of an era in the field of AI and computer vision in China.

Though Tang Xiaoou might not be a household name globally like Baidu or ByteDance, his influence and contributions to the AI landscape, particularly in China, have been profound and far-reaching. His legacy is intertwined with the growth and development of many of China’s renowned AI scientists.

Tang Xiaoou and his venture, SenseTime, played a pivotal role reminiscent of the historical “Whampoa Military Academy” in China. This academy is famed for training many of China’s modern military leaders, some of whom eventually joined the ranks of the Nationalist Party’s rivals, leading to the academy’s downfall. In a similar vein, SenseTime, under Tang’s guidance, has been a cradle for AI talent in China, propelling the country’s internet industry into the AI era. However, this widespread proliferation of AI expertise and applications across various internet companies did not directly translate into benefits for SenseTime itself.

Tang’s death in 2023 brought a renewed recognition of his monumental role in the AI industry – a ‘Prometheus’ of modern times. His vision and leadership not only shaped a leading company in AI but also fostered a generation of innovators and thinkers who have pushed the boundaries of technology and its applications.

As we delve into the story of Tang Xiaoou, we uncover a narrative of brilliance, perseverance, and a relentless pursuit of innovation that has indelibly marked the landscape of artificial intelligence, both within China and globally. His journey, from a visionary academic to a trailblazing entrepreneur, encapsulates the spirit of an era where technology and human ingenuity converge, reshaping our world in ways previously unimaginable.

01 The Unexpected Guests

In the autumn of 1997, nestled in the easternmost part of the Teaching Building 3 in the West District of the University of Science and Technology of China (USTC), the Information Processing Center welcomed two unexpected visitors.

One of them was Li Xuelong, a student from the USTC’s class of ’94. As a senior, he was preparing for postgraduate studies within the same university. Seeking a mentor, Li arrived at the Information Processing Center, aiming to familiarize himself with research under the guidance of senior students and then join the team of Professor Liu Zhengkai, the center’s director, specializing in image processing.

Founded in 1988, the Information Processing Center (IPC) was part of the Department of Electronic Engineering and Information Science (also known as the 6th Department). It had become a hub for students like Li, often referred to as “IPCr’s,” and was known for its vibrant research atmosphere.

Upon his arrival at the center, Li recognized many familiar faces, but one person stood out. He was stylish, spoke with a sense of humor, and wasn’t from the 6th Department – he was Tang Xiaoou.

At that time, Tang Xiaoou, in his early thirties, had just returned from a seven-year academic stint in the United States, still relatively unknown in the academic world. The handover of Hong Kong in 1997 led to a series of policies aimed at attracting talented professionals from the U.S. to work in Hong Kong. Having graduated from the Massachusetts Institute of Technology (MIT) in the U.S., Tang chose to join The Chinese University of Hong Kong.

Before taking up his new position, Tang wished to revisit USTC, his alma mater, leading him to Hefei and, subsequently, to the Information Processing Center. The center, primarily focused on visual information retrieval and pattern recognition, shared several technical aspects with Tang’s interest in facial recognition, prompting frequent exchanges.

During these interactions, Tang met many juniors from the 6th Department, including Li Xuelong. Li was an active member of the center. Although Tang’s USTC background was in Precision Machinery and Precision Instrumentation, not directly related to the 6th Department, he and Li exchanged many ideas during his visit.

After his visit, Tang went on to start his tenure at The Chinese University of Hong Kong, while Li continued his research at the Information Processing Center. What many didn’t anticipate was how this seemingly ordinary academic visit would mark the beginning of a rapid development in computer vision at The Chinese University of Hong Kong, a journey sparked by the meeting of Tang Xiaoou and Li Xuelong.

Tang Xiaoou joined the Department of Information Engineering at The Chinese University of Hong Kong (CUHK). During his Ph.D. at MIT, he delved into machine vision and cutting-edge areas like facial recognition, hoping to continue his deep research in these fields.

The Information Engineering Department at CUHK was established by the Chinese-American scientist and 2009 Nobel Laureate in Physics, Charles K. Kao. Initially focused on communications, including coding, optical, and wireless communication, the department remains a global academic stronghold in the field.

When Tang started his teaching career, he coincided with the burgeoning development of the global internet, witnessing the rise of a hot new field: “multimedia”. Around 1997, “multimedia” was as buzzworthy as AI is today. With communication infrastructure in place, there was growing interest in the content being transmitted, such as images. However, image processing was a weak area for CUHK’s Information Engineering Department, which Tang aimed to strengthen. In July 2001, he established the “Multimedia Laboratory” (MMLab), focusing on system performance and facial recognition.

Initially, MMLab had few experts in image processing, except for a postdoc named Liu Jianzhuang, who partnered with Tang. Liu had just graduated from CUHK and, after a stint at Nanyang Technological University in Singapore, returned to CUHK in 2000 and is now at Huawei’s Noah’s Ark Lab.

At that time, Tang was experiencing the anxieties common to young academics. In Hong Kong, where Cantonese is the primary language and most students are locals, he lacked connections, funding, and students. The establishment of MMLab was a significant gamble, but Tang decided to take the risk.

In 2001, despite Hong Kong’s economic and cultural prosperity and its greater internationalization compared to mainland cities, the best graduates of Hong Kong’s top universities planned to further their studies in the U.S. Tang, despite his persuasive skills, struggled to attract top local students to his research. Thus, he thought of a “roundabout” solution: inviting Li Xuelong to visit.

Li accepted Tang’s invitation and spent half a year at MMLab, which broadened his horizons. At 25, during his time at CUHK, Li learned about CVPR, a top international conference in computer image processing, and realized the importance of international academic exchange, a concept previously unfamiliar to him at USTC.

After completing his Ph.D., Li chose to first teach at universities in the UK, to engage with international frontiers of knowledge, before returning to China – a decision influenced by his visit. Teaching at the University of Ulster and then the University of London, he introduced cognitive computing into the field of information and secured a tenured position at the University of London within just four years. By 2009, when he returned to China as one of the first group of returning professors, he was already a renowned computer scientist internationally.

Later, Li’s achievements in the field of vision earned him recognition as an IEEE Fellow in 2011 and a National Outstanding Young Scientist in China. He was elected as an ACM Fellow in 2020. Now, he teaches at Northwestern Polytechnical University, nurturing many outstanding AI talents and undertaking significant research projects.

Importantly, after Li’s visit and subsequent return to the mainland, many students from the USTC Information Processing Center, inspired by his experience, became interested in Tang’s lab and hoped to pursue their studies in Hong Kong. This interest from students like Wang Xiaogang, Tao Dacheng, Lin Dahua, and Cao Liangliang ended Tang Xiaoou’s recruitment difficulties at CUHK.

02 Wang Xiaogang and Tao Dacheng

If we turn back the clock to 2001, the decision of a brilliant student like Wang Xiaogang to join a newly established and then obscure laboratory seems almost inconceivable. In this context, Li Xuelong’s influence played a significant role.

Wang was part of the USTC’s prestigious class of 2000 (9600). At that time, gaining admission to USTC was an achievement in itself, and being part of the 00 class meant you were among the elite of the elite: Education in China wasn’t as hierarchically rigid back then, and many students who could have attended top universities like Tsinghua or Peking University chose USTC instead (for example, Liu Qingfeng, the chairman of iFlytek). Generally, those who made it into the 00 class were among the top three USTC applicants from each province every year.

Moreover, Wang Xiaogang was one of the most outstanding students in his class. During his time at USTC, he received the highly coveted Guo Moruo Scholarship, an award every USTC student aspires to, with a prize of 2,000 yuan. Another USTC alumnus who received this honor is Deng Li, the first person globally to successfully apply deep learning in industry.

At the time, Li Xuelong was pursuing his Ph.D. at USTC’s Information Processing Center and maintained close relationships with everyone there. Wang Xiaogang, also conducting research at the center, caught Li’s attention. Aware of Tang Xiaoou’s desire to recruit talented students to Hong Kong for research, Li recommended Wang.

Tang, a fellow USTC alumnus, quickly recognized Wang Xiaogang’s potential and promptly persuaded him to join the MMLab, which had been established less than two months prior.

A year later, Li Xuelong recommended two more classmates from the 9706 batch, Xu Dong and Tao Dacheng, to go to Hong Kong. Tao Dacheng, in particular, had an outstanding foundation in mathematics and demonstrated strong innovative and programming skills.

Xu Dong shared with Leifeng.com that at that time, the hottest field in China was actually telecommunications. Many students aspired to join labs specializing in telecom technologies, lured by the prospect of high-paying internships at companies like UTStarcom, known for producing Little Smart (Xiaolingtong) phones, where they could earn as much as three thousand yuan a month. AI, on the other hand, was considered a niche field, and concepts like facial recognition research or CVPR, which Tang Xiaoou was working on, were virtually unknown to them.

After hearing Li Xuelong’s introduction to CUHK and Tang Xiaoou, Tao Dacheng was intrigued, while Xu Dong was less inclined to make the move and chose to stay at USTC until his visit to MMLab in 2004.

Wang Xiaogang and Tao Dacheng enrolled in the two-year Master of Philosophy (MPhil) program at The Chinese University of Hong Kong (CUHK), with Wang starting in 2001 and Tao in 2002. Under Tang Xiaoou’s guidance, they focused on two key areas: self-directed research skills and publishing in top-tier conferences, such as CVPR, ICCV, and ECCV in the field of computer vision.

This emphasis on publishing at leading conferences was a mindset Tang developed during his Ph.D. at MIT. He believed that research was akin to a martial arts competition; to prove one’s worth, one must compete at the most prestigious forums. Even if one achieved fame presenting at lesser-known venues, it wouldn’t compare to the recognition from competing with the world’s top teams.

Inspired by Tang, the lab members were deeply committed to high-level research. During his master’s, Wang Xiaogang alone printed several boxes of research papers. His solid work ethic and diligence resulted in publishing five papers at CVPR and ICCV over two years.

Their efforts paid off spectacularly. In 2004, MMLab published seven papers at CVPR, with Wang Xiaogang, Tao Dacheng, and Li Zhifeng (from the class of 9400) each having two papers accepted. This was a notable achievement, considering CVPR’s global acceptance rate of just over 300 papers that year.

Globally, very few students published at top conferences, so upon completing their master’s, both Wang and Tao received offers from prestigious institutions for their doctoral studies. In 2004, Wang went on to join MIT’s Vision Group, while Tao was accepted at the University of Cambridge. However, Tao ultimately chose to pursue his Ph.D. at Birkbeck College, University of London, under the tutelage of the renowned computer vision scholar, Professor Stephen J Maybank.

After completing his studies in the U.S., Wang returned to work closely with Tang Xiaoou at MMLab, elevating the lab to new heights. Together, they co-founded SenseTime and Wang became the president of SenseTime’s Research Institute, playing a pivotal role in advancing the commercial application of computer vision technology in China.

Tao, after completing his Ph.D. in the UK, joined the Hong Kong Polytechnic University as an assistant professor. Before joining JD.com, he served as an Australian Laureate Fellow at the University of Sydney and was elected a fellow of the Australian Academy of Science before turning 40. He is currently the inaugural dean of the JD Explore Academy and a senior vice president at JD Group.

During their time at MMLab, they established a culture of aiming for top conference deadlines, pulling all-nighters before submissions, and extensively reviewing scientific literature. For instance, when Lin Dahua joined CUHK, Tang handed him several boxes of papers left by Wang before he went to MIT. Lin set up a rigorous reading schedule, aiming to read a set number of papers daily. By the end of his two-year master’s program, he had reviewed over 2000 papers, covering virtually all the significant works in computer vision at the time, thus laying a solid foundation in the field.

03 Tang Xiaoou’s Departure, Yan Shuicheng’s Arrival

In 2004, Yan Shuicheng arrived at The Chinese University of Hong Kong (CUHK) in what could be described as an ‘unexpected’ turn of events.

Yan Shuicheng was the second-highest scorer in the 1995 college entrance examination in Hengyang, Hunan. He was admitted to Peking University’s Department of Mathematics, where he completed his bachelor’s, master’s, and doctoral studies consecutively. Starting in 2001, Yan began an internship at the Media Computing Group of Microsoft Research Asia (MSRA), working under Zhang Hongjiang. It was during this time that he met and became acquainted with Tang Xiaoou, who often visited MSRA.

In 2002, Tang co-authored a paper with Gao Xinbo (President of Chongqing University of Posts and Telecommunications) and Zhang Hongjiang (President of the Institute for AI Fundamental Research) in the IEEE Transactions on Neural Networks titled “A Spatial-Temporal Approach for Video Caption Detection and Recognition,” which led to a connection with MSRA.

Upon completing his Ph.D. in 2004, Yan was considering a postdoctoral position at an overseas university. At the same time, Tang Xiaoou had secured funding of 6 million Hong Kong dollars from the Hong Kong Innovation and Technology Commission for a facial recognition project. Simultaneously, Tang was invited by Zhang Hongjiang and Shen Xiangyang to lead the Visual Computing Group at MSRA. Needing someone to lead the project at CUHK, Tang reached out to Yan Shuicheng.

Consequently, Yan Shuicheng joined CUHK as a postdoctoral researcher, focusing on (3D) facial recognition.

Yan Shuicheng was the first student from Peking University or Tsinghua University to join MMLab. Before his arrival, most of MMLab’s students came from USTC. After Tang Xiaoou’s visits to MSRA, he began recruiting students from Beijing’s top universities, such as He Kaiming, leading to MMLab being colloquially referred to as the “Tsinghua-Peking branch.”

Within a few months, Xu Dong also visited MMLab, followed by Liu Qingshan, Tang Ming, Lin Dahua, Cao Liangliang, and others. After Yan Shuicheng joined, he brought along his classmate from Peking University, Xu Chunjing. Xu Chunjing initially joined as a research assistant and later pursued his Ph.D. He is now the director of Huawei’s Noah’s Ark Lab’s Computer Vision Laboratory. The first from MMLab to move to Huawei was Liu Jianzhuang, followed by Zhang Wei, Li Zhengguo, and others.

The addition of Yan Shuicheng, Xu Dong, and others was crucial for MMLab’s continued development, especially since Tang Xiaoou was often required to stay in Beijing due to his engagement with MSRA. Although research in Hong Kong was more advanced than in mainland China at the time, it still lagged behind U.S. institutions. Tang, then an associate professor, saw MSRA as a branch of the global tech giant Microsoft, and aligning with MSRA meant striving for global excellence.

The research culture at MMLab was similar to MSRA, emphasizing paper publication and burning the midnight oil to meet conference deadlines, making Yan Shuicheng and Xu Dong well-suited for the environment.

Yan Shuicheng and Xu Dong were close collaborators during their internships at MSRA. Xu Dong worked under Zhang Lei, while Yan Shuicheng worked with Zhang Hongjiang. Zhang Lei’s team was part of Zhang Hongjiang’s larger group, leading Xu Dong to collaborate with Yan Shuicheng in research. In one year, they jointly submitted seven papers to top conferences, with five being accepted, reflecting the close research collaboration between them.

During his two years at MMLab, Lin Dahua achieved remarkable success: he published seven papers during his master’s program and was admitted to MIT for his Ph.D., fulfilling his initial goal when he decided to pursue his studies at CUHK.

The conversation that persuaded Lin Dahua to study at CUHK was a typical example of a “Tang-style” meeting:

It was 2003, and Lin was on an exchange program at the Hong Kong University of Science and Technology (HKUST), where he developed an interest in computer vision under the influence of Quan Long. When Li Xuelong mentioned that Professor Tang Xiaoou at CUHK was looking for students to join his research, Lin wrote to Tang, who immediately arranged a meeting.

Lin’s journey from HKUST in Clear Water Bay to CUHK near Shenzhen’s Luohu took over an hour by shuttle bus and subway, leaving him exhausted. However, during their brief meeting, Tang mentioned that Lin had a good chance of getting into MIT. This came as a shock to the 21-year-old Lin: MIT was the top institution globally, a place he hadn’t even dared to dream of, yet Tang casually suggested the possibility.

“Professor Tang is really an excellent HR in the development of his entire career,” Lin later commented with a laugh.

Tang spent more than half of his time attracting top talent. Lin told Leifeng.com that Tang’s method of persuasion wasn’t about explaining what someone could do in his team, but rather about painting a picture of an incredibly attractive future and providing a clear and feasible path to achieve it.

It’s said that Tang used a similar approach when persuading young talents like Wang Xiaogang and He Kaiming to join his team. He would persistently pursue his chosen candidates, continuing his efforts until they agreed or until it was no longer possible.

By bringing these talented individuals together and fostering a sense of personal ambition and career progression, Tang ensured that the team consistently produced outstanding results.

Lin was certainly convinced and didn’t hesitate much before deciding to go to Hong Kong for his studies. Later, just like Wang Xiaogang, Lin went on to MIT for his Ph.D. and became a key figure in continuing the development of MMLab. He is now the director of MMLab.

Returning to the stories of Yan Shuicheng and Xu Dong, their time at MMLab coincided with a critical juncture in their careers as they were transitioning from doctoral studies to academic positions. Their visit to Hong Kong added an ‘international flavor’ to their resumes. After a six-month stint at MMLab, Xu Dong returned to MSRA for another nine months before moving to Columbia University in the United States for a postdoctoral position. Similarly, Yan Shuicheng went to the University of Illinois at Urbana-Champaign (UIUC) for his postdoc, working under the mentorship of Huang Xutao, a pioneer in the field of vision from the Chinese community.

When it came time to pursue academic positions after completing their research in the United States, both Yan and Xu initially focused their job search on Hong Kong and Singapore, which were geographically close to Hong Kong. Huang Xutao supported this decision, advising them to ‘Return to Asia’ since they were not native Ph.D. graduates from the United States.

Xu Dong accepted an offer from Nanyang Technological University (NTU) in Singapore, later moved to the University of Sydney, and eventually returned to Hong Kong to work at the University of Hong Kong. Yan Shuicheng initially planned to join Xu Dong at NTU but eventually accepted an offer from the National University of Singapore. Later in his career, whether working at 360, Yitu Technology, or Shopee, Yan contributed significantly to the development of numerous outstanding Chinese researchers.

04 The Rise of Deep Learning

MMLab began to gain widespread recognition, with computer vision researchers around the globe becoming aware of this laboratory situated in northern Hong Kong. Lin Dahua shared with Leifeng.com that when he attended CVPR in 2005 to give a presentation, despite being just a master’s student, many people already knew of him and Tang Xiaoou.

However, it’s important to note that at this time, Tang Xiaoou’s fame and status were still not as prominent as they are today, and MMLab, when compared to top-tier labs at institutions like MIT and Stanford, was not yet at the same level.

The real turning point came in 2009. That year, two significant events elevated MMLab’s status: First, a paper co-authored with MSRA won the first CVPR Best Paper award for an Asian institution (with He Kaiming as the first author), bringing widespread acclaim. Second, Wang Xiaogang returned to collaborate with Tang Xiaoou, helping MMLab to timely embrace the burgeoning field of deep learning. This pivot not only propelled the lab forward but also made it and its associated members a central topic in discussions about the history of computer vision in China.

That same year, Tang Xiaoou was elected an IEEE Fellow, one of the highest honors in the field of electrical engineering, acknowledging his achievements in pattern recognition and video processing.

Tang Xiaoou’s decision to join MSRA, akin to his prompt decision to buy a house in Beijing the day after accepting the offer to lead the Visual Computing Group at MSRA, showcased his strategic foresight. MSRA served as a pivotal connection point for Tang with mainland China, where he discovered talented students like He Kaiming and Cui Jingyu. After leaving MSRA in 2009, Tang joined the Shenzhen Institutes of Advanced Technology of the Chinese Academy of Sciences (SIAT), leveraging its substantial computational resources to help MMLab become one of the earliest teams globally to successfully implement deep learning.

Upon Wang Xiaogang’s return to CUHK in 2009, although he joined the Department of Electronic Engineering, he maintained a close relationship with Tang Xiaoou and MMLab. The two found a synergistic way to guide students: Tang identified cutting-edge research directions, while Wang led the students in executing these projects. This collaboration led to a surge in productivity and influence among AI researchers.

During this period, MMLab’s doctoral students included Luo Ping, Ouyang Wanli, Zhou Bolei (currently at UCLA), Zhao Cong (the first AI lead at DJI), Xu Bing (co-founder of SenseTime), and others. Research assistants like Zhao Deli (a leading figure in Alibaba’s DAMO Academy’s vision team) also contributed, with the team growing to about forty members.

Turning back to deep learning, Tang Xiaoou first heard about it through MSRA. In 2009, Chinese scientist Deng Li, working with Geoffrey Hinton—one of the three giants of deep learning—applied deep learning to large-scale speech recognition at Microsoft Redmond. They achieved in a short time what a large Microsoft team took months to do, causing a stir in the industry.

Sensing the potential of this new direction, Tang speculated that if deep learning could achieve remarkable results in speech, it could potentially do the same in vision. He immediately discussed this with Wang Xiaogang.

While Wang’s doctoral research at MIT focused primarily on probabilistic graphical models in computer vision, he was also interested in neural networks and firmly decided to explore them further.

Xu Dong told Leifeng.com that during his 2011 visit to Huang Xutao’s group at UIUC, he also heard about neural networks. He found the concept intriguing, but as he was focused on visual domain adaptation at the time, he didn’t have the bandwidth to delve into it until 2012 when AlexNet gained widespread recognition. This made him admire Tang Xiaoou and Wang Xiaogang’s early research instincts even more.

Initially, Tang Xiaoou and Wang Xiaogang selected two students, Ouyang Wanli and Luo Ping, to explore this less-trodden path in deep learning, while the rest of the team continued with their existing research.

Ouyang Wanli was part of the Department of Electronic and Information Engineering at CUHK and studied under the renowned video coding expert, Professor Zhan Weiwen. When Wang Xiaogang began his teaching career, he co-supervised some students with Zhan Weiwen, which is how he came to know Ouyang Wanli.

Luo Ping joined CUHK two years after Ouyang Wanli, in 2011. Before that, he studied at Sun Yat-sen University, where he conducted extensive research in computer vision under the guidance of Zhu Songchun (now the Dean of the School of Artificial Intelligence at Peking University) at the Lianhua Mountain Research Institute in Hubei. Even before starting his Ph.D., Luo had already published papers at top conferences like ECCV.

Luo Ping shared with Leifeng.com that initially, their computer vision research utilized Boltzmann machines, a precursor to deep learning. After joining CUHK, Tang Xiaoou assigned him to work on facial recognition. Initially, Luo planned to use Boltzmann machines for this project, but he eventually shifted to deep learning, which yielded better results. This prompted him to decisively change his research direction.

Wang Xiaogang played a pivotal role in MMLab’s exploration of deep learning. After Tang Xiaoou decided to venture into this field, Wang diligently promoted it, closely monitoring the research progress of Ouyang Wanli and Luo Ping, and collaborating with them to review relevant international papers and oversee research directions.

Typically, the interaction between Ph.D. students and their supervisors isn’t very frequent, but Luo Ping recalled that during his first year, he almost saw Wang Xiaogang every day. Wang, whose office was on the fourth floor, would often visit Luo on the seventh floor to discuss their work.

At that time, the conditions for researching deep learning were quite challenging. Early deep learning frameworks like Caffe had not yet appeared, and only a handful of teams globally could successfully implement deep learning from scratch. They initially wrote code in C++ and ran it on CPUs. Luo remembered completing his first deep learning paper for CVPR on a personal laptop.

Tang Xiaoou held a position at the Shenzhen Institutes of Advanced Technology, allowing the CUHK team to collaborate deeply with mainland scholars and secure funding for numerous CPUs. At that time, they hadn’t considered using GPUs, resulting in months-long experiments without promising results.

Wang Xiaogang, then an assistant professor under the tenure track system, faced considerable pressure. Nevertheless, they persevered, never considering giving up, accumulating a wealth of experience before the rise of AlexNet.

Being slightly ahead of others created a significant lead. From 2011 to 2013, MMLab published 14 deep learning research papers at ICCV and CVPR, accounting for half of the deep learning papers accepted at these top conferences globally (29 in total).

CUHK scholars became legendary in the global field of computer vision. Because of individuals like Tang Xiaoou and Wang Xiaogang, along with an increasing number of renowned scholars and talents at CUHK, the university consistently ranked in the top three, often first, in ‘Computer Vision’ in computer science rankings.

By 2014, CUHK’s research in deep learning for vision had transcended academic boundaries, showing real-world applicability and industrial potential. In March, the GaussianFace facial recognition algorithm achieved a 98.52% accuracy rate on the LFW database, surpassing human recognition rates for the first time. In June, the DeepID series of algorithms raised this accuracy to 99.55%, breaking through the threshold for practical application.

Capital markets, with their keen sense, quickly took notice. IDG’s Niu Kuiguang immediately flew to Hong Kong to meet Tang Xiaoou. After viewing several demos at MMLab, IDG swiftly invested a rare and substantial multi-million dollar angel round. This marked the dawn of China’s vision AI market: in October 2014, SenseTime was established.

05 Courage First, Fortune Follows

Lin Dahua returned to CUHK in August 2014. Following Wang Xiaogang, he was the second student to have graduated from MMLab, worked abroad, and then returned to teach at MMLab. After him, others like Ouyang Wanli, Luo Ping, and Zhou Bolei also spent various lengths of time at MMLab, witnessing its evolution into a stronghold of computer vision research in China.

Lin recalled to Leifeng.com that he decided to return to CUHK in 2013, at which point he was unaware of Tang Xiaoou’s entrepreneurial plans. After completing his Ph.D. at MIT, Lin worked at the Toyota Research Institute in the United States but was inclined towards academia. He held a deep trust in Tang Xiaoou, so during a New Year’s visit back to China, he made a special trip to Hong Kong to meet him.

Tang Xiaoou invited Lin Dahua and Wang Xiaogang for a meal, where they discussed the significant opportunities in both Hong Kong and mainland China, especially given the national investment in artificial intelligence. Beyond macro trends, Tang gave Lin some practical advice from his perspective: the most crucial aspect of being a university professor is research resources – students, funding, and the freedom to research.

Lin knew from his experiences interviewing at various U.S. universities that starting an academic career was challenging. For example, one of his friends from MIT had to write over a dozen grant proposals a year, almost one per month.

In addition to funding, establishing a source of good students was crucial. Having gone through this process himself, Tang’s advice resonated deeply with Lin. After this meal, Lin was almost decided on returning to CUHK.

Upon his return, MMLab was busy preparing for the establishment of SenseTime, and Lin fortuitously became part of the founding team.

People close to Tang Xiaoou told Leifeng.com that MMLab’s success and CUHK’s prominence in the global field of computer vision could not have been achieved without Tang’s emphasis on talent.

In 2016, when Ouyang Wanli was seeking an academic position, he initially struggled to find a job in Hong Kong. This was due to a tendency among Hong Kong universities to be reluctant to hire their own Ph.D. graduates.

Tang Xiaoou actively intervened when Ouyang Wanli struggled to find a teaching position in Hong Kong due to the local universities’ reluctance to hire their own Ph.D. graduates. Tang negotiated with the Dean of the Faculty of Engineering and Information Technologies at the University of Sydney to create a new position. He offered to fund half of Ouyang’s salary, with the university providing the other half, thus securing a place for Ouyang. During his time at the University of Sydney, Ouyang made significant research contributions.

In 2015, when Xu Dong was teaching at the University of Sydney and sought funding for a new project but faced repeated rejections, Tang Xiaoou, despite the financial constraints of SenseTime’s recent establishment, provided Xu Dong with approximately 100,000 AUD (about 450,000 RMB) to support his research.

Xu Dong summarized that Tang Xiaoou’s team has always been ahead in vision research: whether it was facial recognition or deep learning, they began exploring these fields when few others paid attention, allowing them to progress faster and further.

Even in the realm of generative AI, which has recently gained popularity, MMLab was an early explorer. Luo Ping told Leifeng.com that as early as 2012, he worked on generating frontal faces from profile images. In 2014, their work on face generation, which could take any facial angle and generate faces from any other angle, was published at NIPS (now NeurIPS). Interestingly, the generative adversarial network (GAN) model, often credited as the origin of generative AI, also emerged in 2014.

Luo recalled that at MMLab meetings with students from Wang Xiaogang’s department, each student had to succinctly summarize their project in one sentence. Tang Xiaoou’s requirement was that this sentence must be so precise that anyone listening could immediately assess the research’s value or the paper’s likelihood of acceptance.

Tang’s criteria for choosing research topics were simple: either pioneer a new direction or conclude an existing one. After completing his Ph.D., Luo Ping spent time at both MMLab and SenseTime before returning to the University of Hong Kong in 2019, where he continues to apply these standards in guiding students and conducting research.

Remarkably, Tang Xiaoou’s choice to walk less-traveled paths was joined by a cohort of USTC alumni like Li Xuelong, Wang Xiaogang, Tao Dacheng, Lin Dahua, Xu Dong, Cao Liangliang, and others, followed by Yan Shuicheng, Xu Chunjing, Qiao Yu, Li Zhengguo, Ouyang Wanli, He Kaiming, Zhou Bolei, and Luo Ping, who forged new paths.

Leading the way, they received the gifts of their era: most of those who emerged from MMLab achieved top scientific honors like becoming academicians, ACM Fellows, IEEE Fellows, and AAAI Fellows, all around the age of forty. They became key figures in AI development, leading vision AI at various companies.

Hong Kong, once a city many aspired to, may have lost some of its former glamour, but the lives shaped by the brief intersection of a generation with this city are everlasting. They navigated through uncertainty and continue to explore the unknown.