Japanese telecommunications firm NTT announced a series of research projects that could pave the way for better AI and more energy efficient data centers.
At a press conference in San Francisco, NTT researchers said the company has created a new large language model (LLM) integration that can see and process graphical elements of documents. It also said it has initiated a new field of science, “the physics of intelligence,” to study sustainable and trustworthy AI. It teamed up with Harvard University to study brain science.
And it successfully demoed its new all-photonic network for distributed data centers.
The latter was announced by Yosuke Aragane, NTT’s vice president of the IOWN Development Office. He said that separating large data centers into different locations in suburbs can reduce costs and improve energy efficiency when using NTT’s fast fiber connections.
NTT has more than 330,000 employees and $97 billion in annual revenue. It invests more than $3.6 billion in annual research and development, and five years ago it created an R&D division in Silicon Valley. At its Upgrade 2024 event, the firm talked about its progress in R&D in San Francisco.
“Our mission, our task is that we upgrade what you think as normal to the next level,” said Kazu Gomi, president and CEO of NTT Research, in press conference.
NTT low-latency network in U.S. and U.K.
Aragane said that NTT’s IOWN All-photonics network (APN) demonstration has achieved very low communication delay for connected data centers, with applicability for AI analysis and financial services.
He noted that the biggest challenges around data centers in urban areas are the high cost and shortage of land in downtown areas of cities, as well as high costs of electricity in those areas. NTT’s researchers are experimenting with spreading the data centers out into the suburbs and connecting them with fiber-optic cables delivering data at 100 or 400 gigabits a second.
In the United Kingdom, NTT showed that its data centers that were 100-kilometers apart had less than one millisecond of network delay using APN connections.
The APN-connected data centers in U.K. and U.S. reduce delay variation to a tiny fraction of what prevails in conventional networks.
And NTT said network performance can unite geographically distributed data centers, addressing real-estate constraints and green-energy requirements.
In the U.K., NTT connected data centers north and east of London via NTT’s Innovative Optical Wireless Network (IOWN) APN, and communication between them was realized with a round-trip delay of less than 1 millisecond. In the U.S., data centers in Northern Virginia achieved similar results.
The goal of this initiative is to transform geographically distributed IT infrastructure into the functional equivalent of a single data center.
The data center market is under severe local constraints. Carbon dioxide emission restrictions and land
shortages have made it difficult to build data centers in urban areas, forcing operators to turn to the
suburbs. Yet with geographically distant data centers, delay of communication, or latency, can be very
high, making it difficult to meet customers’ needs for low latency.
In separate demonstrations, NTT and NTT DATA connected data centers in the U.K. (HH2 in Hemel Hempstead and LON1 in Dagenham) and in the U.S. (VA1 and VA3 in Ashburn) using APN equipment from NEC. These U.K. and U.S. data centers are 89km and 4km apart, respectively.
Measurements in tests conducted over 100 Gbps and 400 Gbps links showed the two APN-connected data centers in the U.K. operated with less than a millisecond (approximately 0.9 milliseconds) of latency, and with a delay variation (sometimes called jitter) of less than 0.1 microseconds.
According to cloud connectivity provider Megaport, typical delay between data centers at an equivalent distance exceeds 2,000 microseconds (two milliseconds). In the U.S. case, delay over the much shorter span was approximately 0.06 milliseconds; and delay variation was less than 0.05 microseconds.
By contrast, conventional networks with general Layer 2 switches experience delay variation of several microseconds to tens of microseconds. In other words, the APN cuts latency in half, and jitter by orders of magnitude.
The APN delivered very low latency required by current and emerging use cases. These include distributed, real-time AI analysis, such as industrial IoT and predictive maintenance, smart surveillance
systems, smart grid and energy management, natural disaster detection and response and more.
NTT DATA is also conducting demonstrations in the financial sector, where low latency is required for remittances, settlements and transactions. Another advantage of the IOWN APN is that it enables line
activation simply by adding wavelengths, without needing to install new dark fiber. As a result, data center operators can respond very quickly to customer demand.
“The demand for data centers is increasing, but two challenges stand in the way of building new facilities. The first is a lack of land inside cities where data centers can be placed,” Aragane said. “And sometimes we cannot acquire enough electricity for the client. That’s why we would like to utilize more useful or more energy efficient data centers.”
In addition, concentrating all the resources for a data center in one location can represent a security risk. He said NTT wants to use data centers for the next generation of AI, but it has to all be much more energy efficient.
Asked about the flaws in the current “dark fiber,” Aragane said that NTT needs to use NTT’s APN fiber networks in order to reach the speeds necessary. I asked about the cost savings that result with distributed data centers. He noted they’re working on lower costs and that improving the bandwidth will result in lower the costs each year, and clients can share a single pipeline of data to save money.
NTT’s AI tool for understanding images and graphics visually
NTT also demonstrated a new visual machine reading comprehension technology that enables large language models (LLMs) to understand the graphical elements and layout of documents, including diagrams, graphs and icons.
Developed in collaboration with Jun Suzuki, a professor at Tohoku University’s Center for Data-driven Science and Artificial Intelligence, NTT researchers have applied this technology to NTT’s proprietary, lightweight LLM, tsuzumi.
When tested against 12 different visual document understanding tasks including answering questions, extracting information and classifying documents based on human-written instructions, NTT’s model outperformed the open-sourced multimodal LLM LLaVA as well as OpenAI’s GPT-3.5 and GPT-4.
“LLMs have become capable of handling high-level natural language processing tasks with high accuracy, and multimodal models, including those that integrate vision and language, are beginning to emerge,” said Kyosuke Nishida, a senior distinguished researcher at NTT, in a statement. “However, there remain significant challenges in comprehending documents or computer screens that contain both text and visual information, such as charts and tables.”
“By integrating our visual machine reading comprehension technology with tsuzumi, NTT aims to give
‘eyes’ to AI-powered tools, unlocking new applications and functionality,” said Nishida.
First unveiled by NTT in November 2023, tsuzumi is an energy efficient and low-cost LLM available in two versions: an ultra-lightweight version with 600 million parameters and a lightweight version with seven billion parameters.
While traditional LLMs require large amounts of power for training, tsuzumi’s smaller size drastically reduces the energy consumption and costs associated with training, inference, and tuning, making it a more sustainable and cost-effective option for businesses.
Researchers envision four initial, primary use cases for enterprise deployment of tsuzumi and the visual machine reading comprehension technology, including: customer experience solutions including call center automation; employee experience solutions for tasks involving manual searching and reporting, including electronic medical recordkeeping in the healthcare industry; transforming the value chain for industries including life sciences and manufacturing; and software engineering for systems and IT departments, including development and coding assistance and automation.
Tsuzumi currently supports over 20 languages, including English and Japanese, and programming languages. NTT is currently conducting commercial trials of tsuzumi and has consulted with over 500
companies from around the world on the potential introduction of the technology into their systems.
NTT publicly demonstrated its visual machine reading comprehension technology in the United
NTT Research gives $1.7M for program with Harvard Center for Brain Science
Gomi said that NTT Research announced that the NTT Research Foundation, a 501(c)3 organization, has made a gift to establish the Harvard University Center for Brain Science (CBS)-NTT Fellowship Program in the new field of Physics of Intelligence.
The two-year gift, renewable for up to three more years, creates a fund that supports post-doctoral research in the physics of intelligence, an emerging field that uses physics to tackle fundamental questions in intelligence, bridging computer science, neuroscience and psychology. If renewed, the donation could total over $1.7 million.
Harvard retains full control over the administration of the fund, which can be used to support two post-doctoral researchers, technology, facilities, travel, guest speakers for seminars and meetings, and other associated costs.
This new program will amplify themes that have emerged through a pre-existing relationship between Harvard CBS and the NTT Research Physics & Informatics (PHI) Lab. Under a 2021 joint research agreement, the two organizations undertook shared research into natural and artificial intelligence.
Physicists and AI researchers from the PHI Lab’s Intelligent Systems Group, led by Hidenori Tanaka, who is also a CBS associate, have established fruitful alliances with neuroscientists and psychologists at Harvard. In collaboration with CBS, Tanaka and PHI Lab Intern and CBS Affiliate Ekdeep Singh Lubana published a paper at ICML 2023 addressing bias in AI using insights from cognitive science.
More recently, the team delivered two papers at NeurIPS 2023, addressing the science of generative AI (diffusion models) and the application of AI to neuroscience (recurrent neural networks). The former, co-authored by PHI Lab Scientist Maya Okawa, who is also in residence at CBS, and Lubana revealed insights into how generative AI systems learn to imagine, and includes an alert about the costs of fairness.
The latter, co-authored by Stanford University Ph.D. Candidate and PHI Lab Intern Fatih Dinc proposed an algorithm, based on recorded neural activity in mice, that yielded training speeds 100 times faster than traditional approaches. Tanaka has also collaborated with Harvard Assistant Professor of Psychology Tomer Ullman, also affiliated with CBS, on the behavioral patterns of large language models (LLMs) to be published at ICLR 2024.
While an associate at CBS, PHI Lab Scientist Gautam Reddy published papers with CBS Director Venkatesh Murthy on feedback into hierarchically organized sensory systems and adaptive algorithms for shaping behavior. Now an assistant professor at Princeton University, Reddy also published a notable paper on “discontinuous learning.”
“We are thrilled to support the Harvard Center for Brain Science at the dawn of the Physics of Intelligence,” NTT Research President and CEO Kazuhiro Gomi said. “As history teaches us, inventions – consider the steam engine, electricity, liquid crystals – can lead to new fields in physics. Today, AI is playing that role and giving us an opportunity to explore fundamental questions involving the science of intelligence, as well as address some urgent practical problems, like the need for computational systems that are unbiased, trustworthy and green. We believe this gift will advance those shared goals.”
The Harvard CBS is pursuing an ambitious mission. Scientists affiliated with the Harvard CBS study the structure and function of neural circuits; how they give rise to computations that govern thought and behavior; how those circuits change, develop and vary; and what circuits can become amiss or disordered, yet potentially ameliorated.
“We’re grateful to the NTT Research Foundation for this generous gift, which comes on top of several years of unique intellectual contributions and collaboration,” Harvard CBS Director Murthy said, in a statement. “The Harvard CBS looks forward to developing ideas around the Physics of Intelligence through fellowships, seminars, talks and other activities that both enhance and align with our diverse approach to neuroscience.”
Shaping this ongoing academic cross-pollination is the dramatic rise of powerful AI, which provides a new
subject of study for the science of intelligence. “The interdisciplinary environment at Harvard’s Center for Brain Science has been crucial in nurturing this emerging field, which harnesses physics-based approaches to investigate foundational questions about intelligence,” PHI Lab Scientist and Harvard CBS Associate Tanaka said. “By conducting quantitative experiments and theoretical modeling involving AI systems, which are more accessible to testing and measurement than the human brain, we aim to uncover the universal mathematical principles that underlie intelligence in all its forms, both natural and artificial.”
NTT in Silicon Valley
“NTT Research is a little bit unique as an organization in Silicon Valley,” Gomi said. “We have an office set up in Sunnyvale, California, basically the heart of Silicon Valley. Something unique about the research is that we are engaged in basic fundamental research. Our mandate is to choose fundamental important topics and then dig down into it.”
In five years, NTT Research has published over 450 academic papers. And it has won seven awards for the best paper in a conference over the five years. Researchers have won awards for optics, physics and cryptography. It is working with 15 different cooperation partners at research departments.
The researchers are doing work in quantum simulations to come up with a new paradigm for computer systems. There are more than 20 researchers working on that alone. They’re coming up with things like new materials for photonics integrated circuits.
“We believe that the new use this technology will open the new door of photonics integrated circuit in a way that we should be able to get through photonic space computers someday,” Gomi said.
Another group is studying the brain and what kind of computation it does. Another group has been working upon something called physics of intelligence. And it has teamed up with Harvard to do brain science. The team has 10 researchers in its cryptography lab and how to device security systems that can resist the anticipated quantum computing attacks. NTT plans on bringing a new encryption tech to the commercial markets.
And Gomi said there are teams working medical informatics, creating “digital twins” of the heart and other body systems. By creating simulated hearts, he said it might one day be possible to simulate your reaction to drugs in order to figure out how much of a dose to give you in real life.
Once the digital twin is done, he said, “We are looking at a next step, which is the autonomous closed innovation system, which means that this this dopamine system can precisely represent your heart. So once that is done, you can control that how much dosage of drugs you should inject into your body, to bring your body back to normal as quickly as possible.”