Fujitsu chosen for GENIAC project, starts development of large language models for logical reasoning

• Enhancing reliability of generative AI, accelerating application of generative AI in business operations

Kawasaki, Japan – Fujitsu today announced that it has been chosen for the “Research and Development Project of the Enhanced Infrastructures for Post-5G Information and Communication Systems/Development of post-5G information and communication systems,” a subsidy project launched by Japan’s New Energy and Industrial Technology Development Organization (NEDO) under Japan’s Ministry of Economy, Trade and Industry’s (METI) Generative AI Accelerator Challenge (GENIAC) initiative to enhance Japan’s capability to develop generative AI ⁽¹⁾. Under the subsidy project, Fujitsu will start research and development on technology combining knowledge graphs with large language models to realize LLMs that enable logical reasoning.

Fujitsu has been conducting R&D of generative AI technologies for application in business operations. On May 10, Fujitsu in cooperation with a team of researchers announced the release of Fugaku-LLM ⁽²⁾, a large language model with enhanced Japanese language capability, and plans to continue to strengthen the development of specialized LLMs for various industries and businesses.

As part of its R&D activities, Fujitsu has been focusing on the problem posed by hallucinations in large language models (LLMs), a phenomenon in which generative AI creates plausible but incorrect or unrelated output. Fujitsu aims to realize a new technology that strengthens the reliability of LLMs by combining LLMs with knowledge graphs, ultimately enabling the application of LLMs in operations that require high levels of compliance and explainability, including tort determination and basis search in the legal field, internal control and accounting audits in finance, and symptom search and diagnostic support in the medical field. Fujitsu anticipates that the newly developed technology will enable the realization of an LLM for logical reasoning that creates clear and comprehensible outputs.

Fujitsu plans to offer the new technology to the Japanese market at the end of fiscal 2024, as a step to further enhance its technologies to protect conversational AI from hallucinations and adversarial attacks announced in September 2023 ⁽³⁾.

LLMs that enable logical reasoning

To address the task of reliability in the output of LLMs, Fujitsu started development of LLMs specialized in knowledge graph generation and inference, one of the knowledge processing technologies, to make LLMs logically infer answers according to the knowledge graph generated from natural language regulations and rules. Fujitsu plans to position this technology as the core technology of a generative AI trust technology that eliminates output instability and ensure accurate output in compliance with large and complex regulatory and internal regulations, a technique Fujitsu aims to realize in fiscal 2024 as part of its company-wide AI strategy ⁽⁴⁾.

As part of the GENIAC program, Fujitsu will develop two specialized LLMs: an LLM for knowledge graph generation, which converts natural language documents into knowledge graphs to form knowledge, and an LLM for knowledge graph inference that searches for relevant information on knowledge graphs for a given question, aggregates it logically, and answers it.

To efficiently develop two specialized LLMs in a limited development period, Fujitsu will first develop a pre-trained LLM that is common to both specialized LLMs. The advantage of this LLM development is that it enables LLMs to handle both documents in natural language and knowledge graphs by adding a bilingual corpus ⁽⁵⁾ to the pre-learning data. Specialized LLMs are then developed simultaneously by carrying out instruction learning ⁽⁶⁾ for knowledge graph generation for the first model, and instruction learning for knowledge graph inference for the second model.

Future Plans

Moving forward, Fujitsu will offer the specialized LLMs for knowledge graph generation and inference, evaluation scripts for knowledge graph generation and inference tasks, and other knowledge and know-how acquired during this project via Hugging Face ⁽⁷⁾, GitHub ⁽⁸⁾, the Fujitsu blog, and the GENIAC community in compliance with terms of use, OSS licenses, and copyright of the data and software used. Fujitsu further plans to release the new technologies via Fujitsu Kozuchi, Fujitsu’s dedicated AI service that allows users to accelerate the testing and deployment of advanced AI technologies.

[1] Business overview :
Project name: Research and Development Project of the Enhanced Infrastructures for Post-5G Information and Communication Systems/ Development of post-5G information and communication systems (Subsidy project)
Project overview :
Research and Development Project of the Enhanced Infrastructures for Post-5G Information and Communication Systems

[2] “Release of ‘Fugaku-LLM’ – a large language model trained on the supercomputer ‘Fugaku’”
(Press release, May 10, 2024)

[3] “Fujitsu launches new technologies to protect conversational AI from hallucinations and adversarial attacks”
(Press release, September 26, 2023)

[4] “Fujitsu AI strategy strengthens data integration, generative AI capabilities with dedicated platform and new Fujitsu Uvance offerings”
(Press release, February 14, 2024)

[5] Bilingual Corpus :
A dataset for machine translation that combines sentences from different languages, such as Japanese and English, in the form of translations

[6] Instruction learning :
A system in which instructions and example answers to a new task are given to an LLM in a series of sentences for learning.

[7] Hugging Face :
Platform for worldwide AI data set sharing.

[8] GitHub :
A platform used worldwide to publish open source software.