Compressed ideographs -visualized-

Exhibited at ICC (2022)


 This work is an interactive installation that allows viewers to experience the creation and visualization of kanji characters by actually inputting characters in a new series of kanji characters named “Compressed ideographs”, which are created by DALL-E, a deep learning model that differs from any of the methods used to create the six kanji characters (hieroglyphs, fingerspelling, kaiyi, phonetic, transcriptions, and pseudonyms) created in history.
 Since the second century, kanji characters have been created and classified into six categories (Rikusho) according to their origins. Kanji characters themselves are still being created for newly discovered elements, for example, but they are created by people using existing methods. In today’s increasingly complex and diverse world, is it possible to explain the world using only kanji characters created using conventional methods? In this work, we used a deep learning model to create a seventh category, which we named “Compressed ideographs”, which can be applied to any text.
 We used a transformer model called DALL-E to generate the kanji characters, and the authors trained it on a large number of pairs of kanji characters and sentences describing the meaning of the kanji characters. In this way, for any string or sentence input by the viewer, a kanji character is generated that is compressed into a single character. At the same time, arbitrary character strings and sentences entered by the viewer are vectorized into 300 dimensions by the Doc2Vec model trained by the authors, and then their location in the 3D space created by the dimensionality reduction algorithm UMAP is calculated. The newly generated kanji characters are then placed together with a huge amount of existing kanji characters in a 3D space that represents the meaning of strings and sentences. In addition, the relationship between the two is visualized by displaying the kanji characters that are closest in meaning to the existing kanji characters, and by randomly displaying a large number of similarities to the existing kanji characters.
 Through the experience of plotting kanji characters that reflect complex features in the meaning of the text by AI, viewers can explore the gap between characters that have been created and fixed by humans and those generated by AI.

 Compressed ideographs -visualized-は,表意文字(1字1字が一定の意味を表す文字, ideographs)が語と意味の多様化に順応した文字体系を持つために,テクノロジーによって新たな記号の作り方を模索する作品です.


September 25th, 2021.


Scott Allen (Direction, Machine learning, Visual programming)
Keito Takaishi (Machine learning, Visual programming)
Asuka Ishii (Machine learning, Video)
Kazufumi Shibuya (Machine learning, Visual programming)
Yuma Matsuoka (Visual programming)
Atsuya Kobayashi (Sound)
Nao Tokui (Technical advice)
Keio University SFC Computational Creativity Lab (Nao Tokui Lab)


やまなしメディア芸術アワード2021 優秀賞
アジアデジタルアート大賞展FUKUOKA 一般/インタラクティブアート部門 入賞


NeurIPS 2022 “Machine Learning for Creativity and Design”


Video by Asuka Ishii
Yuri Nakahashi (Cast)
NTT InterCommunication Center