AI with a Focus on Semantic Technologies Set to Advance Digital Stationery Applications

The DSC is continuing to make strides toward establishing digital ink as a universal and openly used format. Recently, DSC members Wacom and German Research Center for Artificial Intelligence (DFKI) announced a new proof of concept (PoC) to advance how AI, especially semantic technologies, can provide knowledge workers with a wider range of digital stationery applications, from note-taking to collaboration services.

The new PoC is made possible by combining technology and research from two of the DSC’s Technology Work Group members: Wacom, DFKI and former member MyScript. The new PoC uses WILL™ (Wacom Ink Layer Language) and DFKI’s evolutionary corporate memory system called “CoMem” (https://comem.ai) to semantically analyze digital handwritten text instantly and contextually to offer smart services. The PoC integrates MyScript’s handwriting recognition technology for text and graphics segmentation as well as text recognition. MyScript technology’s usage could be extended to address other use cases, such as the recognition of mathematical expressions or diagrams traditionally used in education-based note-taking.

Figure 1: By combining research around AI, especially semantic technologies, the new PoC enables smart services, such as note-taking, which requires semantic technology, digital ink text recognition and digital ink graphics recognition to work (image courtesy of Wacom).

By distinguishing between handwritten text, hand-drawn diagrams or doodles, the new PoC can provide smart services for knowledge workers. First, the PoC identifies the digital ink data created by each handwritten word, letter and symbol. Then, the system interprets whether the data is a date, contact, location, topic, diagram or another type of concept. Once the system recognizes the context of the digital ink data, the PoC can then offer the user a wide range of knowledge services, from scheduling the next meeting with the right collaboration group to providing online links to all the related concepts listed in the group’s notes.

The PoC helps users build personal knowledge graphs, known as the Personal Information Model (PIMO), which can be created for a team, group or entire organization. The PoC can compare handwritten script against existing personal and cross-company knowledge graphs – from terminology to objects – to make note-taking easier, quicker and more intuitive and efficient in the following ways:

Charts and Equations: Hand-drawn charts and equations include prompts that offer to turn these into digital graphics
Location: Online access to an address, directions or map is added when the system recognizes the location as a place
Organizations: A hyperlink to the company’s website or other information is included once the system recognizes that the name is an organization
Events: Semantic understanding of an “event” would trigger prompts such as adding the dates to the user’s calendar, inviting others to attend or providing a hyperlink to go online to learn more information
Topics or Entities: Personal topics, contacts, projects, processes and concepts within a user’s notes can be semantically interlinked and embedded in personal as well as enterprise knowledge graphs. Users can glance at the description provided, choose to open the link to browse more in-depth information or decide to view the information later within the knowledge graph and more

Looking ahead Wacom, DFKI and other members of the DSC, including Montblanc, Samsung, E Ink, and Fujitsu Client Computing Limited, among others, will continue to seek out new ways for AI to further enhance WILL’s universal digital ink technology, and to provide real-world benefits to the digital stationery market.