Leaf++, augmented reality for leaves

Ancora sulla realtà aumentata e la città… raccolgo e giro…

 

Leaf++, augmented reality for leaves

at

ISEA 2011 in Istanbul
DEOL 2011 in Helsinki
Mindtrek 2011 in Tampere

Leaf++ is a research project by FakePress Publishing, Art is Open Source and the University of Rome "La Sapienza" in which an ubiquitous social and performative ecosystem is built on leaves using augmented reality and computer vision techniques.

In Leaf++ a collaborative environment available on mobile devices allows people to take pictures of leaves in natural environments and to use them to train a distributed computer vision system which allows to visually recognize the various types of leaves.

Leaves in the environment thus become naturally available markers that can be used to disseminate knowledge and information: text, sounds, videos and interactive experiences can be attached to the various types of leaves and become instantly accessible to everyone using Leaf++.

We are presenting several use cases for this system:

– a knowledge environment through which information about plants can be attached to plants themselves and accessed through augmented reality, thus realizing a tool for education and scientific research

– an application dedicated to the concept of Third Landscape defined by Gilles Clément, in which augmented reality is used to train our vision into spotting, recognizing and gaining awareness of the interstitial gardens disseminated in our cities, their importance for biodiversity and their role in urban ecology; Leaf++ also acts as a realtime emergent mapper of the Third Landscape, allowing us to research the evolution of plant ecosystems in our cities;

– a tool for urban awareness focusing on the idea that plants have long become part of a peripheral part of the vision of urban dwellers, who have lost contact with the knowledge associated with them, including seasonalities, uses, traditions and origins;

– a disseminated social network in which user interactions take place "on leaves";

– a tool for artistic performance, in which the computer vision system trained by the ubiquitous community to recognize leaves allows performers to actually "play leaves" in an "augmented reality concert for leaves";

– a series of experiments in which the possibility to use the visual profile of leaves to generate other media is used to create solutions to aid people with disabilities, for example by being able to recognize leaves by hearing them;

All these use cases will be presented under various forms at the various venues and publications: a presentation across arts and sciences at ISEA; a description of the possibilities to create ubiquitous social applications using this kind of technology, at Mindtrek; a workshop/performance at DEOL to present the possibilities for education, knowledge and performance.

All the software is Open Source and is distributed under a GPL license by FakePress Publishing and Art is Open Source.

The research has been performed by Salvatore Iaconesi, Luca Simeone, Oriana Persico and Cary Hendrickson, with substantial support provided by the long list of organizations, collectives, events and publications which you will find listed at the project's website.

We are currently searching to extend the Leaf++ team, to further explore our research directions, and to apply these technologies to new ones. If you are part of a research group which is interested in these themes (augmented reality and computer vision technologies used at the intersection of arts and sciences to create ubiquitous publishing, knowledge/narratives dissemination and natural interaction experiences) please feel free to contact us.

More information is constantly updated on the project page here:

http://www.artisopensource.net/category/projects/leaf-plusplus/