Artistic Interfaces for Social Exchange

Augmented Reality Panel Notes

2010-04-20T09:20:00.000-07:00

Media Lab brand of research:

Today I spoke with Pattie, and she told me how she is interested in more people working in the area of projected augmented reality because it has such a broad range of possibilities enabled by mobile media and mobile projection. She mention that keeping things simple is often key to making it work.

----
Critical mass of "projected augmented reality research"

Levelhead - playing a game with a virtual character that is in a virtual world.
(nice project reference)

Transparent Cockpit - very nice project that shows what is outside - by removing the physical objects that are in front of you.
(also see through people, the use of projection to go behind things - remove physical world)

Twinkle - interacting with the real world. Very nice project with augmenting the world around you.

-----

Augmented reality for remote augmentation

Guiding light project - for navigational assistant - embedding a compass or other device to get the orientation of the device. (location based interaction)

Bslam - (example project)

Pet as a navigation guide - virtual pet as a walking companion in the night.

Augmented reality pool...

----------

What is the robot used for information systems - Digital Desk to Sixth Sense..

Lamp exists in an ecosystem with other devices - wants to make it invisible.

BonFire - mounting the cameras behind the computer - uses computer vision to track hands etc..

Small form factor - eyeglasses vs. flash in the pan. Elitist system would emerge - social affordances of shared interface... INTERSTING Point

--------

Augmented Human Conference - find the proceedings and look at them..

Sleeping in Oragami

2009-07-07T09:11:00.001-07:00

I just want to make note of Robert Lang's amazing origami talk at TED. I wrote him today to say that someday I would like to build a canopy bed that folds and unfolds.

What if you would sleep inside a piece of origami, wouldn't that be beautiful?

seth

NOTES FROM Lucy Shuman- Agents of Interface

2009-05-11T13:13:00.000-07:00

Lucy Shuman is

What resources are available to humans that are not available to machines? Xerox park began to bring these questions into light 20 years.

Automata - A device for washing Hands, by Inb l-Rassa alJuzari, syria, 1206.

Reconfiguration - what kinds of functions are enabled by particular associations between people and things? It's our relationship to things that enables their function.

Redistribution of agency - anesthesiologist represents a transfer of agency from human to the infrastructure - machines act as extensions of the surgeons body - the complex relationship does not act to mediate, but increases the intimacy between the patient and the doctor.

The interface provides a deeper source of understanding where we interiorize the models into our understanding.

The zone - a disassociated state where the boundery of the machine and the user dissolves. (I live in this state all the time it seems)

Interface is not a place where two things come together - but a contextual point where many different perspectives of agency come together.

Alphred Gale - Art as Agency - at what point do machines appear to have human-like qualities?

Mix of alignment and slippage is the nature of our interactions with machines. Sherry Turkle calls machines "evocative objects" - what about evocotative relational interfaces?

Sha Xin Wei - Responsive environments like TGarden - software tracks gesture - by providing tand even thickening the senuous response, we make fertile the substrate for agency. The responsive system does not need to know what the human is doing - it continuously synthesizes responses to the gestures...

We need to locate our interfaces in a more extended network of relations - we draw bounderies in a way that make unavailable - she is advocating expanding the frame by which we consider the interactions.

If we recognize the agency of things - we can bring the human back out - by demystifying we encourage a re-enchantment in our relationships with artifacts.

"Agency is not an attribute but the ongoing reconfigurings of the world" - Karen Barad, 2007, Meeting the Universe Halfway..

Gesture is the analogue interface - we have come to think of responsiveness as having a congnitive model - but if we shift back - lets let machines be machines - lets see what the possibilities are for forms of responsiveness that doen't have this cognitive modeling.

Mutual constitution - the agency is in the "unit person-gun" - we cannot reduce thing to person or artifact - the responsibility lies in both things.

Transparancy is seen in terms of familiarity - what we see as obvious is a statement about our relationship to them. Thats what makes them transparent - this also points to the degree to which things are black-boxed - and the degree to which they are open.

Memory is a metaphor it's part the figural - aiding in the configuring of the device. Is memory a function of the brain, memorializing, socializing, and interaction. There is a kind of linkage and generative tethering between memory as it operates - it's very helpful to think about it as linked to human memory and other times its important to recognize it's differences.

What about agency as a moment in time to a particualar configuration whose elements are in flux? Agency is performative, dynamic, temporal - some things are pretty well established - and other things are more in flux -

Agency depends on context...

Exiting reconceptualization of relationships between humans and machines - it's about moving away from AI - and these trandtional narratives about the human - moving towards more human understandings the narratives - what are these new media? What are their dynamics that are distinctive, for creating material environments in which humans can engage in new ways. Participatory design projects, ongoing processes of reconfiguration - her agenda is to celbrate those ways of doing design...

If everyone is a designer - what is the role of the professional? It's create the realm of possibilities for people to design... you just have to keep telling stories... always looking at your audience and ajusting and elaborating multiple stories in industry. (she calles this recipient design)

We live in worlds where the thing that is valued is the fetishized object - a story about long term relationships and gradual transformation goes against the grain of the present discourse. We just have to keep domonstrating what the alternatives are

Relational Interfaces Brainstorm with Dan

2009-05-05T11:59:00.000-07:00

I met with Dan Paluska today to discuss relational interfaces, things which require the participation of 2 or more people, and this quickly led to a discussion with many forks. Most of them relate tangentially to developing a socially oriented aesthetic which tries to engender face to face interactions.

FORKS:
Group Organization
Giant Wheel Idea - the notion that if you get a critical mass of people to do something every once in a while (like turning a giant wheel) it brings the community together...

Along these lines groups like "Improv Everywhere", Flash Mob use the internet to organize large groups of people to participate in the SAME activity. There is great power in numbers, despite what the activity the people are participating in is..
(see also "Here Comes Everybody" by Clay Shirky for further reading)

Also mentioned the article "Hellhole" in the New Yorker, about the consequences of solitary confinement, our ideas are not real until shared or validated within a community.

Imitation and Synchrony:
Mirror Neurons - Both of us have researched the notion of imitation and synchrony in terms of shared experiences.

Normalized Synchrony Interface: Another way to explore this notion of seeing other people do the same thing as you is to create a series of Digital recording that are normalized to match your motion.

Interfaces which only work by Stillness can engender social awareness - how can we encourage or require stillness through an interaction?

Transforming Mirror - Dan mentioned that we have a "mirror face" and there is something about computational delay which gives us a more authentic view of our face. What kind of a mirror would tell us something about someone else - (as an aside, what if you made a transforming mirror that becomes someone else by using facial matching algorithms?)

I, Thou, It:
Culturally and contextually in our relationships it's helpful to have something to focus on outside of the people. An activity, context, thing, game - IT are the rituals and inventions - the food and landscapes that we wander through together. IT is the medium of a relational interface.

IT is Gemba: the "actual place" or "real place" of value in the interactions between people. Doing so in a "Kaizen" way - that which is always improving or holistically improving all aspects of our relations is one goal of a relational interface.

Open Software, Open Practice
Keeping secrets is killing us. One approach to work is to be open and fuck whatever people appropriate, and better if they do if your ideas are about making things. Tools and Ideas.

Venn Diagrams

2009-03-25T13:44:00.001-07:00

Venn diagrams seem like a rich area to explore in terms of relational aesthetics. :)
seth

Interfaces for relationships

2009-03-25T13:42:00.001-07:00

Pattie mentioned that there might be interfaces for reflection during a relationship to help people make better choices and to give families a way to contribute their thoughts to the framework. I suggested a kind of Life journal - which someone gets when they are born and have throughout their life - and cannot delete - it's always there, but somehow just for them - perhaps the password is your body in some sense, as a key to opening this repository. It could be something like an ongoing biography of your life. (very interesting note)

Recent Ideas I'd Prefer not to forget but never have time to follow up on.

2008-12-09T15:42:00.000-08:00

harmonograph for two people - the forms generated result from subtle collaboration and translate into sound as well.

Has anyone made an iphone app that recognizes bird calls?

Jaques Dudon's artworks were very inpsiring: http://aeh.free.fr/
I'd like to make a two person version of this.

Hallmark - 12 season cards (panoramic) - going there in January
Tappers that sync to your own music! (people would love these)
Cards as a key or token into digital worlds...

Collaboration for time based portion of the MemTable. Mark Watabe Jan 6

EndLighten Sheet - ordering visible Leds for an artwork printed on transparency

A utility surge protector built into the AC adaptor of a Mac Laptop? Has anyone done this?

Fiber Optic Textile weavings as feedback systems between people.

Tufts university proposal 2010 engendering community.

My other Blog: http://designingsociablemedia.blogspot.com/

Wikicon - Visual Icons Database?

2008-02-10T11:02:00.000-08:00

Has anyone ever created a visual database of icons in vector or scalable format? A collection of cultural images that by popular vote that are simple line drawings and shape forms immediately recognizable as modern hieroglyphics?

I'm looking for images with a uniform look and feel that could be substituted for words when people speak. What if I translated your words into images realtime? Would that process make you think of the content of your words in different terms? If such a database existed and could be accessed like a wiki - wikicon would be the title of the project....

I'd love to use such a thing to convert Speech - to text - to image - but skip the text part. It would be humorous - and perhaps profound at the same time...

I'm not the first to think of this apparently, see: Building a Visual Database for Example-based Graphics Generation

David Small Lecture Notes

2008-01-28T12:20:00.001-08:00

David Small came to visit the media lab today. I was particularly interested in meeting him because his work is the most expressive text based animation I've ever seen. He was direct and honest in his approach - the kind of candor that shows experience, but also privilege. Here are some highlights from his talk:

-Human beings are very good at scanning information and pulling what they are looking for out a large amount of information. (designed one of the first large displays)

-Perhaps the way to envision a conversation is to change the display to be like a camera - or how does a 3rd person speak in the group? Which leads to the thought of...

-Science Building piece is about the "celebration of trans-formative power of ideas".

-Future of Displays is a primary focus of his vision - looking at materials that work in both daylight and at night - something that can be opaque and translucent.

-Everything is Design - lets blur the boundaries between textual and visual information.

What is the Future of Design? - how can we have displays that are everywhere - but are not just focused on advertisement. How can we respond to the actual user - orienting to the user. How do we respond to the orientation of the viewer?

In order to make multi-touch work - he thinks we need to have things that are naturally attractive to people - like his "stone" project. The object needs to "telegraph" to you what it is or how to use it. The object need to be inherently understood.

Later, I spoke with David about my idea of visualizing a conversation - and he said something to the effect of: What if the computer was the 3rd person? - Which led to a dialog about film, and 3 people talking and what the dynamics are in a system like that. This may lead to the subject of another blog.

Prod, Poke, and Press information

2008-01-18T10:01:00.000-08:00

I sat down with a friend and talked through my current vision of this Voice/Touch/Word Garden I've been brewing about and researching. I've written many of the scripts that would like words people speak to a semantic network, but hadn't described the interface in detail. Here are my notes from our discussion. They won't make much sense to someone stumbling on this, these notes are more for my collaborators.

-Life cycle of information should be ephemeral - like the leaves on a tree and cycle through seasons. Things die by natural selection.
-I'm not going to try for any particular metaphor, I like the idea of the form of the creature/plant being half creature and half plant like a plant would be if it could move. I want it to surprise you with it's strangeness, and then reading the results to surprise you as well.
-No Menus! If I can avoid it, I want the interaction be be purely gestural. David has this idea that interfaces should be more natural, like playing a musical instrument. Here are some gestures I would like to be able to detect:

Gestures: Short Touch, Touch and Hold, Poke, Cover, Stretch, Drag, Shake, Tap - The next question is: What do these gestures mean to people and how well do they translate? I like the idea of the results being unpredictable - and the user having to finesse the information from the interface.

-Focus should be on the present conversation - not a history of what people have done before.
-Legibility should go in the direction of the person who last spoke? Or should it orient in the direction of the other person? Maybe an interface with words shouldn't be a table at all - perhaps it should be like a whiteboard where both people can participate, or a sphere or globe that people can touch and manipulate what is inside....

Could there be such a thing as a multi-touch spere? If it were spinning?

Keep the interface simple (don't require them to put their hands on the table and then speak).

Negation/Confirmation - could these gestures be implied by Touching and holding and then the plant shrinks, but if you prod or tap it grows? I like the idea of a Grow/Kill interface being the first steps to interaction.

Prodding a plant or creature spawns another plant, sometimes connected, and sometimes it's own plant. Perhaps the original term grows and continues to spawn more and more as it is prodded.

I'm interested in visualizing conversations as environments. The question I've been thinking is, what if a conversation could come alive?

Mutual Voice to Touch Information System

2008-01-08T16:57:00.000-08:00

I needed a more formal description of what I mean by linking voice and touch for correspondence purposes, so this is a current encapsulation of the idea I have for a touch table. -Seth
----

The Ambient Intelligence group at the MIT Media lab is working on a table that visualizes the connections between speakers around the table and the terms they each contribute. The table has a touch surface, and when a user places their hand on the surface a nearby microphone is activated. Individual phrases and words are recognized and emerge from the hand of the user as animated text objects. These objects vary in size and color depending on the amplitude and frequency of the individual voice, and can be released onto the table's environment through simple gestures of the hand.

The table could be seen as a "brainstorming environment" where users contribute to the content of the table by speaking and can manipulate and influence the environment with their hands. Unlike a traditional conceptual map, the nodes of the map will have behaviors and seek connections with other nodes on the table through a semantic concept net. The environment will evolve autonomously until it is influenced by user input.

We are interested in social applications that visualize vocal information, link it to a network, and facilitate direct social interaction between multiple around a single interface. The system is intended as a research platform for vocal and tactile interaction with digital content.

Visual Ecology: a Conversation With Kate Hollenbach

2008-01-08T15:38:00.000-08:00

Kate and I are sitting together in the lab drinking tea and exploring what we see as possibly a new way of describing a visualization. We were talking about what would happen if you give data a personality? here goes...

Visual Ecology:
A visualization technique in which the information elements behave as organisms in an ecological system interacting with each other and the topology of the digital landscape.

What happens when people can contribute and influence the content of a visual ecology? We would like to design a system with open ended constraints - and architecture that evolves in response to contribution.

We ask:
What happens when you give data a personality?
How do you get people to explore data visualization?
What happens if the data seeks you out?
What if it interacts with you as much as you interact with it?
What happens when data and people are mutually engaged?

Evocative Metaphors:
-A real-time system that evolves in response to a large group of people (concert/active audience)
-A Garden where data grows and dies and branches
-A Pond of creatures whose behavior is determined by their content
-Any flock with varying individual and group behaviors
-A web or network or net or root system
-Elements with attention getting behaviors (spastic wiggling, irregular & dramatic motions, directed motions, annoying, grotesque, alarming, noisy, they could even say needy things)
-Human Body (avatars, limbs, representations of the viewers body)
-A transforming mirror
-Giving inert elements awareness (eyes)
-Transforming content that we normally think of as static
-Using simple animation techniques to make data more playful

In his article, Transforming Mirrors (1995), David Rokeby presents four paradigms that an interactive work can embody:

-a navigable structure or world,
-a [self-sustaining] creative medium,
-a transforming mirror,
-or a automation

What we are talking about here is perhaps all four - an amalgam in computational form of an interactive art piece, data visualization, animation, and social interaction. Well, it's the start of a framework anyway. no harm in dreaming. :)

If What I said Came out of my Fingers

2007-12-21T09:20:00.000-08:00

So I just picked up Joan DiMicco's thesis on Small Group Interaction through Visual Reflections of Social Behavior. I thought that I'd take some note of the salient points and post them here for further reflections and sharing.

Investing Energy
I want to keep thinking actively about how we can design meaningful interfaces for social engagement, and really take advantage of the rare opportunity I have here at the media lab to really do something remarkable and have a place to install it. I'm still going through the motions of doing things that have been done, just to get a foundation - but I'm excited about the lack of truly engaging applications I have seen for multitouch surfaces - lots of form factors but not so many applications that are not just gimmicky. The challenge is to overcome the Wow factor and have something that people will want to continually invest energy in.

Quotes from her Thesis
She says, p 16: "I believe technology can be of most use to groups if it is built with the intention to:
1) encourage vigilance in the discussion,
2) instill a resistance to group polarization, and
3) increase the sharing of information, three methods for avoiding common decision
flaws."

(So, encourage engagement, democracy, and trust. Sounds good.)

Automated sensing of behavior and salient events in a meeting or group context is an
emerging field of study. Experts in vision systems, speech recognition, and gesture and
facial recognition are currently advancing our ability to recognize different behavior and
events in naturalistic settings.

First she presents a voice recognition system where salient parts of peoples conversations trickle onto the table. (Her interface uses Speech Recognition (IBM ViaVoice) -- Semantic Filtering (WordNet, or ConceptNet) -- Social Facilitation(balancing out contributing voices))

My Idea... words trickling out of your hand...:
What if people spoke and using directional mics and vision tracking you could have what they say trickle out of their hands. Then they could confirm, deny, give, or set free the information onto the table... word worms crawl out and migrate to people on different sides.

Afterthought
She uses a headset - (i would prefer a mic clipped nearby) & focused on the amount of speech over it's content.. perhaps in the end because you have to train people to do speech recognition you would have trouble really getting an accurate system going.

Conclusion
The overall emphasis of her thesis was on group dynamics and systems that can facilitate awareness of how you behave within a group - through an awareness of how much you talk and how often. I feel like what people really need is a system that is also the task - not an outside task where the technology is monitoring that task, but a system that evolves and creates a relationship between the environment and the people interacting with the environment. Such a system would be:

-Created but not governed by the people using it.
-Responsive to Sound and Gesture around the table
-Slightly enigmatic, not transparent, rather more mysterious
-Playful and Animated
-Illustrative of the connections between those people around the table
-Modifiable - user can negate or modify content they don't like
-More responsive to cooperative motion and engagement.
-Encourage teamwork through a "discovered" or "hidden" ruleset.

Seth

Lecture by Tinsley Galyean (magical technology)

2007-12-11T16:51:00.000-08:00

Setting
I'm sitting right now in a lecture at CSAIL, the computer science and artificial intelligence labratory at MIT. Tinsley Galyean, CEO of a company called Nearlife that makes interactive installations for high paying clients around the world - including many that I have seen at the museum of science and industry in Chicago. Tinsley is a former MIT media lab student - and now is deeply immersed in the business of running his company.

I think many of the projects are executed in beautiful and ephemeral ways - visualizations that can be powerful and have a deep effect on children and human understanding.

A Business Approach
In his lecture, he says things like, when in doubt leave it out, or "be a user", or lets get physical, does it feel good, The WOW not the HOW and Easier than Pie -- Eye Candy is not enough? He says we need "Crack for your Eyes", and convince the client that "it was their idea"... the list goes on. It reminds me of the simplified formulas for success that came out in the 50s and 60s - "THINK AND GROW RICH" -- was his lecture designed for corporate environments where his content must meet a set of criteria for interactive content?

Of course interactive installation is attractive and catchy, but I was discouraged by his formulaic approach to the content of the installations. So I asked him about it - after the lecture and he said that since he has been doing this for 15 years and that he might seem jaded because of that. Perhaps I have been isolated from this world - places like Snibbe Interactive or Nearlife are composed of artists and technologists trying to facilitate interactive experiences within the limitations of a corporate environment. He even mentioned that he and Scott were contemporaries together at Brown. It seems that Scott's work has never lost it's focus on social exchange however.

Social Exchange:
Someone also asked if he had ever made something for social exchange, for example in the bar scene - and he said: We always try to design things so that they are a collaborative social experience - but not directly addressing social exchange, I think this is an important distinction.

Conclusion:
I talked to Tinsley afterwards - and liked his disposition. After that I felt bad about asking such a directly oppositional question. I think my own fear of not being able to make things I truly care about outside of the institutional umbrella was revealing itself. I don't ever want to get jaded though... no one starts out jaded.

Golan Levin Returns to MIT for an Artist Lecture

2007-12-10T13:11:00.000-08:00

After seven years of innovative technology work since leaving MIT, Golan Levin returned last week to give a lecture at the Media Lab. I took a bus back from NYC in a hurry to try and catch his lecture, knowing that I would get to talk to him again in person after visiting his office 3 years ago when applying to Carnegie Mellon. At the time I knew so much less than I do now about how one makes an interactive artwork, but it was easy to recognize the playful, profound, and technically proficient nature of his work then, as it is now.

The most endearing part of the lecture was Golan's intimacy with the audience, and his playful and highly visual presentation of the work. His explanations of the conceptual nature of his work are more implicit and cursory. I was hoping for something different than an overview of projects, but found this to be interesting because I have seen many of the projects presented by Zack Lieberman as well over the last few months while attending his workshops. It was probably the last time the John Maeda and Golan will get to meet at MIT - as John is going to be President of RISD - an appropriate but sad transition for the media lab community.

I took notes during lecture - more as a way of triggering my own thoughts during the time, and so will post them here for friends. Later Golan came and visited our group, and I was reminded of his candor and decisive excitement. I was glad to finally see him for a brief time after having missed the opportunity to work with him for logistic reasons years ago.

My Notes:

-His work is about supporting flow states - focused on the feedback of the viewer
--First work at MIT: Audio Visual Environment Sweet
--How can one create a really plastic audio visual substance? - mapping is present and felt... I like this term "plastic" - mailable, but with it's own essential form...
--What about: drawing with a gas - his Audio Visual Sweet was about making 7 systems for plastic graphics
--characteristics of live performance were missing from the Audio Visual Environment
Quote: Joy Mountford - "mouse is a narrow point of control" - like sucking human expression through a straw.....
--Like Lieberman, explains the shape of vowels with a quote from Wolfgang Koehler about fonestesia
--Hand gestures as a locus for interactive performance...(and the hands themselves for that matter)
--These days everyone has a multitouch system - (well isn't that true, but do they have good apps for them? These systems are bound to become more widely used, we need good thought about them... I'm not giving up yet, but I see his point..)
--Reader response theory - what if artworks could be created by looking at them? (Yes, gaze is still a rich area for exploration as the technologies improve - I still think it would be amazing to detect moments when people make eye contact without having them wear anything.)
--Recent Work: Eye Tracking for the rest of us. - and image constructed from the people who view it
--There are ways of programming with your body - rather than in a visual way - live coding... is it possible to program with gesture??? (this relates directly to Mitch's question after the lecture about moving away from code - and the word programming, to "allowing people to mess with the underlying rules behind a system - thats what programming is after all)
--music - what are you thinking about when you are moving in the world?
--Again, say "play with the rules underneath" instead of "programming"
Quote: "No installation is done until you have deployed it at least three times..."

In response to Hiroshi Ishii's question about profoundly affecting culture, (how can an artist do this, as Hiroshi always says "take it to the next level, asking new questions is more important than doing sometimes....") Golan said:

Three things:
1) He will predict the future - artwork becomes a desktop widget
2) Artworks become a footnote - predicting the future incorrectly
3) Make something surpassing, sublime or uncanny quality that makes people experiencing things over and over again... (here he mentioned Listening Post, by Mark Hansen which strangely I stumbled into this morning in Times Square, looking for a place to print my bus ticket!!!!)

Well I guess we all are trying to shoot for the listening post effect on culture.... but in the meantime we will make playful interactive pieces and try to shape this plastic medium of software that Golan so masterfully employs. Until next time...

Artists and Scientists, how big is the door?

2007-10-27T20:44:00.000-07:00

I was talking with a friend tonight at dinner and something he said is resonating with me. "Artists carve out a door just big enough for themselves, and scientists carve out doors much bigger than themselves."

In some ways I think this is true, and that this needs to change. Artists are valued for their unique innovations, and scientists for their contribution to an overall field, or a small part of a problem. If artists worked together to do things like: Processing.org (and see related at the bottom of the page) then we would move away from high art and more towards building a cannon of knowledge that others can learn from and use.

This perspective is beginning to take root in new media exchanges because of the technical constraints of the medium, and the desire to share things with other people because we have benefited from their work... like instructables: http://www.instructables.com/

This is one of the things that I love about open source initiatives and people who share knowledge - it benefits us all. The authors get social credentials and the viewers can utilize the information and find inspiration for their own work.

DiamondTouch Tables at MERL

2007-10-22T15:36:00.000-07:00

Today we visited MERL in Cambridge, which stands for Mitsubishi Electronic Research Laboratories. We were there to see the diamond touch table technology, developed for multiuser touch engagement with digital content. I'm particularly interested in this type of table because it uses capacitive sensing, which allows you to differentiate between multiple people acting on the same content. The responsiveness of the table is nice but limited, without any object recognition, proximity (hovering), or pressure sensing abilities.

What I am learning is that the technology for tangible interfaces has been deeply explored and developed, but the applications (besides perhaps some of the artistic folks working in Jeff Han's much talked about group at NYU) are lacking - especially in the realm of engagement and collaboration. OK - so that is a bold statement. Visualization, Mapping, Data browsing is vastly improved, but what about the social applications?

MERL is unique in that the table differentiates between users. This could create an intimacy between people collaborating on the same content that would otherwise be hard to achieve - and when hands touch this also can be sensed. Here are some rough areas that I think would be interesting to address:

Small Icebreaker Games: Two people come up to the table and are presented with a collaborative task: cultivating a garden, adding to a knowledge base, exchanging information with each other, or sorting and seeking information on the table together.

Communal Environments that evolve and change as people influence them - it could be more than a message board - an interactive space that shows a history of sketching and social exchanges that have happened around the table. I'm interested in presenting graphics that don't make sense right away but reveal their nature and content as they are played with, through that process leaving an impression on the table, like a history of communal drawings endlessly layering on themselves.

Right now I imagine that different types of gestures, a fist vs. individual fingers - or how many fingers you extend and the speed of your movement are all things that might be lost to a MERL table - but I think if you present a metaphor that in reality would require touching, like the surface of a pond or puddles or a sea of words - people would forget all about the interface and consider altering the content.

The key is perhaps to maximize the unique capabilities of each technology and see where that leads you - if you are always comparing your experience to an ideal than you will fall prey to your own limitations as well. I'd like to try to maximize the differentiation and sensitivity of the diamondback interface by focusing primarily on those features.

Conversation with Warren Sack

2007-10-22T15:20:00.000-07:00

The media lab is a wonderful place to work in because you are continually inspired and surprised by the connections you find between people, and the quality of conversation.

Today I met Warren Sack, and artist and media theorist from UCSC who is at Colombia this semester considering the possibility of drafting a new text for digital arts students. What a great idea! He has found that the framework setup by Lev Manovitch's book, is rather limiting and poses a framework that could really be expanded on. I agree with Warren, but I have a feeling he might go in the theoretical direction, responding in a more conceptual than practical way; instead of positing a new contemporary framework for teaching new media arts.

I'm all for a 2 year outline that gives students a framework for continual learning - one in which during the first 6 months a student can map out where people are working, what the tools are and how he might engage in that vast landscape. Here is what I wrote to Warren after our conversation:

Things that are missing in the current teaching documents:
a) Mapping - Students need a general map of the history of these "new and old media" - as well as a concept map of the tools. By six months they should be able to focus on mastery in one area that can be applied or collaborated with other students.
b) Exhibition and Archiving of work - can we post these things in places other than rhizome - where they add to the learning community, so that digital arts curriculum can have a community like the one in processing, but not just centered around that language.
c) The connection between Data and Image (SQL or another mean) is weak in the arts - and too technical. Is there a way to make an arts interface for data manipulation? (or has this been done?)
d) Physical Computing is forgotten. Students get caught up in Gui land - they need to play with wires - and move outside of that parameter - crickets, arduino, artbus, ezio, are all steps in that direction but no where are they mapped, and is that possibility stated as a distinct direction that those skills could inform. Without programing pic chips - how can we introduce microcontrollers?

I'm all for a course that touches on all these sort of like a "How to Make Anything" for digital arts. Sound good?

Research at the Media Lab

2007-10-18T11:49:00.000-07:00

So a brief introduction - I'm an artist, technologist, and researcher at the Media Lab at MIT with an arts background - and some cognitive science as an undergrad. I worked as art artist at the Art Institute of Chicago for two years prior to this - much of that is documented here.

What follows are thoughts and research on interface design. I'm interested in creating meaningful interfaces for physical environments that utilize digital information - but more in an organic playful way than a utilitarian one.

Specifically, I'm researching table top and gestural interfaces, a field many people here are calling tangible programming. What does this mean? For those of you not around the HCI folk, (Human Computer Interface) it means we use sensors to track peoples motion in an environment, and translate those gestures into input for a computer. Our traditional input devices are a keyboard, mouse, joystick - in physical computing we are interested in seamless interactions where people can act intuitively to influence information on the computer or in their environments.

David Merrill, my officemate at MIT, calls our traditional input devices "single point of contact" devices. He is working on something called siftables, little stamp-size video screens that can sense how they are handled and how they relate to each other. See: siftables

I will be writing software apps for siftables - and in my own research, trying to build a table that responds to hand gestures. This is not the first time someone has worked in this area, so much of this blog at first will attempt to provide a history of some of the work I find intriguing. I'll try to post in casual but informative way for those of you also interested in this field but more from an arts perspective.

The interesting exchanges between arts & sciences are often divergences from specialized academic work. Right now the questions on my mind - and future subjects for writing are:

1) How can we utilize vision tracking research without relearning increasingly complex algorithms?
2) How can interfaces be socially meaningful, and really engaging for communities?
3) Can something disguised as a game overcome peoples social inhibitions?
4) Is there a way of bridging our online identities (in various communities) with our everyday exchanges with people? And would we want to have this sort of connection or do we want them to be separate?
5) Will Common Sense knowledge databases for computing provide a valuable means of relating the information that we have, or will it create more problems by making too many assumptions? (see common sense computing at mit)
6) What applications of tangible computing really engage people? Are interfaces like Microsoft Surface the future of UI or can we expect more accessible and less expensive seamless interfaces?
7) From Bill Buxton to Jeff Han - what do we have to learn from previous designs?

Thats it for now - from here I'll try and be more specific than conceptual - but this at least sets the stage for future research.

~Seth