EV Graduate Seminar - AD508 F08

import JMyron.*;

JMyron m;

void setup(){
int w = 640;
int h = 480;
frameRate=30;
size(w,h);
m = new JMyron();
m.start(640,480);
m.findGlobs(1);
println(”Myron ” + m.version());
}

void mousePressed(){
m.settings();
}

void draw(){
m.trackColor(255,255,255,255);

m.update();
int[] img = m.image();

//first draw the camera view onto the screen
loadPixels();
for(int i=0;i<width*height;i++){
pixels[i] = img[i];
}
updatePixels();
background(int(random(5)),int(random(5)),int(random(5)));
//draw edge pixels of globs (this and the next chunks of code are chokers)
int list[][][] = m.globPixels();
stroke(235+int(random(15)),240+int(random(10)),250);
for(int i=0;i<list.length;i++){
int[][] pixellist = list[i];
if(pixellist!=null){
beginShape(POINTS);
for(int j=0;j<pixellist.length;j++){
vertex( pixellist[j][0]  ,  pixellist[j][1] );
// print( pixellist[j][0]  +” ” +  pixellist[j][1] );
}
endShape();
}
}

}

public void stop(){
m.stop();
super.stop();
}

Here is a draft of my writing born from discussions we had in class.  I think you can read hints of these claims in my article responses on this blog.  If time allows, I will also include some materials from my work-in-progress “community” game. I stopped the writing a little short, as there are still a few points that need to be made, but it turned out to be 8 pages without double spacing. I’ll email a double-spaced word document too.

The topic nestles well between computer vision as a solution and a few calls for chapters/presentations I have found.

Abstract:

This paper seeks to outline a set of fundamental observations about the instructive nature of video games. It stands as a resource when considering the content of games, the design intentions, and the game’s relationship to general society. It is, as asserted in this paper, true that games introduce fundamental approaches to problem solving and conflict resolution. These introductions serve as a kind of prescription which may be translated into philosophical approach. This paper outlines three instruction-related characteristics of video games. These are the games tendency to require learning for Efficacy, understanding over recitation, and diagnosis the system. These three observations are then related to two solutions, the use of appropriate win conditions and the use of direct agency through computer-vision technologies.

http://www.lgrace.com/documents/games_as_teachers4d.pdf

<!– @page { size: 8.5in 11in; margin: 0.79in } P { margin-bottom: 0.08in } –>

BlueTooth Discovery Project Proposal

by Kevin O’Neill

Wireless device detection offers many possibilities for communication and observation. Bluetooth is a relatively newly accepted form of wireless protocol that is gaining in popularity rapidly. While the mystery surrounding the WiFi and cell phone signal span of the wireless spectrum has all but vanished, the novelty of Bluetooth has sparked a social convention and stigma attached to the concept and has dissociated it from the more mundane wireless protocols. Although the technology has been around longer than the technology of cell phones that have access to internet networks, there is an industry driven power that the phenomenon of Bluetooth has associated with neon blue LED lights and hands free cellular devices.

In a statement on the novelty of the Bluetooth phenomenon, I propose to construct a system of Bluetooth discovery devices that interact with people who may not be aware of the commonality of the technology in a public space. Two laptops will be placed at either end of a darkened or dimly lit hallway and paired up by Bluetooth at the limit of the distance of their signal strengths (aprox. 30-40 feet apart from each other). When people who have Bluetooth enabled devices enter the hallway, their devices become detectable to both of the laptops. When this happens, a sound file of a ghostly voice will be triggered letting the hallway passerbyers know the spirit of Bluetooth was angry at them for having been forsaken.

The two laptops will retain a constant query conversation using a LabView program which will impart Bluetooth device ID’s, names, and amplitude of frequency that are being detected at each moment. When a device ID and name is detected on both lists, the laptop detecting a greater amplitude in frequency will trigger a sound file that will be played on speakers situated on the laptop’s respective side of the hallway. The effect is intended to be an immaterial essence that follows the Bluetooth device holder throughout the haunted hallway taunting them for having one its “BlueTeeth”.

While the choice to detecting devices with Bluetooth follows the motion of passers by as opposed to camera based motion detection is another way of monitoring people, the purpose of using the mode is to illustrate the mysticism surrounding the Bluetooth technology. Many people may be unaware of the Bluetooth capabilities of their own personal devices, or at least unconscious of it in the moment that as they enter the dimly lit hallway. The hallway itself will stand out because of the lack of lighting, but the bodyless voice that selectively picks on people will raise their awareness of the technology they posses by presenting itself as the “Ghost of Bluetooth”.

Using processing, a laptop, and a video camera, I have arrived at a series of images that contain approximately 5 minutes of landscape captured from the side window of a car while being driven at speeds of 60-80 miles per hour, each.

Like Edward Muybridge’s horses; the landscape, vehicles and objects that get passed by the camera are portrayed in the forward direction and things that passed the camera appeared to be going the other way.

Some of the questions for myself at midterm were; Do I use a photoshop or other means aside from processing? Do I use a HD camera for video capture? What decisions am I making that curate the images? I decided that processing and a standard video camera were suitable to create new ways of looking at landscapes both in form and time. Using natural and existing light sources, as well as using lighting from different times throughout a day provided a feeling of a Van Gogh sensibility of the landscape. Even though the travels captured different spots, driving through Illinois and Iowa provided a fairly static set of image options. Fields, roads, cars, signs, lights. The biggest variables were light, and time.

My next step after capturing the base images were to treat them. I set out to create a video painting. Taking a single still, breaking it down into color fields and dragging it across a canvas slowly while carefully keying opacity, size, and sharpness.

In addition to this presentation I have also created a simple side by side display of the images as a set.

The processing programming, which can use an existing web cam or a video camera via firewire connection, was pieced together with several different existing sources and tweaked along the way to create a suitable end result. The code takes a video input, takes a line of pixels at a time over the chosen amount of pixel width and generates a tif file upon pressing the space bar.

My pre-final presentation thought was that the images would best stretched out to a 16:3 aspect ratio so that things seemed “normal” or slight in proportion to what we normally see. But there is enough imagery and recognizable elements to each image for a point of entry for the viewer to step inside and feel the compression of that much time in a small space. If Albert Einstein could get clausterphobia which experiencing his theory of general relativity it might look something like this. Roads go in new directions, small hills reach new heights, and we the people are next to invisible.

import processing.video.*;
Capture mycam;
int video_width     = 1500;
int video_height    = 1500;
int video_slice_x   = video_width;
int window_width    = 1500;
int window_height   = video_height;
int draw_position_x = window_width - 1500;

boolean b_newFrame  = true;

void setup(){
mycam = new Capture(this, video_width, video_height, 24);
size(window_width, window_height);
background(0,0,0);
}

public void captureEvent(Capture c) {
c.read();
b_newFrame = true;
}

void draw() {
if (b_newFrame) {
loadPixels();
for (int y=0; y<video_width; y++){
int setPixelIndex = y*video_width  + draw_position_x;
int getPixelIndex = y*window_width + draw_position_x;
pixels[setPixelIndex] = mycam.pixels[getPixelIndex];
}
updatePixels();
draw_position_x = (draw_position_x - 1);
if (draw_position_x < 0) {
draw_position_x = window_width - 1;
}
b_newFrame = true;
}
}

void keyPressed() {
if (key == ‘ ‘) {
saveFrame(”line-####.tif”);
exit();
}
}

Hear are the pages for our presentation on Dieter:

http://www2.uic.edu/~koneil2/508/Dieter/etall.html
http://www2.uic.edu/~koneil2/508/Dieter/versus.html
details forthcoming…stay tuned…

SAGE (Scalable Adaptive Graphics Environment)

I.    Introduction
Several unique events gave me the preconditions for exploring this paper. On the first day of class in EV Grad Seminar, we were given a tour of the cavern by Laura Wolf. For me the 55-panel LambdaVision display stood out. I asked whether these devices were available to us and Laura confirmed this. Thus in my proposal to my advisor of my independent study, I chose to research using LambdaVision, and thus came across its main software component, SAGE. This in turn also led me to research the Global Lambda Integrated Facility (referenced also by many other names. I  wrote a proposal to use the equipment, and began working immediately on content. The technical challenge of actually using SAGE within the context of a performance art event was considered. Several of the drawbacks seen early on were: the LamndaVision is mostly immobile, therefore the Cavern would have to be used as the performance space (not ideal). My lack of hands-on familiarity with the system made it difficult to envision how the actual performance interaction would work. Once my project was approved, however, I was given the go ahead to actually get some hands on experience with the system. I was given an account on the cluster and a 4-panel display workstation (papyrus) was at my disposal to experiment with SAGE. I was also briefly given access to the LambdaVision demo station (yorda) to experiment with the 55-panel display. The luxury of working with yorda was such that this account on the cluster and its SAGE installation is handled by the Technical staff at EVL, and all of the latest applications are installed and working properly. On my own account on papyrus, I had the pleasure of installing and configuring SAGE 3.0d for my own use.
A. The scope of this document
This paper attempts to shed light on the disparate elements of the SAGE software developed by the Cavern Group at EVL. Noting several built in presumptions and assumptions as part of a complete working methodology, this paper will address the ability to assess SAGE software as an outsider to EVL’s common practices, and High Performance Computing and Communications in general. This paper uses technical, anecdotal, and other research methods.
B. The main presumptions and assumptions of this paper are:
1. there is basic clarity within the SAGE documentation.
2. like most software and associated documentation SAGE goes through sporadic revisions as part of its development cycle.
C. The Goals of this document:
1. to assess the current state of SAGE and draw conclusions from it
2. to provide an outsider’s view of the documentation of SAGE, specifically its correctness and usefulness
3. to add to the knowledgebase of SAGE

II.    SAGE @ EVL
A. SAGE is EVL’s answer for the need for a software environment for a cluster-based high-resolution tiled display system. This type of environment can handle high resolution graphics, HD video and audio streams, and volumetric datasets, accessed by multiple users on separate continents using Lambda networks. The computing environment for SAGE consists of several components
1. Hardware
a. high-resolution tiled display; Tiled displays can come in many shapes and sizes. papyrus is a 2×2 display by multiplying th pizes, we can geta count of the total number of pixels.. Yorda is a 5×11 display with a total of 105 megapixels.
b. cluster-based rendering capability; What makes the system viable under this load is the core of the SAGE application. SAGE couples specific sets of tiles to nodes on the cluster, thus enabling distributing the workload.
2. Network
a. ~ 10 gigabit for data onto tiled displays (lambda); Lambda Networks are used to move data from one location (display) to another.
b. SAGE UI can be utilized from a laptop; The SAGE UI is installable separately on a laptop without he undeslying SAGE engine. This enables a user to connect locally to an existing tiled display and manipulate it.
c. see also WEB UI (as of this writing untested by me); THe WEB UI is used for screen sharing. Screensharing is utilies the vnc:// protocol. Separate VNC client and server software is required. THe WEB UI must be installed separately on a laptop actually running SAGE.

3. Software
a. underlying operating system (Mac, Windows, UNIX); regardless of the operating system you use, the correct most up to date version of SAGE must be installed.
b. required libraries to run SAGE; SAGE required the use of preinstalled libraries (see documentations)
1. some available at EVL; Some of the libraries are modified libraries optimized for SAGE use, and are available from EVL.
2. some third party. Other libraries are utilized and may be updated via third parties. Most are open-source.
3. some built in to OSs. Other libraries required for SAGE may be rolled into a pre-existing operating system installation and may be different based on the operating system version.
c. SAGE components (sage libraries, sage UI, etc)
1. most ship with basic package; demo applications and add-ons ship with the basic package.
2. must be configured separately; the main thing SAGE needs to know is your tile display resolution and network IP addresses.
d. applications
1. many of the original applications have been deprecated in favor of newer more robust applications
2. some of the old applications still work
3. newer applications may or may not be available form EVL, instead they are available from third parties.

B. SAGE development began in the 2003-2005 timeframe, but is still being developed (version 3.0e) in November 2008. SAGE builds upon work on on Teravision, and the applications Vol-A-Tile and Juxtaview. The main network component of the software relies upon QUANTA, which uses reliable blast UDP. SAGE mainly acts like a virtual frame buffer. The UI is built around Freespace Manager, a window manager that allows collaborators to manipulate data on the screen. This is now known as the SAGE UI.

C. As of this writing, I am informally involved in Supercomputing ‘08 in Austin Texas, where EVL along with UCSD are sharing a booth demoing SAGE. EVL is also participating in the SC08 Bandwidth Challenge showcasing Global Visualcasting - Collaboration in Ultra-Resolution Work Environments. The yorda machine used for EVL demos is being staffed by EVLers. There is an HD camera hooked up to stream EVL/Chicago live to the show floor. Similarly, the yorda machine is being utilized locally and remotely to help visualize the Supercomputing ‘08 demos and the Bandwidth challenge.

III.    SAGE Documentation & Support
A brief overview of the sage website shows that is serves as the main point of entry for current and potential SAGE users. The site is divided into PR, documentation and download sections, many of which have old but not critical information. Features include a general description, available applications, developer list, links to papers, download, documentation, community of lambda-based users, related projects, photo and video gallery, funding and contact information. While the whole website serves as a basic overview, the documentation section breaks down more specifically. The main lessons learned from the SAGE documentation is that while the installation instructions are a version (or two) old, much of what is written there is still relevant. Installing the 2.5 way may introduce some redundancy, but is otherwise fairly solid in scope. However, given the nature of complexity, it would make sense to update these instructions. On the cavern forum, SAGE support is the only set of active threads, and questions are answered by Ratko Jagodic and Luc Renambot within a day or two (or sooner) of their posting. The ReadMe files that ship with the most current versions of the software (3.0e) have not been updated. An add-on to the Quickstart page, the SAGE Web UI sheds light on some very useful features os SAGE for desktop sharing, but are unevenly documented and could be foregrounded.
A. SAGE Website resources
1. Documentation section
a. Quickstart (v 2.5); This section is out of date. Notably however, this is where the WEB UI information appears. Information for fsManager.conf and stdtile-1.conf files remain unchanged.
b. FAQs; The FAQ section probably should be updated with information from the Cavern forum.
c. Full Documentation; discussion of some of the technical issues, while instructive, are deprecated because they simply “work” in the background
d. pdf; same as Full Documentation
2. Cavern forum; this area has the most up-to-date information. The organization of the document is difficult to follow. Some of the questions and answers refer to versions 2.5 and earlier, specific applications and libraries, and specific hardware and network environments.
3. SAGE papers; many papers are very useful and delineate the usefulness and technical issues and environment of SAGE, and specifically complement the technical documentation of SAGE. See the bibliography. Of particular importance are Jeong, B., Collaborative Visualization Architecture in Scalable Adaptive Graphics Environment, and Renambot, L., Jeong, B., Hur, H., Johnson, A., Leigh, J., Enabling High Resolution Collaborative Visualization in Display Rich Virtual Organizations.
4. related papers; Global Lambda Visualization Facility (GLVF), The OptIPuter Project, LambdaVision sections of the SAGE website and their associated documentation.

B. SAGE Web UI page

C. ReadMe Files (from available downloads)
1. SAGE app
Readme.txt: README file for SAGE v2.5 written by Byungil(Brent) Jeong, Ratko Jagodic NEEDS TO BE CHANGED FOR SAGE 3!!! (Ratko, 07/12/04)
2. SAGE UI
3. other packages

D. Discussions with Lance Long, Jason Leigh, Robert Kooima, Ratko Jagodic, Maxine Brown, Luc Renambot concerning SAGE

IV.    Conclusions

A. Viability of Artists Using SAGE

B. Recommendations for SAGE documentation

V. Selected Bibliography

Renambot, L., Rao, A., Singh, R., Jeong, B., Krishnaprasad, N., Vishwanath, V., Chandrasekhar, V., Schwarz, N., Spale, A., Zhang, C., Goldman, G., Leigh, J., Johnson, A. SAGE: the Scalable Adaptive Graphics Environment. Proceedings of WACE 2004, , 09/23/2004 - 09/24/2004.

Vishwanath, V., Leigh, J., Shimizu, T., Nam, S., Renambot, L., Takahashi, H., Takizawa, M., Kamatani, O. The Rails Toolkit (RTK) - Enabling End-System Topology-Aware High End Computing. The 4th IEEE International Conference on e-Science, 12/07/2008 - 12/12/2008.

Renambot, L., Jeong, B., Hur, H., Johnson, A., Leigh, J. Enabling High Resolution Collaborative Visualization in Display Rich Virtual Organizations. Future Generation Computer Systems 25 (2009), Elsevier B.V., doi:10.1016/j.future.2008.07.004, 07/24/2008 - 07/24/2008.

Leigh, J., Brown, M., Johnson, A., Renambot, L., Jones, S., Moher, T. Global Cyber-Commons: Collaborative, Instructional Environments for the Virtual School of Computational Science and Engineering. Virtual School of Computational Science & Engineering, Great Lakes Consortium for Petascale Computation, 06/28/2008 - 06/28/2008.

J. Leigh, M. D. Brown. Cyber-Commons: Merging Real and Virtual Worlds. Communications of the ACM, 01/01/2008 - 01/31/2008.

Leigh, J., Johnson, A., Renambot, L., DeFanti, T., Brown, M., Jeong, B., Jagodic, R., Krumbholz, C., Svistula, D., Hur, H., Kooima, R., Peterka, T., Ge, J., Falk, C. Emerging from the CAVE: Collaboration in Ultra High Resolution Environments. Proceedings of the First International Symposium on Universal Communication, Kyoto, Japan, 06/14/2007 - 06/15/2007.

Jeong, B. Collaborative Visualization Architecture in Scalable Adaptive Graphics Environment. IBM Visualization and Graphics Student Symposium, TJ Watson Research - Hawthorne and Yorktown, New York, 04/26/2007 - 04/27/2007.

import JMyron.*;

JMyron m;

void setup(){
int w = 640;
int h = 480;

size(w,h);
m = new JMyron();
m.start(640,480);
m.findGlobs(1);
println(”Myron ” + m.version());
}

void mousePressed(){
m.settings();
}

void draw(){
m.trackColor(255,255,255,255);

m.update();
int[] img = m.image();

//first draw the camera view onto the screen
loadPixels();
for(int i=0;i<width*height;i++){
pixels[i] = img[i];
}
updatePixels();
background(0,0,0);
//draw edge pixels of globs (this and the next chunks of code are chokers)
int list[][][] = m.globPixels();
stroke(220,220,220);

for(int i=0;i<list.length;i++){
int[][] pixellist = list[i];
if(pixellist!=null){
beginShape(POINTS);
for(int j=0;j<pixellist.length;j++){
vertex( pixellist[j][0]  ,  pixellist[j][1] );
// print( pixellist[j][0]  +” ” +  pixellist[j][1] );
}
endShape();
}
}
}

public void stop(){
m.stop();
super.stop();
}

Complicit Warnings
AD 508
Benjamin Thorp

Complicit Warnings involves computer vision and also is grappling with issues of surveillance, complicity (of those being surveilled) and the potential for liberatory action. The work engages how people experience spatial forms when they are filled in with dynamic and rich multimedia information and how we’ve adapted and reacted to the knowledge that we’re constantly being watched, and how this knowing has effected our behavior in private and public spaces.

There has been much debate recently about spaces such as shopping or entertainment areas or other spaces where various information can be accessed wirelessly and wether this is a techno-aesthetics that promotes freedom or yet another encroachment of the consumer/capitalistic/panopticon.

In “The Poetics of Augmented Space” Lev Manovich ends the section entitled “Learning from Prada” with this idea: “In other words, architects along with artist can take the next logical step to consider the ‘invisible’ space of electronic data flows as substance rather than just as void - something that needs a structure, a politics, and a poetics.”

The ideas that this text has generated are ones that I want to engage in a multimedia, sight specific installation that will appear as a skin of light/information on parts of a public space/structure.

EV Seminar Fall 08 Project proposal

Short Description:

The proposed project would treat a viewer in much the same way as most people are treated, as a data source.  In essence the project would  “data-ify” the viewer.  This would be done by realtime processing of video of the viewer and overlaying a data trail on the user.  This data trail would involve both physical data and “virtual” or abstract data.

Necessary Equipment:
= camera
= projector/screen
= computer

Sketch: