EM Sounds of a 15-inch MacBook Pro (In Glorious Stereo)

Recorded via stereo 400 millihenri coils. The song you hear part way through is Julian Cope’s ‘Laughing Boy’ from the album ‘FRIED’. It started playing on YouTube after restarting the computer, which evidently never goes (electromagnetically) completely silent during a restart. It is recorded here not acoustically, but through electromagnetic emissions due to movement of the computer speaker coils.

1999 and counting

I recently noticed that the number of citations to my published work had reached 1999. Profs in Canadian universities typically have several hundred to several thousand citations. It seems to be less common for profs in private Japanese universities to have a substantial citation rate. Japanese profs with some well-cited publications are more usually located mainly in the (former) National universities or National research institutes. There are some outstanding researchers to be found in places like Todai.

Researchers care about citation rate because it is a measure of the impact their work is having on the research community. Influential work tends to be cited more often. If you want a quick estimate of the impact a prof has made on knowledge, you can easily check to see if any of the work they have published has been frequently cited. The most commonly used index, for several years now, is Google Scholar. The reach of g.s. is global and it is not restricted to publications written in English, but indexes works in the languages  of contemporary scholarly activity (Japanese included). Currently, researchers interested in contributing to world knowledge write some (or all) of their articles in English.

Citation rate has been used for many years in most universities and research institutions to make decisions about hiring and promotions. This is somewhat true in Japan too, more commonly in the research-intensive (former) National universities. Research impact, as measured using citation rate has also been used to rank universities and research institutions. Try searching Google Scholar for Harvard University, M.I.T., or Caltech. Now try Ritsumeikan. Interesting (or maybe not)? You can (roughly) estimate the knowledge output of entire cities this way: try out London, Paris, Tokyo, or Kyoto.

Google Scholar can provide insight into a given researcher’s most influential work. In my case, so far our work on face and facial expression classification has had the largest impact. As it turns out, we were the first to study the use of wavelets, a special kind of mathematical function, for processing facial expressions. This subsequently became a popular approach to analyzing facial expression images. I heard from a researcher inside one of the largest global media and electronics companies that ‘smile shutter’ technology was influenced by this research. Indeed, the work was demonstrated to that company (which I won’t name, but you can probably guess) and the other major consumer electronics companies in the mid-nineties. That said, I don’t think automatic smile detection was a particularly interesting application of the work. I am much more interested in the fundamental scientific aspects of the research.

So, how to do work which will be frequently cited? Simple – do cutting edge original work in an interesting field and publish it in a leading peer-reviewed journal or conference. The skill is in knowing where to cut the edge. That was Peter Medawar‘s idea about the art of doing science.

Another way to have frequently cited papers is to frequently cite them yourself.  There’s no rule against it, but excessive self-citation is something like masturbating in public because it’s trivially easy to spot this behaviour on the various citation indices. It would be easy enough to link to a few examples of colleagues who don’t seem to be shy about having a lot of self-stimulated citations.

Citation rate may be important but it is not everything: my favourite works, on musical interfaces, and visual perception of art, are less highly cited than the face classification work, though, when published, some of these works did attract attention by the popular mass media. However, the single most influential thing I have done in my research career – organize a workshop held in Seattle in April, 2001 – barely hits the radar on Google Scholar.

For most people, the best ways to do influential work is to collaborate with others who have shared interests and complementary skills. I rarely work alone and that applies to all the work I’ve mentioned above. It’s much more interesting and productive to work with another person or a small group.

SIGGRAPH Asia 2012: Advances in NIME

Prof. Sid Fels (UBC) and I will be presenting our course on Advances in New Interfaces for Musical Expression at the upcoming SIGGRAPH Asia conference to be held at the Singapore Expo in November 2012. Looking forward to spending more time in Singapore.

The SIGGRAPH Asia web site is using our photos as the main banner for the courses page, and these are also visible on the preview slideshow playing on the main conference page.

Kugelschwung – Pendulum-based Live Music Sampler

Kugelschwung is the result of a second year Human Computer Interaction project by six Computer Science students at the University of Bristol. These students should be roughly the same age as students in our seminar. The interface is simple but works very well and the concept is brilliant. The work has been accepted for presentation at NIME-12.

StopMotion Recorder

This video is a first effort using the StopMotion Recorder App on the iPhone 4s. Playback frame-rate was set to 4 fps. Images were captured manually an irregular intervals according to the movement of the subject. The ‘Vintage Green’ setting was selected in the App settings. This app is quite easy to use, but by the same token it’s fairly restrictive.

Two more clips with manual, irregular frame acquisition and the same playback rate (4fps):

Elektron Musik Studion 1974 Stockholm

This video offers a glimpse at an earlier era in electronic and computer music production – as well as what it was like to use a computer in the early 1970s. My first experience with computers dates from this around this time. It is interesting to reflect what has and hasn’t changed in the nearly 40 year interim.

Point cloud painting with the Kinect

This short video by Daniel Franke & Cedric Kiefer is one of the most aesthetically impressive uses of the Microsoft Kinect I have seen yet. Apparently they used three Kinects. Not sure whether the visuals could be rendered in real time because there is clearly interpolation between the 3D views involved in producing this video. Also for real-time use this probably involves programming in C++, or at least Openframeworks. For anyone interested in the Kinect, it’s worth trying to find out more about what went into producing the video. Some links are given:

onformative.com
chopchop.cc

There’s full quality version of the video available online:

daniel-franke.com/unnamed_soundsculpture.mov

And a ‘making-of’ video on Vimeo:

Here is a statement by the artists:

The basic idea of the project is built upon the consideration of creating a moving sculpture from the recorded motion data of a real person. For our work we asked a dancer to visualize a musical piece (Kreukeltape by Machinenfabriek) as closely as possible by movements of her body. She was recorded by three depth cameras (Kinect), in which the intersection of the images was later put together to a three-dimensional volume (3d point cloud), so we were able to use the collected data throughout the further process. The three-dimensional image allowed us a completely free handling of the digital camera, without limitations of the perspective. The camera also reacts to the sound and supports the physical imitation of the musical piece by the performer. She moves to a noise field, where a simple modification of the random seed can consistently create new versions of the video, each offering a different composition of the recorded performance. The multi-dimensionality of the sound sculpture is already contained in every movement of the dancer, as the camera footage allows any imaginable perspective. The body – constant and indefinite at the same time – “bursts” the space already with its mere physicality, creating a first distinction between the self and its environment. Only the body movements create a reference to the otherwise invisible space, much like the dots bounce on the ground to give it a physical dimension. Thus, the sound-dance constellation in the video does not only simulate a purely virtual space. The complex dynamics of the body movements is also strongly self-referential. With the complex quasi-static, inconsistent forms the body is “painting”, a new reality space emerges whose simulated aesthetics goes far beyond numerical codes. Similar to painting, a single point appears to be still very abstract, but the more points are connected to each other, the more complex and concrete the image seems. The more perfect and complex the “alternative worlds” we project (Vilém Flusser) and the closer together their point elements, the more tangible they become. A digital body, consisting of 22 000 points, thus seems so real that it comes to life again.

VR and the Computer of the 21st Century

I want to make note of a powerfully brilliant paragraph from a highly influential article [1] by Marc Weiser published in September, 1991:

Perhaps most diametrically opposed to our vision is the notion of virtual reality, which attempts to make a world inside the computer. Users don special goggles that project an artificial scene onto their eyes; they wear gloves or even bodysuits that sense their motions and gestures so that they can move about and manipulate virtual objects. Although it may have its purpose in allowing people to explore realms otherwise inaccessible – the insides of cells, he surfaces of distant planets, the information web of data bases – virtually reality is only a map, not a territory. It excludes desks, offices, other people not wearing goggles and bodysuits, weather, trees, walks, the infinite richness of the universe. Virtual reality focuses an enormous apparatus on simulating the world rather than on invisibly enhancing the world that already exists.

It is perhaps superfluous to say that Mark Weiser and his colleagues at Xerox PARC were prescient and that, comparatively speaking, things have gone in the direction of ubiquitous computing, rather than virtual reality. And perhaps this is not surprising, many of the brightest and most interesting human interface researchers were at PARC from about the early 70’s onward. Their work made the future we now see in the largely usable, mobile, embedded, sensor-laden socially-networked devices of recent times.

[1] Mark Weiser,  The Computer for the 21st Century, Scientific American 265(3): 94-104 (1991).