ICM Final Project

 

A project that sonifies the streets of cities all over the world. Sumi Lee and Chloe Choi have created different visualizations based on the generated sounds through text art and geometric shapes using P5.JS

 

Visualize with generated sounds

 
 

Initial Proposal

As Sumi and I are collaborating from opposite sides of the world, we wanted to create a program that would sonify the people on the streets and observe the variety of music it would create.

Our inspiration comes from these two projects on how they created sounds with people:

Sidewalk Orchestra

Pose Music

Our project will also have a infographic component to visually compare people’s behaviors across different cities

KakaoTalk_20201122_231228552.png
KakaoTalk_20201122_231224531.png
 

Progress

Prototyping Mechanics

We started off creating a sketch that would create notes when colliding with certain spaces using PoseNet. We were thinking of either a row or grid structure but chose the row when we realized many people seldom moved up and down.

P5.JS Link


Fleshing it out

We moved the project to a local file when we added the videos due to size limits.

We also added text to label each location and implemented Craig Reynold’s Steering Behavior on the text whenever a person walks by.

Our biggest problem was getting each video to load with their respective PoseNet working for multiple people.

 

User Feedback

  • Many people were wondering if the balls were creating the sound. An informative and visual effect ended up confusing the audience but we thought it would be ok since the text and video were not scaled and aligned properly at the time.

  • The project feels like you’re looking out the window

  • Should each city have different sounds? For example, using different scale or different instruments.

Future Endeavors

The feedback that our project feels like looking out the window really stuck with us. Due to everyone having to stay home, we thought it would be lovely if we can create a public website where anyone can use their camera to record the streets and hear music real-time. We can also allow people to look into other people’s currently active streams to look out of numerous windows.