The metaverse of sound - the sounds of the past

We all have a past and society is not different. And having been around longer than most, that history is often vast, complex and filled with layers upon layers of wonderful nuance. And it is often well documented to boot. So for the next metaverse experiment we’re diving head first, or ears first, into history and culture.

Culture differs from nature. Nature is ever present and I would not be surprised to find birds chirping away and nature doing its thing 24x7. The same thing does not work for culture. I would like to be told about local landmarks and historical places but that interaction must be triggered. To have a tour guide overlaid forever repeating the same phrases over and over and with no option to start it or stop it would drive me crazy.

We can’t have that. So I implemented a crude workaround. When a certain type of sound node is within your vicinity a low chiming sound is played. You can then choose to interact with that piece of art or landmark and get both more information on your phone as well as a series of recordings telling you about it.

Quick and simple.

Before going on to the actual history and culture that we will inject into the metaverse let’s address the technical debt we had since the last experiment. When moving through a forest it might be okay with the sound scape being all around us but now that we’re moving in an urbanized environment we really need to get some Dolby into our lives.

To do so was not hard at all, we have the sounds position and that of the listener, by taking the listeners direction into account, by accessing the devices compass, some basic math does the rest of the work for us. Now we are all set to get on with this project.

Most of the technical aspects can be charged with the last experiment so progress was rapid. After solving the problem above I set out to find some interesting landmarks to showcase and to enhance with the sounds of history.

The last part was simple. My hometown was built around the manufacturing of weapons, muskets to be more precise, and part of the old living quarters and the old factories still remain. As silent monuments of the part. So to breed some life back to these environments felt extra pressing. To once more allow the echoing of hammers to be the dominant sound of the city.

Sweden has a surprisingly large collection of historical sounds from different places and different time periods. Maybe not from the smiths themselves but from other important and mundane events. Snapshots of day to day life of a time long since past. So if I just put in some extra effort I could really recreate history. I decided not to go the extra mile and just reimagine history with stock sounds I have lying around from a failed attempt at game creation.

After having spent some time placing sounds around the area, hammer strikes and carpentry around the workshops and murmuring and communal chatter around the common areas I feel pretty pleased with the progress.

Next up is culture. Time to find some landmarks. Luckily for me my sister works at the local art museum so I called her for help. She told me about local public art and their importance, about the artists and their driving forces and what there creations represent. About this point I just find some local, close by, art installations and completely make up the artist and information. The experiment is about the concept and the tech and not the information that is presented.

In a later iteration this could be reworked to a proper demo or proof of concept and at that point content will be king again. But at present that is not the end goal.

So I made up a couple of artists, found some close by public art and added it to the map. Time to get to testing again. So I set out with my earpods set to ambient mode, batteries fully charged and only armed with a Corgi and a smile. Let’s see what we get this time.

Since the nature module was turned off nothing much happened at first. Just a normal walk with my dog. When approaching my first landmark I could hear a faint chiming noise calling for my attention. I decided to keep on walking. Later on, more chimes. Some coming from my right and some from my left and some appearing to be in front of me. When I turn a corner or turn around they stay put and their intensity only grows as I approche.

Here I hit my first roadbump. When you turn the sounds placements react according to that change, but, since the direction is measured by your phone which often is in your pocket this does not take into account that your head may move independently. The soundscape only reflects on your body’s direction and not what way you’re looking. This is somewhat immersion breaking but the effect is still cool. Going in to this face I already knew that it would happen but decided to go on in spite of it. I’m strange that way.

This does impact the experience when visiting the historical environments that I had planned out. Standing still, the experience is quite nice. Hammers are hitting anvils and sounds of carpenters and murmuring of voices from the past are giving the environment some added depth.

Moving or looking around breaks this illusion as you find that all that perceived depth is just paper facade.

Since it is more realistic the leeway is much less and the end result is found lackning. I think I prefer the old version without the ‘Dolby like’ addition.

When it comes to the landmarks the experience is much better. Walking past them they invite me in to learn more about them. As I come closer my phone shows me a sneak peak of the monument, building or piece of public art and when I’m ready I start the recording. A computerized voice tells me more about the statue and for it is fake, both voice and facts, it is quite the nice learning experience. I can see this working as this lets you walk around freely and take in a new city. To be able to roam free and explore but with the added benefit of a helpful guide that only tells you about the things you want to hear, and is easily silenced when you lose interest.

All in all part two was a mixed bag. Not a home run but still an interesting round to play. Back to the drawing board.