The metaverse of sound - the augmented forest

What could be more pristine and flawless than nature itself? The simplicity of and yet grandiose nature of nature is awe inspiring. Take a forest for exemple, it’s a theming ecosystem of animals and plants, some large as trees (simply because they are trees) and some small and microscopic and largely unseen. It’s a perfect place of order and the beautiful machinery of nature. So how could we improve on it?

My first experiment under the banner the metaverse of sound came to me walking my dog (a Corgi of course) on a bleak night in early mars. Snow everywhere, darkness, yet more snow and the sad realization that spring is far from here. And summer is just a distant memory.

What would make this stroll through a dark dingy forest even better? More birds, probably, or maybe wolves? The sound of a calm summer’s breeze or a babbling brook? My conclusion is all of the above.

If I were to add a layer on top of the forest, on top of our reality, what if any effects would this have. Is it just a cheap gimmick or something that brings some new value of perspective to the world? Clearly this must be tested right away.

But how does one go about creating something like this? To create a non-visual representation of some sort of parallel and enhanced reality of sounds. Never to be seen but still heard.

After figuring out I would probably need to reuse the same base concepts for the other experiments I went to work on realizing a world beyond worlds. A quick sidenote, I will not bore you with all of the details, with endless lines of code or intercrete details about the techstack. If you want to know more just ask me on LinkedIn and I can go into endless details. But for the sake of the sanity of all other readers (if any) I will leave it out.

So first things we need to create a representation of a world that the user can walk, run or drive through that we later can populate with sounds. Since humans most often are land based a two dimensional approach felt the simplest to implement. And since no visual data would be needed, a representation of this world would only be a list of coordinates, sounds and other metadata like volume.

This data would, just like a map, be needed to be divided into datasets to facilitate ease of use and not forcing a user in Spanie to download the soundscape of downtown Detroit or even worse Eskilstuna (in Sweden).

These map tiles or rather sound tiles would then be loaded on demand depending on where you are in the world. To be able to walk around new map tiles might be needed to be fetched after the first batch. So depending on your movement direction new tiles are to be streamed to your device so that you can see new soundscapes in front of you and be blissfully unaware about the sound tiles at your back. For simplicity’s sake all of this could be left out and the map just be hardcoded. But where is the fun in that?

Once this was all sorted, and a crude yet effective codebase in place, I started to populate my newly created world with sounds. Sounds of animals, the brook mentioned before as well some minimalist classical music. Most of the sounds I choose are meant to compliment nature’s calm, well nature, and give a some resemblance of hope of an early spring in this frozen hellscape that is Sweden in winter.

The placement and volume of these sounds would determine where they started and the intensity of them. All was set. Time to go testing.

Once the plan was set I was off testing. I planned a route through my parallel dimension and set my earpods to ambient mode so that I could remain present in our world yet add to it from the virtual. For safety, I brought my dog, and set out on a stroll through a snow filled landscape. A landscape not lacking in beauty or sunshine but devoid of all life or sounds. No birds chirping, no flowing water or calm breeze in the trees.

Once I hit the first placement, just as the pine and fir forest began, I was taken back by the surreal experience of tropical birds and the dynamic and ever changing sounds of a teaming rainforest. Strangely this did not feel as forced and fake as one could imagine. It felt calming and relaxing even though it’s far from what I would call normal.

Walking along the trail I planned for myself I walked more familiar soundscapes with the classical birds of Swedish spring and summer. This just felt right. The sounds are so familiar to me and related to the end of winter that the immersion was almost complete.

Lastly and more gimmicky there were the wolves. I saved them for a more dramatic part of the day. Dusk. I wish I hadn’t!

Once walking into the clearing I could hear them. Distant (or at a low volume) at first but as I ventured forth the sound grew in strength. The same surreal experience as with the tropical birds but now with an added lingering fear. Not a real fear of course, although I live close to a wolf flock’s habitat, but that sensation that you get when watching a scary movie. Only in this case the movie being real life. I don’t know if I would recommend it.

So for the next batch of testing I decided that daytime would be the perfect time of day. My family got the job as beta testers. They were, as with all of my side projects, mildly impressed but agreed that the sounds did add something extra to the experience. I got some feedback and a bunch of suggestions about improvements.

My daughter sumericed the experience this far with.

Too many birds and not enough wolves.

So I have my work cut out for me. For the next experiment I will try to figure out how to simulate the placement better with some home grown Dolby. I think this will greatly enhance the already interesting experience by giving the world more depth. By turning around you would be able to pinpoint where a sound is coming from and move towards it or back away. At present this is only deducted by moving forwards or backwards and listening for an increase in volume but as a first draft this is still spot on.

On to the next one. The sounds of the past.