Extended reality (XR), once considered a cutting-edge concept, is slowly entering mainstream awareness as digital realms grow more immersive. What exactly is XR, and how are Malaysian creatives taking to it?
By Deric Ee
The rapid adoption of digital technologies across creative industries has given rise to more immersive art. Artworks capable of reacting with observers in real-time, due to the widespread availability of devices such as sensors and smartphones, are appearing more frequently in our cities.
Extended Reality (XR)—an umbrella term that covers Virtual Reality (VR), Mixed Reality (MR), Augmented Reality (AR), and other technologies within the spectrum of real-and-virtual combined environments—plays a central role in the evolution of immersive art. With XR, artists now have the power to place audiences into artworks rather than restrict them behind lines of
tape on the floor.
XR engages users in different ways with each subset: AR adds a digital layer to a physical environment, VR transports us to a virtual environment, while MR merges AR with VR to enable real-time interactions between digital and physical objects.
Google Cardboard is still free to build! Visit the site for instructions on how to turn your mobile phones into VR headsets.
It’s impossible to mention AR without noting the groundbreaking success of Pokemon Go and Instagram’s video filters: the technology has since been utilised by big businesses like Ikea and Sephora to help customers shop wisely.
VR surged in access prior to the pandemic as VR gaming centres experienced a short-lived boom locally, and initiatives like Google Cardboard transformed smartphones into DIY VR headsets. MR is not yet as commonplace, though the upcoming mixed reality headset from Apple could very well change this in 2023 by blending the digital realm with our physical environment.
While XR is considered a relatively new development in Malaysia, AR and VR have already gained a foothold due to improved access to XR technology.
Reimagining reality in Penang
Penang-based sculptor Aboud Fares found AR to be an exciting element to explore and utilise in his ecosystem of work, so he put together a series of digital artworks for two exhibitions earlier this year in George Town.
NOW featured 16 AR sculptures in Hin Bus Depot from 11 February to 2 March 2022, viewable through a smartphone app which Aboud had developed. His app, Perk Reality, also allowed users to view Town-Cats, a separate AR exhibition which placed Aboud’s digital sculptures of cats on the city streets.
‘NOW’ by Aboud Fares featured works in AR which were visible on smart devices upon scanning QR codes. Image Credit: Aboud Fares
“I’m using AR as a tool to exhibit art, not as an art form by itself,” noted Aboud when enquired about his process. “Presently, I design my sculptures through VR and then 3D print them. The AR comes in to help exhibit the work that I’ve created digitally and have produced physically.”
Read more: Why cultural assimilation is important for Syrian sculptor Aboud Fares
In the physical world, NOW appeared as a collection of QR codes on display panels. But scanning these codes with a smart device made Aboud’s brightly-coloured phantasmagoric 3D sculptures appear on the device’s screen.
According to Aboud, his Penang exhibition was his most successful to date, attracting a stronger turnout than previous showcases in Miri and Taiwan. He also made these AR artworks available for sale as NFTs on OpenSea in order to continue running Perk Reality, which has amassed over 500 downloads on Google Play alone.
Elsewhere in Penang, Maitree’s Cahaya XR generated plenty of interest by utilising AR and VR to reimagine and retell local stories during its debut showcase at George Town Festival in July. The project also introduced programmes to propagate more public encounters with XR art and technology, even hosting an AR workshop for kids.
Infusing VR into Malaysian filmmaking
The SeaShorts Film Festival is putting together its sixth edition with the theme of ‘Future’ and VR features prominently in the festival programme.
Responding to the rise in VR filmmaking beyond Southeast Asia, the festival has put together an exhibition to explore alternative futures for VR in this region. Beyond Mirage has 12 VR works from local and international artists, including five award-winning VR films from Taiwan.
Representing Malaysia are four VR films which approach VR for storytelling in vastly different ways. One example is Mahen Bala and Zarif Ismail’s immersive 360° VR documentary, Cēb bah hēp, which takes viewers deep into one of the world’s oldest tropical rainforests where indigenous tribespeople are struggling to maintain their way of life.
Still from Mahen Bala and Zarif Ismail’s immersive documentary ‘Cēb bah hēp’ which started as a photography project on the Batek tribe in 2020.
“We started as a collection of photographs of trees endemic to the Belum rainforest thanks to funding from the INXO Art & Culture Foundation,” detailed Mahen Bala and Zarif Ismail. “While we were there photographing the trees and the landscape, we realised the importance of context in promoting a deeper understanding of our ecological landscape.”
Working with Ministry XR, they began filming their environment in 360° videos to transport viewers into the rainforest to be among the trees, soon developing an interactive visual archive of the Belum rainforest before doing the same in Taman Tugu. This eased them into creating their first 360° VR documentary.
“Our focus has always been education,” they added. “We believe in the power of VR to offer alternative perspectives as viewers are given the space to experience and discover on their own terms.”
Probing, sensing the path ahead with SeaShorts
While the Malaysian creatives featured in SeaShorts are getting started with 360° VR storytelling, their international counterparts have built a name with immersive experiences which blur the boundaries between real and virtual with a high level of interactivity.
Malaysian-born media artist and Beyond Mirage curator Dr. Lim Kok Yoong notes that different formats of VR—360° VR, 6 Degrees of Freedom (6DOF), and “interactive”—provide different user experiences.
“These are all forms of immersive art,” explains Dr. Lim. “To experience these exhibits, participants will be wearing a head-mounted display (HMD) to immerse themselves in the artworks.
“With 360° VR, users will be able to view their surroundings. But with 6DOF and interactive VR, users are able to use controllers to manipulate the digital realm, for instance by picking up objects and traversing the VR environment.”
Korean filmmaker Sojung Bahng’s ‘Anonymous’ is an interactive 6DOF experience about an old man remembering his life with a twist: viewers ultimately assume the role of the man’s dead wife.
SeaShorts, which takes place from 21-25 September 2022 in MMU Cyberjaya, also includes Taipei Economic and Cultural Office in Malaysia (TECO)-sponsored programmes such as the Path Towards XR Industry forum for local creatives, and a workshop on Adopting Live-action VR as a Narrative Approach by Ming Yuan Chuan.
These timely programmes prompt creatives to consider how convergence culture, demand for interactivity, and the rise of XR can enhance their craft as global creatives orient themselves towards making immersive art.
So is immersive art changing creative industries for the better?
In light of the potential of XR and immersive art, the common manner of presenting art—physically-grounded displays in an exhibition space—can risk seeming passé. Indeed, immersive art may promise a more impactful experience, but what’s the catch?
In February this year, the New Yorker posted a critique of immersive art after visiting several exhibitions and found that art spaces were using immersive technology to push ticket sales and attract public engagements, to the detriment of the artistic experience.
As immersive art requires a wide variety of expertise, they are often accomplished by “full-stack” art studios with resources to employ both the artists and technology required for immersive art projects. But as art studios grow more massive, art could become more and more “industrialised” and go mass-market to recoup investments.
The ‘Immersive Van Gogh’ exhibition in New York includes yoga classes choreographed with moving artworks from Van Gogh’s catalogue. Image Credit: Immersive Van Gogh
As of 2020, there was a noticeable trend of tech companies becoming art patrons in the West. Could this be a sign of things to come in Malaysia too?
“It seems inevitable that these values will infuse the work, in the same way that N.F.T. art, born of a technological possibility, has its own visual culture, one that appears to emphasise algorithmic generation and proficiency in graphic-design and rendering software. The work looks commercial, because it is fundamentally about commerce.” – The New Yorker (10 February 2022)
While the haste of the West helps us pre-empt issues we may soon face as XR technology and immersive art expands across the world, it is worth noting that there is still time and space for us to consider how to utilise these new tools in more meaningful ways.
Deric Ee is a Seremban-born writer, producer, and community manager with a background in live events and media. He is currently based in KL to manage and develop content for students.
Cover Image: Still from Vimala Perumal’s immersive film ‘Rojak Cow Cow’ follows an Indian housewarming tradition to highlight Malaysia’s diverse cultures.