Case Study: Leveraging AR/VR Type Technologies to Enhance Customer Experience, presented at ACBD23
[Applause] Hi. Good afternoon everyone. I’m Tsukasa from Tokyo. Actually this is my second time to be a part of this speaker faculty here, I'm so excited to be here again. But I'm actually a little bit nervous to speak in front of this international audience because I'm basically all educated in Japan and I know very little about the building and construction market internationally. But fortunately today I speak
about the case studies so hopefully I can leave some examples for you to take away. So today, my topic is about the VR / AR / metaverse things to enhance the customer experience, but before I go into the main part, let me recapture this: Why are they actually important for us? I think they are important for the stakeholders who are keeping on requiring better visualization for what to decide. They are important for the project because they help complete the project model life cycle. And, they will be important for our future because I think the metaverse can extend the common data environment. Let me explain about those topics, but first of all, let me introduce about myself. I'm Tsukasa Ishizawa, I’m a PhD, I'm a registered architect in Japan, and I'm a LEED AP. I'm a group lead for Computational Design Group, Takenaka Corporation,
and I'm a project associate professor in The University of Tokyo. I'm a YouTuber, I have a channel with a little bit more than a thousand subscriber where I'm talking about tips and tricks for the modeling software, and a little bit about the life of architects. About takenaka; actually Takenaka Corporation is one of the top five general contractors in Japan. On top of the corporate scale, it will be characterized by its history, whose origin can date back to the year of 1610. After the company transformed into the modern corporate style,
and since then, we were engaged with numbers of the landmark projects like Tokyo Tower, Tokyo Dome, cultural facilities and mixed complex… basically we do everything around the buildings, but we don’t do the civil works. We are fully engaged with the building projects. Actually we are a design-build contractor. In fact, 70 percent of Takenaka‘s projects are designed-and-built. Simply it means that, in our Takenaka project, first our design team starts working usually from the predesign phase and we run the rough cost estimation at the end of BD phase; and this is when our construction team embarks and start talking about the design also. sometimes they come to me and advise that “Hey maybe you will need
to have some more clearance from the neighbor building because we need to do the scaffolding.” It's not really competing; it will give us some good constraint for the design thinking. And once it goes harmoniously then we can deliver the project on time, within the budget. This became possible in Japan mainly because of its cultural background. This picture is of the middle of 13th century and the person circled at the center of the picture is a master builder called toryo. This toryo is said to master five different skills in one personality; namely, the building design, estimation, carpentry work, drawing, and detailing. It means that, in Japan, it was quite a norm to come to a single personality
to talk about the building project regardless if it’s design or construction. The separated concept of architect and builder was actually imported at the middle of 19th century when the Japan was in the process of the modernization, but somewhat that process was fusioned into this existing Japanese culture; and this is how the modern Japanese general contractor was formed. Okay, so let's start talking about AR / VR thing. You will use either a head mount
display or a mobile device like smartphone or tablet. In the Virtual Reality (VR) you'll be fully presented within the digital environment, so you will have the immersive experience in the digital build environment, whereas in the Augmented Reality (AR), still you see the actual real world and some information like the geometry or attribute will be overlaid onto the real world object. So on the site the designed building will be popping up in front of you, and in the Pokemon Go the Pikachu will appear in front of you. That's how the augmented reality works.
And in the Mixed Reality (MR) it somehow goes beyond. In this explanation it will allow you to do the manipulation or interaction with the object whether it's digital or real, but simply this mixed reality is a fusion of these two technologies. Overall, we might call them as an extended reality (xR) and today we mostly focus on the VR / AR things.
Looking at the hardware market growth, in terms of the shipment it became four times more in the last five years. It's gaining the popularity so rapidly. One more important aspect is about the Building Information Modeling (BIM). Actually I'm a BIM guy and I'm not supposed to be here to talk about the VR / AR things but it is a very good opportunity for me. This is an interesting survey done by Dodge Data and Analytics. They say that BIM users, compared to the non-BIM users, have a great advancement that advantage in terms of the adoption of digital design intelligence tool like VR / AR, reality capture like LiDAR, mobile devices like wearables, and the digital twins. So now the BIM is an infrastructure to adopt the further digital transformation.
That is why we started focusing on taking the project model directly into the extended reality presentation. We collaborated with the Unity specialist and internally developed some kind of add-on that allows to import the 3D models which is sometimes from Revit, sometimes from ArchiCAD, or Rhino, or SketchUp it doesn't matter. Just we take the model into the VR or AR presentation. One more important aspect is that this environment allows the telepresence or telexistence. In the demo on the left hand side you see the different architects from different cities are joining into the same environment to talk about the design simultaneously. This is
what we can do thanks to this kind of digital technology. One more thing. We can visualize what’s basically invisible. In this example we are visualizing the airflow by the arrow or bubble. Or we simulate the direct sunlight
intake. Sometimes we can do very simple acoustic reflection simulation. Those are so essential for the environmental performances and building sustainability, so these are so helpful to align the understanding of the stakeholders for the decision-making. Let's go into the examples. The first one is about the San’ei headquarter office.
The building’s overall shape looks boxy but actually the internal view is a little bit complexed. This is because this office is for a steel fabricator located just next to their steel shop, they wanted to have this office as their technology showcase. So we designed a building with the slanted wall and the Voronoi-like volumes which was quite difficult to understand in the conventional forms. So just we had the CEO and we gave a head mount display to him and he tried to manipulate the design to understand what's going to be built. Another project is about Chuo University. This building was characterized by the
huge atrium space at the center. Now you see the rooftop design changed; and actually the design of rooftop and material selection has a great impact to the building sustainability. So we prepared several design options and we ran the environmental simulations in advance. We took those design options and simulation results into the VR environment for the client to choose a specific design options to see the impact to the thermal comfort. So in the video you will see the flying arrows and actually this show the airflow with the
temperature. Once they change the design it will be reflected back to the arrows and they will see the differences. They don't actually feel that but they are visualized for the thermal comfort. This is an AR project and this is the use case for the inspection. This building is called EQ house for Mercedes. They wanted to have this finely detailed building, and this was possible because of the well-made design model. We took this model for the AR inspection. Because the
building has quite complicated shape, we wanted to simplify the thing for the auditor to focus on. You can see at the right bottom picture that this is highlighting the sprinkler heads with the dimensions so that for any people that once they wear a head mount display they can focus onto the items that they need to inspect. Rather than adding the things to the real world, this is to filter down the information to focus on to the things that we really need to discuss. One more thing, This is the PoC project for the device called Atom by XYZ Reality. This is an AR device but this is integrated with a safety helmet called smart safety helmet. This obviously allows the on-site AR. This is durable, and this provides hands-free environment for the
site foreperson to see the items to be built. For example, on top you see the MEP services, and on the middle you see the drywall installation. In terms of the accuracy there's a comparison at the bottom, which is quite satisfactory. Only the thing is about
the affordability; at the moment, we can't afford this device for every site person, but if it's getting the popularity then we can step forward to the paperless construction. So it's going, advancing, but I have some complaint about the use of the model. These pictures are from my past project where I was playing a role of the BIM manager. I might say that the people including the clients, the corporate members were so collaborative for completing the project model. But actually how they use it in the project was always
like that. They prepared a projector or monitor and they show up the model and they manipulate it for three to five minutes — and they come back to the paper agenda and paper drawings, and keep on talking inside this. Nothing’s wrong in this conventional working style but it obviously looked like something additional or something top-up to the existing workflow. Once we
illustrate it's gonna be like this. Once it goes very successfully, the building information model can spare some effort to create 2D drawings or the quantity take-offs: but sometimes that might be the best. Of course we can derive out from the model in terms of the VR / AR presentation, but it still remains as the position of an additional tool-set to the conventional things. One day I started thinking like this. In last ten years or even in last three to four years, we got
so familiarized to have a meeting online, like Zoom or Microsoft Teams, whatever you name it, but still to date we prefer to have the meeting face-to-face basis inside a physical meeting room, which is completely fine. But those spaces, whether it's virtual or real, have nothing to do with these spaces we are going to create. Of course. It's not built yet. But how about virtually? It's somewhat there already because someone designed something. Then why don't we host the meeting inside the project model and track the project activity all the time inside the project environment? The reason why I started to think like this was because I have some bitter experience. In
one project, the client required me to update the models to the cloud environment every week. I did so. But what I found was that nobody opened the model for a week; and almost forever. What was my effort overall? Yeah so the client wanted to make sure the things are updated and that’s the client job I know. but I wasn't so satisfactory about those kind of things; then the idea was that clients need to be more exposed to the model we've created. I know that sometimes it's too much to ask a client to open up the model to manipulate around and see inside, because they have their own job. Then why don't we originally have the meeting inside the digital venue that we've already
designed? You can be inside (or above, beneath, nearby or wherever you want) then we start talking about the project. And the things will be digitally recorded, time-stamped and sometimes transcribed, tagged and stored in the Common Data Environment (CDE). It's going to be possible. Then a concept will be like this; originally we imagine the project metaverse space and we locate the project model inside and all the project model or project activity will be inside this environment. I named this concept as the metaverse as common data environment; and actually I I'm going to publish a paper in coming academic conference so if you want to know more about this concept please find it in the conference proceedings. So now is a time to talk about the metaverse! Wikipedia says that in the colloquial usage a metaverse is a network of 3D virtual worlds focused on the social connection. Yes, VR and AR are the essential technology to enable this
kind of virtual environment experiences, but in the metaverse we are more focusing on the communication, interaction, transaction happening inside this digital workspace. This is very useful framework called digital twins-native continuum. We, as a building specialists, tend to think that the metaverse is all about the digital twins which corresponds to the real world. But actually it's not. We can imagine the worlds like Fortnite, worlds like Minecraft, world like Super Mario, whatever. This digital native space is all possible and people can communicate inside those spaces. Actually this is spectrum and we can imagine the blend of these two different elements so this allows us to think about the new potential or possibility of the data we already have.
But let's start from very straightforward example. This is our first metaverse project called Takenaka Dorm Reunion. This is our Takenaka dormitory. This is our design-build project, and this is exclusively for company’s first-years. And now we have the design model we took it to the AR environment and we invited the company people to exchange their latest issues. This was quiteactively done because due to the pandemic
the people's behavior was quite restricted. We realized that making a conversation inside some spaces related to building environment, it's quite valuable. So we extended a little bit. This is something different. Recently we expanded the office a little bit, but this was not a design-build; this was actually a retrofit. So we didn't have the detailed design
model. But instead, what we did is that we took the 360 video device called Ricoh Theta, we walked around the building and we took it to the simple metaverse platform called Buralit. We call for the people and they join to this virtual tour. Still this is a tour along this pre-recorded video, they can spot on and they can focus on the specific point they wanted to see because it's 360 video. And we can communicate over voice chat, text chat, or even with the stickers just like the social networking tools. This activates the conversation about the build environment, so why don't we all the time take this 360 video tool and share the building experiences with our teammates? It will activate the discussion about what we build and what's good what’s bad for the designs. It's going to be useful.
If it's useful for the internal people it's gonna be true for the rest of us. Then, this is a proposal for a property called Chochikukyo (聴竹居). Chochikukyo was originally designed in 1928 by an ex-Takanaka architect. This is a housing property. Recently Takenaka acquired this facility and now
it's designated as a national important cultural property. Because this property is located in very quiet residential area near Kyoto City, it is so challenging to host a big group of visitors to be inside here, physically. But digitally, it's going to be possible. Just we scan around the building and we open up the space for the virtual building tour then it's going to be meaningful for those who are interested. Even we can assign one building specialist to
do the building tour and answer to the Q&As. If it makes sense and we can find a way to monetize this tiny business, then we can think about the better management for this cultural heritage. This is the last example and this is tiny, tiny one. Well, you can see a girl inside and this is my four-year daughter. She loves arts and crafts and she always creates something new
and bring it to me. At first I did some pin-ups so happily, but the space is limited anyway. So I took it into the box, but nothing happens after all. So I imagine that what if I can do the quick scan of the drawing or works, and I can store it in the digital environment.I can design a tiny virtual museum and I can locate her works inside. Then I can do anything! I can open up the rooftop, I can enlarge the pictures, I can invite a group of people, and even I can invite the TV crew like this! So this costs very little, but this is actually expandable. This is not only for the children but maybe for everybody who loves the art. If this makes sense Takanaka will be so happy to design the digital-native museum space. Actually
I talked about this idea to friend of mine, he said “oh… interesting” but nothing happened after all. So if you find it interesting please talk to me I will take it so serious (just joking). Like this so we've seen four different metaverse examples. The first two were under the concept called “Metaverse as a tool.” We were actually focusing on the communication happening inside the metaverse, so metaverse was just a digital venue to host those communications, whereas in the last two examples it will be aligned to this “Metaverse as a target.” There'll be something that they may want to visit because there might be something that only there you can see. So this is
quite useful framework again to think about our purpose to look at the metaverse’s possibility. One more thing. It reminds me about this famous cartoon that a client asks you to create a swings and somewhat your engineer designed something on the left hand side but actually overall what you needed was just like on the right. So we, as the building specialist again, tend to think that the model deliverables should be always like something on the left. It's so detailed, nicely coordinated,
and this is a building actually. But for a layperson, for example to my mother, how my mother recognize about the building is that “here's the door, here's the window, here's the wall.” That's it. And nothing’s wrong with that. In most cases this is good enough. Then we can expand our way of thinking just a little bit; we could make use of the project model but sometimes just a reality capture or even just 360 photo will be good enough to provoke some communication over the build environment then it's going to be cheaper, more feasible, and more activating for the people to communicate. But I know that there's doubts and risks. Some company answered investing in a metaverse is risky because the trend will fade, or there will be issues around cyber security. Will be. And the digital life takes over real life. Hmm. It will more make sense
to me if it's gonna be rephrased like this. “Stop the metaverse, save the real world.” Mm-hmm. But it might be true. The metaverse, VR and AR, all these things look playful and somewhat it's on the buzz, and in terms of the expectation, it's a little bit overheated. In the world around you, there exist tons of issues to be solved. Some people may question that is that the very thing to be prioritized in the real life.
But I might say that maybe it is. VR, AR and metaverse help save the architecture and construction when it is used to preserve the building conditions, or if it's used for education, training or knowledge sharing. Or if it contributes in creating the better city data or it's used for the smart city where it serves for the citizens’ better life. At the first time our company, including myself, expected metaverse to make profit. And that is the top expectation of the companies. But we soon after realize that the expected amount of the actual revenue that we can expect from the metaverse is not directly comparable to the one from our main domain. Then, why don't we think about making the better ecosystem,
or better communication channel for the building lovers, to create a better communication among the buildings specialists or building lovers? That's one potential role of this kind of environments. This is the conclusion and takeaways. The VR / AR things will be important for the stakeholders because better visualization matters. Sometimes the cinematic realism is important but just a visualization “for the rest of us” will just work nice. They are important for the projects
because the metaverse can become a platform. But we all know that not everybody is a metaverse lover so it has to be comfortable, it has to be inclusive, yet non-geeky environment is needed. At the last, for our future; it’s not just for fun but for fans. So let’s think about what we can enjoy only there. Thank you so much. [Applause]
2023-03-02 01:55
