It’s hard to get paid to do what you love.

Perhaps no one understands this better than dancers, but Taku Kodaira and his team at Mikro Entertainment are on a mission to fix that.

But this conversation, and Mikro Entertainment itself, is about much more than dance. Mikro’s marketplace for dance moves is just the first application of Mikro’s new motion-capture technology, and things are just getting started. Today, Taku and I talk about the surprising economics of dance moves, the adoption curve of disruptive technology, dance-move lawsuits. and one very important law that looks like it is about to change.

It’s a great conversation, and I think you’ll enjoy it.

Podcast: Play in new window | Download | Embed

Subscribe: Google Podcasts | RSS | More

Show Notes

  • How you sell a dance move
  • Making a market – who is buying dance moves
  • Why growing up international made it easier to start a startup
  • How copyright law needs to expand
  • One danger in allowing dance moves to be copyrighted
  • Lawsuits against Epic Games over Fortnight dances
  • How big is the motion capture industry
  • The adoption curve for  disruptive technology
  • Why it is impossible for any startup ecosystem to have enough engineers

Links from the Founder

 

Transcript

Welcome to Disrupting Japan, straight talk from Japan’s most successful entrepreneurs. 

I’m Tim Romero and thanks for joining me.

Truly disruptive technology is usually hard to spot when it first shows up. Sure, after the IPOs and the mass market success, everyone claims that they knew it all along. But in the early days, disruptive technology is usually shrugged off as being too simplistic or unprofitable or most often, just a solution looking for a problem. 

When Kodak invented the digital camera, they dismissed it as a toy with no real commercial applications. LED light bulbs were first written off as impractical. And in 1911, the military brass dismissed the airplane as, quote, “a scientific toy with no military value.” All of these seemed like, well, solutions looking for problems. 

We’ll pick up that thread later. But I want you to keep it in mind as we sit down today and we talk with Taku Kodaira, the founder of Mikro Entertainment, who’s developed technology that can create full 3D motion capture models for mobile phone videos. 

Now, Taku’s initial and current application of this technology is the world’s first global marketplace in dance moves, and he has some of the world’s most famous dancers signed up on the platform. 

But this is a conversation that will take us on a journey of how digital dancing is already being monetized in gaming and social media, about copyrights in dance and plagiarism and choreography. And we’ll also explore the new uses and new markets that this technology will open up in the future. 

But you know, Taku tells that story much better than I can, so let’s get right to the interview.

Interview

Tim: So I’m sitting here with Taku Kodaira, the founder of Mikro Entertainment and GesRec motion capture marketplace. And thanks for sitting down with us.

Taku: Thank you very much, Tim.

Tim: Now, what you guys are doing, it’s really amazing tech, but you know, you can probably explain it a lot better than I can. So what does it do and what are you selling?

Taku: Right. So just starting about the name GesRec, we tried to combine two words gesture and recognition and tried to create like a one word. My wife is a dancer, and I’ve been talking to her and she told me all the difficulty dancers are facing, and we just realized, okay, these people are doing so much stuff out there. Is there any way we can try to support them? Right now we are utilizing our technology to capture 3D motion and turn it into the data from the 2D video. And we are creating a marketplace that we sell and trade those 3D motions that’s actually coming from a lot of people, mainly right now dancers and choreographers.

Tim: Okay. So I mean, I totally — I mean, in another, what seems like another life I used to be a professional musician, and I completely understand that even compared to musicians, dancers have a really hard time making money and supporting themselves. So anything that supports that is good. 

Taku: Exactly.

Tim: So the dancers, the choreographers, they take videos of themselves using just a regular cell phone camera. You guys can extract the 3D modeling from this and put it up on the on the website.

Taku: Yes. And what’s amazing is they don’t really need to take the video again, because throughout their careers they have a lot of videos already on their hands, they can just hand us the videos they have.

Tim: So we have these dancers uploading their videos to the marketplace and you guys rendering this into 3D models. So tell me about your customers, who’s on the other side of this, who are buying these dance moves.

Taku: We are actually looking at slightly different customers depending on the categories. The first customer as we saw was the 3D artists or production companies. Currently, we see a big boom where a lot of people are able to work on the 3D models. Because of PCs getting faster, there are more options for the software. The second customer we see is more in terms of like gaming and social media. As you probably know, on the gaming world, the metaverse is really there expanding. Right now people have their avatars and the next thing they want to do is make them do the motions, especially coming from somebody who has creatively created it. And the last one is a little bit similar, social media is expanding to the 3D world plus the avatar. So similar things with the gaming.

Tim: Yeah. I can see how that’d be used that way. I mean, the potential is fantastic. But I mean, who are the customers specifically rather than the user? For example, like Fortnite dancing has been really in the news, it’s the first thing that kind of comes to mind here. So it’s the gaming companies who will buy these models, social media, I guess, would be the social media platforms that would buy these models. So is that the same customers you have or are you trying to do more direct or something broader or something different?

Taku: Right now we are doing things in combination. We are in discussion with the gaming companies and we are also discussion with the social media companies. The problem there is a lot of times, dancers or choreographers are feeling that their work has been stolen. So they don’t get credit, they don’t get the revenues. And that’s something we’re trying to change here. What we can actually do is we can be in between those gaming or social media companies and the ones who’s creating the motions. What we are seeing in the very near future is that they can collaborate together as a kind of like influencer and platform relationship.

Tim: Before we dig deeper into that, because there’s a whole fascinating area of motion rights and an extension of copyright here. But before we get into that, I want to ask a bit about you. 

Taku: Okay. 

Tim: You spent a lot of your time overseas when you were growing up and in your early career. Did that have any influence on your decision to start your own startup?

Taku: I would say so, actually, yes. Just a little bit of my life, I was born in Singapore. I spent a lot of time of my job in Hong Kong, about eight, nine years. And I finished high school in the Philippines. I got a college degree in the United States, a master’s degree in Japan. So that kind of got me into the startup. Also, what we see as a trend in Japan is most of the startups are focused on the Japanese market because we are here. But as Mikro Entertainment, we tried to go into the global markets.

Tim: And you mentioned that your wife is a dancer. Was she kind of your inspiration for starting this or is she involved in building and running the startup as well?

Taku:  Absolutely. So I got a lot of inspiration, especially in the beginning. And now she’s also involved into our business as well. In the beginning, she was more like somebody who gives me inspiration and also tells me how things work in Japan around dancing. And now she’s expanding her career as well. She started to learn about coding, something I can’t do. She’s also into those 3D modeling stuff. So again, this is another area, I can’t really do it myself.

Tim: Oh, that’s awesome. Let me ask you a bit about the technology and then we’ll dive into the business model. So looking at it from the outside, this is kind of amazing technology, right? Because 3D motion capture has always been, you know, people in the motion suits with the ping pong balls stuck on the outside, you know, specialized cameras. It’s been a very specialized and expensive process. So the fact that you can extract it from a single camera view is pretty amazing. So can you dig into that technology? How exactly are you doing this?

Taku: The very simple way I explained is we can take the 3D motion out of a 2D video. And there are some other companies who specialize into this, most of the time what they’re utilizing is AI. Our technology is a combination of AI, physics motion, and some other stuff. We are pretty confident we are one company that can provide the most precise or at least getting very close to those motion capture suit quality motion extracted from 3D video.

Tim: Are you free to talk about the technology? Disrupting Japan listeners tend to be a pretty technical bunch. So can you share anything about the specific technology or techniques you’re using?

Tim: Sure, just a little piece of it. As I mentioned, we are combining all sort of technologies out there. So one thing to mention about AI, for example, we are combining several different AIs. One is taking the finger motion from those 2D video, and another one is taking the whole body motions. And there are another things that combining those finger motions and the body motions. Of course, if we just combine those AI together, then things looks a little bit not correct, I would say. So we’re making sure that the model doesn’t go below the grounds and the model doesn’t go too high. A lot of part I would say is AI. Another one is, as I mentioned, physics engine and there are another cutting edge technologies as well.

Tim: Do you also do some post-processing? Is there a human component of like a final tuning or quality check in the process or is it fully automated?

Taku: Right now there is a little bit of a space. One reason is, a lot of times, although our outcome is fairly okay or fine, we get some requests from the client as well. And one interesting experiment we did is can we extract the 3D motion out of those anime? 

Tim: Oh, yeah.

Taku: The result was actually amazing. Surprisingly, our results provides sometimes smoother outcome compared to the original anime. Because, you know, sometimes, depending on the budget, it’s not really as smooth as the human motion.

Tim: Yeah, but that is interesting. It’s always interesting to find out what AI is good at and what AI has difficulties with because it tends to surprise you.

Taku: Right.

Tim: So in human motion capture, whether it’s animation or live video, what are the parts that the AI tends to get right most of the time and what are the parts that you and the editors just have to double check and go in and make sure it didn’t make another mistake on?

Taku: Something that AI is good at, it really depends on what kind of data we have. We used to differentiate what people can do and what AI can and cannot do. And these days, I think people are switching more to, well, it really depends on the database. What we are really good at, especially when we talk to the clients discussing about the motions extracted from anime, because we are trying to extract something that simply doesn’t exist, there is no right answer to it. Because in between motions, there is simply nothing there and we are trying to fit into the gap. The reason why we sometimes need to alter there is the client says, “Well, you don’t see it here but there is a motion here that we didn’t really put it into anime.” That’s kind of like the place where AI gets confused, because it’s not there and it’s not in the database too.

Tim: Right, right. It’s something that human beings will see without thinking, right, but an AI just doesn’t have that frame of reference. 

Taku: Exactly. 

Tim: Okay, that makes sense. 

Taku: Over time, if we do this for the anime, hopefully everybody’s saying something similar. The AI is going to sense it, “Okay, I know you meant this and this is what you want.” So I think we are gradually getting there.

Tim: Hey, and is there a library that listeners can download or a website where they can upload videos to try this out themselves?

Taku: Right now we are working on the kind of a closed beta but we are planning to open this up to the regular public. But we are actually very open to requests. So if there is anybody who’s willing to have something be processed, please contact us. No guarantee we can work on it but we’ll try our best.

Tim: Well, we’ll definitely put a link in the show notes so everyone will be able to go there and see for themselves.

Taku: Great, thanks.

Tim: Okay. Let’s dig into the business model because I think you guys are on some really new and interesting ground here. So you’ve signed up some really famous dancers onto this platform. And let me ask you directly, what are they expecting to get out of this? What do they want from this platform?

Taku: I wouldn’t say everybody is looking for the same thing. But as we talk to a lot of dancers and choreographers especially, what they want first is the credits. In the past, a lot of times, especially choreographers, a lot of times their name isn’t really shown or credited, very different from what’s happening in the music industry. So a lot of people say, “At least my name should be there. I want it to be credited.” That’s one thing. Another thing we are trying to do, of course, if they have created this work, you get part of the pay. That’s what we think it’s fair. And when we get some revenues on our platform, we’ll try to distribute it as many times as transaction happens. 

Tim: I think those are both admirable goals and understanding desires that the desire to be credited and to make money off of your work is fundamental to every artist, this one included. But it seems like this is breaking new ground for copyright law. Can dance moves be copyrighted? I mean, today, are they considered copyrightable?

Taku: Yes and no. Well, legally, it can be copyrighted. But the difficulty out there for dancing is who’s got the original move. Everybody’s getting inspired. Everybody is inspiring somebody else. So it’s really hard to say, well, this belongs to this person, and this belongs to this person. One of the very famous choreographers in the United States is now copyrighting his choreography that’s shown in the very, very famous song. And that’s being a battle. And I mean, we are very supportive for that kind of movement, but we are a little bit worried about things getting a little bit similar to what’s happening in the music industry.

Tim: But I mean, it does seem to be analogous to the music industry, where you’ve got — I mean, the choreographer is very similar to the composer, the dancer is very similar to the performer, and there’s very clear right given to each.

Taku: What we worried is a very extreme case, just as an example, so one person copyrighted one motion raising right hand. Going forward, somebody else who is going to raise their right hand needs to pay something. What we are worried is more we go into that battle, the next generation dancers and choreographers, their creative work is going to be limited.

Tim: Yeah. I could see that being a concern if it’s done wrong. But this is something that actually seems, I mean, it’s happening now. I mean, dancers have been suing Epic Games over the last year or two because they’ve been copying famous dance moves for Fortnite dances, right?. 

Taku: Right.

Tim: How have those legal cases been playing out?

Taku: It’s very hard to win 100% on those cases, because a lot of times, okay, this part seems to be original but this part isn’t. And a lot of times it ends up kind of like nowhere saying, okay, some part you’re right, some part you’re not right. And in case the one person wins, another person’s going to sue that person saying, “Well, you got some of my parts being copied.” So we are kind of expecting that kind of endless battle there, and that’s one of the reasons why we wanted to take a slightly different approach there.

Tim: I can certainly see that being a concern, especially if we open the floodgates all at once. But how is your approach different?

Taku: So what we are doing, okay, so when somebody is dancing the moonwalk, let’s say, there are hundred ways to do the similar or the same motions. Depending on the dancer’s creativity, it’s going to come out slightly different. What we are saying is although everybody is doing the same moves, if it’s coming from the different person, it’s a different artwork. We try not to call it the right for the choreography because choreography is like one single stuff that can be distributed to a lot of people, but when we say motion is coming from specific person or people and each motion is very different although the choreography may be the same.

Tim: So it’s focusing on, I guess, effectively, performance, the performance itself.

Taku: Right.

Tim: I mean, it makes intuitive sense to me that dance should be protected the same way that, I don’t know, music and literature or paintings are protected, right? It’s clearly artistic expression, right? But without these motion rights being recognized in the courts, how do you make money? I mean, clearly the digital representation of the motion is copyrightable. That’s unquestionably your property. But without the legal framework for motion rights, how do you make sure that the dancers and the rights holders get paid?

Taku: Right now at this moment, we’re pretty much the only one marketplace that’s providing the official motion. So as long as we distribute part of the revenue, it’s fine. I mean, nobody’s making money now and we are and we will be the only one. You know, if somebody else comes into the market, it’s fine. But as far as the ones we are working with, everybody show up on our website, we have a contract with them, they will receive certain amount of revenues distributed from our platform.

Tim: Okay, that makes sense. And your ownership of the digital representation will protect that contract. 

Taku: Exactly. 

Tim: But if you had a competitor, an evil competitor, under current copyright law, could someone just download dance videos off of YouTube, start extracting the motion capture and selling that, with copyright law where it is today?

Taku: Legally, yes. 

Tim: Oh, man. 

Taku: Right now, as I mentioned, we are the only one that can extract as high quality as getting close to the motion capture suit. So hopefully, we get the votes from dancers and the people around the creative industries and make this kind of like a framework or the standard. So whenever any competitor comes in, they will need to work with that standard, hopefully.

Tim: Yeah. But I also imagine, with the technology advancing, it will probably force the courts to re-examine how motion rights are viewed. If we reach a place where people can just from any video extract a digital representation of someone’s performance, then yeah, that seems like it’s clearly breaking the spirit of copyright law, doesn’t it?

Taku: We totally agree. We started this project trying to support dancers and choreographers and creative people around the industry. Although it’s not our responsibility legally, we think we are responsible to create this framework and make sure that’s going to be reflected on the copyright law.

Tim: Is dance plagiarism a thing? Do dancers and choreographers worry about other dancers stealing their moves?

Taku: It’s been a huge discussion for the past years. Another reason we didn’t go to copyright the choreography is that now dancers become influencers. When we look at the TikTok, a lot of people are actually imitating what those other dancers are doing and that’s one of the way they get the buzz. So imitating isn’t really a bad thing as long as you get the credits and part of the revenue is being distributed.

Tim: It’s kind of amazing to think like copyright law was at a time where it was printing presses of the written word and musical scores written out so people could play it in their homes, and 200 years later, we’re still trying to make it work with video.

Taku: Exactly.

Tim: So long term, for Mikro Entertainment, I see two business models in front of you here. I mean, are you looking at being a rights management company like ASCAP in the US or are you planning on being more of a marketplace or maybe just diving deep into the technology? Which direction do you see Mikro Entertainment heading?

Taku: Right now, both of them for the dancers and choreographers and motion isn’t there so we are planning to take both roles. Once things become more formal and those current framework people, especially somebody who’s managing those patents or copyrights, they want to work with us, we are more than happy to work with them, and we can focus on trying to maximize the value of motion.

Tim: How big do you think this market is looking 5, 10 years in the future?

Taku: Well, just looking at the motion capture suit industry, it’s already several hundred million dollars industry. And what’s going to happen next, okay, the metaverse or gaming world is expanding, and the motion becomes like a language there, where everybody buys emotes. And the next thing we’re seeing is it can be applied to a lot of industries. We are in discussion with the medical industries, for example, when we combine the biological data with the motions, we can probably utilize it to make sure people don’t get disease or gets physically damaged. Another area could be robots can be applied to farming as well. And we are also in discussion applying this to security. Everybody’s motion is very unique just like a fingerprinting. We are seeing tremendous potential out there.

Tim: But it seems like this is, I mean, the standard disruptive technology flow. I mean, this seems like a perfect example where you have something that is extremely inexpensive, that is okay, maybe not quite as good as motion capture suits but can get the job done. But it would seem to me that it would open up a whole lot of other areas. I mean, the ones you mentioned are good. I mean, in medical for like rehabilitation, for sports, for just, I don’t know, helping people with their golf game. It seems like there’s a ridiculous number of possible applications for the basic technology. 

It’s really interesting that you’ve got either as a rights management or as a deep tech on motion capture or as the dance move marketplace are all good, potentially lucrative directions. But man, those are three very different businesses, three very different companies.

Taku: Exactly. Our main focus is motion. That’s one of the reasons why I mentioned when the existing companies or the market is willing to work with us for managing the copyright, we are more than happy to work with those companies and focus on the focus areas. What we are foreseeing here is our technology is going to get only better. So we are getting close to the motion capture suits. That means, within the next few years, we are aiming to be as good as or even better than those motion capture suits. I mentioned about the smooth processing about the anime. What we are seeing here is we are studying the muscle movements and we are also doing some research on those expectations, so maybe very soon. We don’t need the whole video but if we just have a little piece of it, we can try to estimate what happened before and what’s going to happen now.

Tim: Okay. Oh, that’s fantastic. Hey, listen, Taku, before I let you go, I want to ask you what I call my “Magic Wand” question and that is, if I gave you a magic wand and I told you that you could change one thing about Japan, anything at all, the way people think about risk, the way people approach innovation, the education system, you could change anything at all to make things better for startups and innovation in Japan, what would you change?

Taku: I would want to change a lot of stuff. But if it’s only one then maybe English communication.

Tim: Why is that? 

Taku: I told you I used to work in large companies and I met so many smart people out there. Working for the global company, English skill becomes kind of like a limitation. And that’s something I also see within the startup. The market in Japan isn’t really as big as ,of course, the international markets, and once we can communicate in English, a lot of things happen. So for example, majority of our employees are bilingual, that’s because we have known Japanese people too. And making English kind of like a standard language made us able to hire the best people in the universe, doesn’t really matter if they’re Japanese, Chinese or American. So organization-wise, it’s got the huge potential out there. And the market, as I mentioned, is larger outside of Japan. So I think we should try to aim the larger market with a stronger organization.

Tim: You know, that’s interesting. A lot of people do talk about the importance of English language skills, but most of the time when people talk about it, they’re talking about it in terms of like exporting, being able to address a global market. But you bring up a very interesting point, that it’s more about being a global company and having staff from around the world and adopting global practices. 

Taku: Exactly. 

Tim: Do you see that happening more and more in Japan? Because there are a lot of startups, at least in Tokyo, are employing a very international staff these days.

Taku: At least around myself, I see more like a trend, especially when we hire engineers. Of course, there are so many great engineers in Japan and that means there are even more engineers outside of Japan. I see things happening and I think something like Disrupting Japan, you guys are also contributing to switch the trend in Japan.

Tim: Yeah. We definitely need engineers in Japan. The more startups you have, the more engineers you need. The more engineers you can get together, the more startups kind of emerge on their own. 

Taku: Right. 

Tim: So it’s almost like by definition, you’ll never have enough engineers. If you get too many of them together, they break off and start forming their own startups.

Taku: Yeah. And you know, not just for the engineers but when we hire designers, marketers, especially when we are aiming for the international market, it’s better to be diverse organization. So we get a lot of interesting ideas even a lot of times coming outside of Japan. 

Tim: Well, I think it’s great that Japan startups are the ones kind of showing industry how to be global companies. 

Taku: Yeah. 

Tim: Well, listen, Taku, thank you so much for sitting down and talking with me, I really appreciate it.

Taku: Thank you very much, Tim. It’s been a pleasure.

Outtro

And we’re back. 

I have to say I love the idea of dancers and choreographers getting paid for their work. And I mean, who isn’t in favor of that. Of course, to make this work, international copyright law needs to be expanded to cover it, and maybe it will. 

As the technology becomes available to extract, digitize and own the movements that someone else has made in a video, it seems like copyright law will be expanded to cover it. It’ll have to catch up. 

And I’m not a lawyer, but I’ve got a lot of experience with copyright from the creator side. I’ve made money performing and selling music, selling publication rights of my written articles. And a few years back, I was paid a few thousand dollars to grant the rights for Disrupting Japan to be used in an educational setting. 

Copyright law is essential, but it was originally designed to protect books and sheet music for 14 years. And it’s now been expanded to cover, well, almost all forms of expression and creation in terms now lasting over 100 years. Copyright law really needs to be rebuilt from first principles to match the age we live in. But you know, that’s a discussion for another day. Now, that day is coming for sure but it’s not today. 

Getting back to Taku and Mikro Entertainment, one of the biggest risks to their current business model is that scaling it up largely depends on the formal expansion of copyright law. Now, that’ll probably happen but it’s hard to know when. During the intro of this episode, we talked about solutions looking for problems. 

Ah, I bet you thought I’d forgotten about that! No, no. I promised we would pick up that thread so here we go. 

In a very real way, Taku and Mikro have a solution looking for a problem, and that’s a good thing. Their first use, the marketplace for dance moves, is a great one. And they already have some of the world’s top dancers on board. Something very good is going to come out of this. 

But the bigger story has yet to be told. As this solution develops, it’s going to find bigger problems. In our conversation, Taku and I touched on a few – gaming, sports, medicine – those are valid and important. 

But the way disruptive technology tends to work is that the big changes are only obvious in hindsight. The most important, the most profitable, and the most transformative application has yet to be discovered. 

 

If you want to talk more about dance moves or motion capture, Taku and I would love to hear from you, so come by disruptingJapan.com/show184 and let’s talk about it. If you leave a comment, I guarantee Taku or I or maybe both will respond. And hey, if you get the chance, please follow us on LinkedIn or leave a review on iTunes or your podcast platform of choice. Or you know, if you like the show, just tell a friend about it. In this age of over-the-top hype and fake it till you make it influencers, an honest recommendation really means a lot. 

But most of all, thanks for listening. And thank you for letting people interested in Japanese startups know about the show. 

I’m Tim Romero, and thanks for listening to Disrupting Japan.