“I see AI in relation to dance as an environment, a learning process, a practice to involve a body into a dialogue with new media.“

Does a machine have the potential to choreograph, and if yes, how do artificial and human intelligence engage in a (folk) dance conversation? In K L O F. cyberographies of folk choreographer Irina Demina and computer scientist/programmer Dávid Samu explore the possibilities and potential of a dialogue between traditional and digitally stimulated choreographies by integrating the traditional folk lexicon with digital machine learning technologies.
There are two dancing bodies on stage. One is a human body. The other is an artificial intelligence. Both had traditional dance training, only that one of them has dedicated five years at an academic dance school, learning folk dances from across the world, and the other one is a machine learning algorithm that has been trained to autonomously create artificial "folk dances" via a code. This begs the question of how "traditions" might be reinterpreted.In order to investigate new options for the creation of "hybrid" dance vocabulary, these investigations use motion capture and machine learning technologies.

The concept of your performance program is: "Humans and artificial intelligence enter into a (folk) dance dialogue." What inspired you to combine AI with dance?

As an artist I am always curious to step on new territories and explore opportunities for dialogues with different disciplines and creative processes. During the first Covid lockdown, when all live performances were cancelled and one couldn’t even go to a rehearsal studio, we set up with Dávid Samu, the programmer of “KLOF”, a “garage” motion capturing studio in the living room and started experimenting with recording dances, just out of curiosity. Since I was formally trained as an academic folk dancer in my teenage years, we recorded some folk-dance samples. And the idea came to synthesise folk dance samples from the whole world into a “hybrid” neo-folk dance, that can be learned and danced by everyone. Such a utopian task that one wouldn’t trust any human creator and would rather delegate to a machine. Dávid started working on the code. And suddenly I found myself in this state of being curious, excited, at the same time a bit unsure about all the usage of AI in combination with traditional or historical dance practices. We continued working on this idea with Dávid, and as a dance artist I have entered this new process of learning to interact with technology and developing certain sensitivities towards it. This may sound paradoxical, but I am curious if this relationship with technology can assist the body to find ways of returning to itself, and to enter a renewed dialogue with senses, habits, traditions.

Can you tell us a bit more about how your AI works in detail and how you taught it to dance?

What Dávid and I wanted was to generate novel dance forms (motion sequences) by freely mixing and merging original motion motives from a variety of traditional folk dances around the world, without the inevitable bias that a human choreographer would bring to this creation process. We decided that we can create a "machine choreographer" for this purpose by using fairly standard generative AI approaches - first train a neural network to encode a wide range of motion forms and motives, and then let the network “hallucinate” or dream novel motion sequences based on these learned internal encodings. As the novelty, unpredictability and sample-diversity of the newly generated sequences was key to the project, we chose a probabilistic AI approach (a variational autoencoder model) and introduced a fair amount of randomness in the AI training process, to prevent the AI to simply memorize our relatively small training data set. Once the AI model was trained, we prompted it to generate synthetized choreographies by setting its encodings to a random flow of internal states, almost like a dreaming process for an AI system.

Which data was used for the machine learning of dances?

To train the AI model, I was recorded dancing representative clips of each of 26 different folk-dance traditions - roughly 1-minute long. We wanted to generate dances in 3D, therefore we had to use motion capture technology for the recordings. As studio-grade motion capture technology is still very expensive, our budget posed a limitation here, but luckily, we quickly found the solution in the form of a cheap and old Microsoft Kinect device, originally created for motion tracking enhanced computer gaming.

People who want to learn certain dances professionally usually do so in a dance studio. To what extent do you think a dance algorithm/ AI can replace or complement a choreographer?

Learning dances in a dance studio by copying a teacher is a formal way, but definitely not the only one of passing dance knowledge. Folk dances are being learned in the community, by practising, this legacy is being passed from body to body, kind of combining and enriching body archives. Dance is way more than a sequence of body shapes and positions of joints in time and space. When you think about dancers who deeply move you, it’s probably not the aesthetics of physical shape, but the ability to express something unspoken through the body, to tell stories without words, to share experiences, to connect to oneself and to others through dance. In this sense AI generated dances cannot substitute the magic of people dancing. But AI can be an invitation to rethink and rediscover creative practices, to renew dance vocabularies and to enrich movement languages with new approaches. I want to believe that technology can serve in this way to reveal previously hidden or unusual aspects of the body, and, at the same time, reflect a renewed vision of ourselves and the way we move and think. 

Dance has a lot to do with spontaneity, harmony, but also body control. How difficult was it for you to create a choreography with a machine? Would you say there is a lead?

I would phrase it like this: the algorithm created a movement sequence, but for this to become a ‘dance’ it needed to be embodied by a human dancer. It was actually quite challenging to learn the 2 minutes long AI synthesized movement sequence, it took us (dancer Viktória Kőhalmi and me) almost two weeks and it was a tough process of learning it frame by frame. Every dancer develops her/his own movement patterns and every choreographer – her/his own dance language. KLOF sequence, created by the algorithm, made us both step out from our comfort zone and break our own patterns of dancing and thinking. Some body positions or transitions were simply uncomfortable, inorganic, kind of made no sense, but this is exactly what was interesting – to challenge ourselves with embodying a different logic of movement interpreting this movement sequence, and to see how it turns into dance. And Viktória performs this dance in an exceptionally impressive and powerful way.

Finally, how can AI further enhance the art of dance? How would you describe the fusion of AI and dance?

I see AI in relation to dance as an environment, a learning process, a practice to involve a body into a dialogue with new media. I think that tradition and innovation can beautifully merge together, so that dance and AI create a curious, mysterious and unpredictable symbiosis. It can provoke and invite to reveal previously unseen, unnoticed or unexpected aspects of the body and expand our sensitivities about the art of dance. I myself would like to explore this relationship further and learn more about programmes combining AI with performing arts.

Thank you for your time and the interesting input.