Machine Learning For Media Professionals

    |    Wednesday February 28th, 2018

What is Machine Learning?

Machine learning is a field of computer science and artificial that tries to answer a question already formulated in 1959: How can computers learn to solve problems without being explicitly programmed?
Different words for Machine Learning and similar techniques are sometimes used interchangeably, especially in the generalists press. Some may ring the bell if you have been reading the news lately: artificial neural networks, artificial intelligence, deep learning… Unless you want to become a specialist in the field you will be relatively safe if you use the term AI, an acronym of Artificial Intelligence, and the most popularly used term.

Feeding on developments like computational statistics, mathematical optimization or data mining machine learning is a more formally defined in the following way: A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.

To understand this we can think of the following situation. Imagine you have a large set of data. You can feed this data to a computer program whose function is to make informed guesses. As you review the guesses and mark them as correct or incorrect the computer program gets verified data that it can use in its next guesses. If the guesses get more accurate as time pases, we can consider that the program (and thus the machine running it) is learning on its own using data and experience. As a human would.

Use Cases

The media industry is traditionally ahead of its times when it comes to use of technology and it has already experimented with machine learning for different purposes. The two practical applications I find the most interesting are scriptwriting and film analytics, specially the predictive ones.

Scriptwriting

One of the possible uses of machine learning is scriptwriting. With the right algorithms and a good set of training data a computer program can write parts of a script. Because these are generated by machines and not by humans they are conceived in a mental schema that’s completely different from ours, so the outputs are many times surprising. An example of this interesting weirdness is a short movie written by a machine in 2016. Sunspring is a sci-fi movie scripted by an artificial intelligence and directed and performed by humans. After watching the movie it becomes clear that human editors and copywriters have a long time ahead in the industry. HBR explains why:

“Before the neural network “read” those scripts, it not only didn’t know how to write a screenplay but also had no knowledge of the English language. It learned some of the features of a screenplay — for instance, that lines of text should be assigned to characters and that stage directions should be included. Again, it learned all this just by reading a few dozen scripts.

What it didn’t pick up from all those screenplays was the art of narrative. Sunspring has no story. Its characters exist only in the sense that they’ve been given sentences to speak. The script shows how far machine learning has to go before it masters storytelling, or becomes “intelligent.” Yet the algorithm’s ability to construct sentences and to recognize basic features of a screenplay suggests that AI could play a role in the future of writing. But that future, at least in the near term, is limited.”

If we trust HBR machines may be far from writing scripts, but they are better at short pieces. They are already being used to write articles. You can test this for yourself using services such as AI Writer or Articoolo, a tool that not only writes but also rewrites existing pieces. For now it is the large publishers who are building the tools that take these capacities to another level. Just in the last year Heliograf, The Washington Post’s news bot, wrote over 800 articles. Some would think that this is only possible because a lot of press articles, like the ones about sport matches TWP automated first, don’t require that much creativity. They are based on data and news usually follow a structure.

Ads, are a different story. They require a certain touch of creativity and everybody knows the best way to sell is to tickle people’s feelings. Coca-Cola, an expert on transmitting emotions, has been using artificial intelligence bots to generate the 5 first seconds and the closing of their ads, as these have an structure that’s easier for machines to understand. A human team then takes over and fills the gaps.

Watson, IBM’s AI, was used to produce the trailer for a horror movie about AI, Morgan. Analysing the movie it could identify the moments in which the action happened, and analysing the images it could recognize whether the characters were happy, sad, frightened… and adapt the cadence of the trailer and its music accordingly. The complete process from the moment the AI watched the movie until the human team finished editing took about 24h. As John R. Smith, from IBM, puts it in an article about the experience: “Reducing the time of a process from weeks to hours –that is the true power of AI.”

Analytics and Predictive Analytics

Machine Learning can be used to analyse big pieces of data and make predictions. This will be a very important development in the next year, but hold on your production projects for now: The algorithms are far from perfect. In an experiment by an MIT lab a machine learning program was given 600 hours of YouTube videos and popular TV shows, and asked to predict what characters would do seconds after specific screenshots. Compared to humans, who could predict over 70% of the actions, computers could guess just a bit over 40%.

Companies like Valossa already offer systems based on machine learning that can automatically label moving images and determine whether we’re seeing a person, a vehicle… the algorithms have been years on the work and can detect specific faces or brands, and even generate a narrative of what is happening on a scene. This is great news for all production assistants and the people in charge of identifying nudity, violence or drug use for movie ratings. Thanks to technology they will soon be able to move away from metadata and video analysis to more exciting and creative tasks.

Using Machine Learning we can put to production all the archive data we have. Machine Learning is already helping Reuters identify fake news in Twitter, and a very interesting study by Google used Machine Learning to analyse popular movies and look for gender biases. Everybody felt women get less screen time, and there have been attempts at approaching this issue scientifically in the past, such as the Bechdel Test, but never had we have so much granular and backed data to claim the need for better female portrayal on screen.
Thanks to data and learning machines the Media Sector’s creative potential and key role in improving society are now more certain than ever before.


Author: Cristina Santamarina is a professional in the world of information technology with 10 years of international experience in the coworking, ehealth and chatbots industries. During her years living in Prague, Berlin and Conakry she has been an advocate of coworking and women in technology and written numerous articles about these and other topics. Back to Spain, with her initiative The Neon Project, she wants to put her focus on building a bridge that brings tomorrow’s technology to today’s people.
Follow her on: @crissantamarina or check http://theneonproject.org to learn more.