Google Duplex: Does it Pass the Turing Test?

Google Assistant's new feature can make a real phone call to make a reservation on your behalf.

In 1950, a renowned computer scientist Alan Turing proposed the question “Can machines think?” in his paper Computing Machinery and Intelligence [1]. He predicted that by 2000, machines will be able to communicate with humans in such a way that they are indistinguishable from humans 30% of the time.  In the same paper, he described guidelines that would later become the golden standard for artificial intelligence development for the years to come. His method of testing whether or not machines are indistinguishable from humans later became known as the Turing Test. Many scientists have attempted to develop artificial intelligence systems to pass the Turing Test, but none are widely viewed to have passed the Turing Test.

Fast forward to May 8, 2018. At the annual Google I/O Annual Developer Conference, Google blew the audience away by showing off Google Duplex [2], a machine learning based natural language processing system built into Google Assistant. Google played a recording of the Google Assistant making a real-life phone call to a hair salon and successfully scheduling an appointment by holding a full 1-minute conversation with the hair salon. Even when the hair salon said there were no availabilities at the time requested, the Google Assistant was able to ask further questions to find the next best time for the appointment and complete the booking. The best part: the hair salon assistant had absolutely no idea that she was talking to a robot. Because of this, many believe that Google Duplex is the technology that comes closest to passing the Turing Test.

Google Duplex is enabled by Google’s research in machine learning and deep neural networks, which are found in all stages of the Duplex process [3]. Duplex uses Google’s Automatic Speech Recognition technology (ASR) to understand the input (e.g. the hair salon), a recurrent neural network (RNN) to process the question and form an answer, and a deep generative and predictive model (WaveNet) to turn text into speech that sounds like a human [4].

For Google, machine learning is an essential tool that enables them to close the communications gap that exists between machines and humans. Companies have made tremendous progress towards artificial intelligence in the past few decades, but so far, success in imitating human speech and behavior has been limited to chatbots such as Eugene [5], with no human-like voice and lacked an authentic connection with the human participant.

Google realizes the importance of enabling people to have a natural conversation with a machine, and the possibilities of a technology that would enable that are endless. However, natural language processing is very difficult. It goes beyond pure interpretation of words, and machines need to take into account sentiment, sarcasm, and hidden meanings among other things [6]. Google also realizes that it is impossible to develop a machine that would be able to hold a natural conversation without the ability to learn and adapt to different scenarios of conversations. Therefore, Google is heavily investing in and developing machine learning systems to enable natural language processing. Machine learning is critical in enabling Duplex pick up on sentiment, hidden meanings, and other nuances to language that are required for Duplex to hold a natural conversation with a real human.

However, Google must consider and address a few concerns in the short term and the medium term before launching Duplex on Google Assistant. First, processing audio inherently means that the call must be recorded, which could infuriate many listeners (those talking to Duplex) if Duplex does not reveal that it is in fact not a human and is recording the conversation. To avoid potential lawsuits and public outcry, Duplex should consider identifying itself as a bot before engaging in the conversation.

Additionally, Google must be careful with the distribution strategy of Duplex. Google promotes an open source ideology for many of their projects, and Google’s current machine learning framework (TensorFlow) is available open source [6]. However, there are concerns about Duplex (or a version of it) being exploited by telemarketers who could use the technology to irritate millions of people around the world every day.

Google Duplex comes close to but not quite passing the Turing Test. Currently, Duplex has only been proven to making reservations on behalf of its users. However, a few questions remain: how will Google scale Google Duplex to be able to cover more applications where Duplex will be valuable? What other Google services will Google integrate Duplex into?

Word count: 738


Endnotes:

  1. A. M. TURING; I.—COMPUTING MACHINERY AND INTELLIGENCE, Mind, Volume LIX, Issue 236, 1 October 1950, Pages 433–460, https://doi.org/10.1093/mind/LIX.236.433
  2. “Google I/O 2018.” Google. Accessed November 12, 2018. https://events.google.com/io/recap/.
  3. “Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone.” Google, https://ai.googleblog.com/2018/05/duplex-ai-system-for-natural-conversation.html.
  4. Van Den Oord, Aäron, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew W. Senior, and Koray Kavukcuoglu. “WaveNet: A generative model for raw audio.” In SSW, p. 125. 2016.
  5. Verduijn, Xander. “Chatbot Eugene Goostman.” Chatbots.org. Accessed November 11, 2018. https://www.chatbots.org/chatterbot/eugene_goostman/.
  6. Socher, Richard. “AI’s Next Great Challenge: Understanding the Nuances of Language.” Harvard Business Review, July 25, 2018.

Previous:

Personalization in the Online classifieds industry – OLX path to a customized experience

Next:

Minority report: machine learning and crime prevention in China

Student comments on Google Duplex: Does it Pass the Turing Test?

  1. I really enjoyed reading this post, especially because I have seen the video of the Developer Conference where Google showed off Duplex. I thought the questions you raised were really thoughtful, and I appreciated your concern regarding privacy laws and other potential regulatory concerns. One topic which you did not touch upon which I’d be interested to hear your thoughts on would be the creation and proliferation of chat bot assistants across all companies or perhaps across all humans. This would mean that humans might one day not have to interact with a bot or with another human at all for these kinds of scheduling needs; perhaps the bots will just talk among themselves! In this case, natural language processing might not even be that important as it is likely these bots will develop their own languages to communicate with each other.

  2. Thank you for the interesting read! I too remember watching the original launch of Duplex and being amazed it its potential.

    I believe the questions you posed are important. In my opinion, I do not believe there are many current Google services, aside from a phone or Google Home ‘assistant’, where this natural language processing technology can be implemented. However, I do see a real potential for Google to commercialize this technology and apply it to customer service process improvement. For example, if I were to call my bank for a simple query, I see no reason why this can’t be handled by a machine.

    If Google were to commercialize this technology into a customer service product, I believe they would need to position it as a win-win for the business and end-consumer. By doing so, they could minimise backlash against the business for eliminating human jobs. For businesses, the benefit is clearly a reduction in customer service headcount. For end-consumers, I would argue it leads to improved customer service levels, as machine-agents waste no time searching for information or transferring calls to other departments, leading to shorter call times and higher customer satisfaction. Indeed, a recent Oliver Wyman report noted that use of virtual agents could reduce call times by up to 20% [1].

    However, I agree, for the sake of our sanity, it should not be applied to telemarketing.

    [1] Oliver Wyman, “Customer Service in the Age of Siri and Alexa”, Dec 2017, https://www.oliverwyman.com/our-expertise/insights/2017/dec/customer-service-in-the-age-of-siri-and-alexa.html, accessed November 2018.

  3. I was drawn to this post because I remember seeing the video of Duplex’s reveal. I was completely left in awe at how human-like Duplex was when scheduling an appointment on behalf of a person. I am now aware that Google made Duplex really well at scheduling appointments but not at other things. It makes sense that they started with a relatively simple use case. This use case is relatively simple because scheduling appointments has pretty much a define set of variables and potential outcomes. It will be interesting to see how they tackle on harder problems like having Duplex understand double meaning, intent, puns, and other parts of speech that are natural to humans but not straightforward for machines.

    The article also brings up the potential of legal issues which is a really good point. I personally wouldn’t want to be recorded by Duplex, especially when I’m acting on behalf of my company where I work with sensitive information.

Leave a comment