Horse and astronaut

Our current state of AI?

Dear readers,

A short note to highlight Gary Marcus’s latest: “Horse rides astronaut”. His newsletter is a must read to understand the hype around AI and AGI. See also my last post around this topic.

“What nearly everyone got wrong about DALL-E & Google’s Imagen, and why when it comes to AI hype, you still can't believe what you read”

Gary Marcus 🇺🇦 @GaryMarcus

🙄 @GoogleAI, “a deep level understanding”? Seriously?! Your system can’t distinguish “a horse riding an astronaut” from “an astronaut riding a horse”. 🙄

Google AI @GoogleAI

Introducing Imagen, a new text-to-image synthesis model that can generate high-fidelity, photorealistic images from a deep level of language understanding. Learn more and and check out some examples of #imagen at https://t.co/RhD6siY6BY https://t.co/C8javVu3iW

Laburnum, 1954, Oil on linen, 40 x 50 inches (101.6 x 127 cm) by Hans Hofmann. Source: http://www.hanshofmann.org/

A few notes here:

• Millions of dollars now being poured into AI research and deployment. This will not slow down (despite scandals) in the near future.

• Our regulations and government policy are yet to catch up to tackle this momentum.

  Jathan Sadowski @jathansadowski

  AI/ML research at places like Google and OpenAI is based on spending absurd amounts of money, compute, and electricity to brute force arbitrary improvements. The inequality, the trade offs, the waste—all for incremental progress toward a bad future. https://t.co/wbySsnSHyS

  

  Leon Derczynski 🏡🌱 @LeonDerczynski

  'I don't really trust papers out of "Top Labs" anymore' https://t.co/KgRKpCvlnD https://t.co/t5ORZ2DmM2

• Our focus and priorities will be around shiny astronauts and horses. Sorry to be pessimistic. However, it will probably take another year or two for this AI hype to cool down.

• People like Gary Marcus and others are critically looking at these hypes and pointing out the crux of the matter. We should listen to them more. Maybe that will create momentum for an AI that we can trust, an AI that can benefit our society.

• The problem is not that AI can draw all these beautiful and detailed images. The problem is that “we” (the companies, media, investors, users, academia…..we are all in it), take these accomplishments as a sign of “intelligence”. It is easy to be swept away by snake oil merchants of AI and technology. But we need to try harder. See my post on this: “From Snake Oil to Theranos.

  Milan Gritta 🇬🇧 🇸🇰 🇺🇦 @milangritta

  Some responses to Gary's valid criticism look like when a religious person is told that God doesn't exist. #AI #AGI #Artificial_Intelligence #ArtificialIntelligence #ArtificiallIntelligence

  

  Gary Marcus 🇺🇦 @GaryMarcus

  Horse rides astronaut: What nearly everyone got wrong about @GoogleAI's #Imagen (and DALL-E), and why when it comes to AI hype, you still can't believe what you read. A #longread into evaluating claims about deep learning, with a cameo from Clever Hans. https://t.co/ZacV6jKxJ6

Here’s another quote from Gary’s piece:

Since we know that Imagen can draw images of horses riding astronauts (if the instructions are specific enough) we know that the failure to draw a horse riding an astronaut given the prompt “a horse riding an astronaut” can only have one explanation: the system doesn’t understand what the prompt “a horse riding an astronaut” means. To borrow a phrase from DeepMind’s Nando de Freitas, it’s “Game Over” for that particular hypothesis.

DeWeese Lab @DeWeeseLab

Like all deep generative models today, Google's #imagen gives beautiful, convincing output, but can't understand + follow specific logical instructions. This inability to reason is a general failure point of modern ML methods and a key place to look for new ideas!

Mohammad Norouzi @mo_norouzi

@DeWeeseLab A red conical block on top of a grey cubic block on top of a blue cylindrical block, with a green cubic block nearby #Imagen sorry. https://t.co/gyeLnG9Lys

Perhaps with enough context and training data, the system can figure out the difference between “horse riding an astronaut” versus “astronaut riding a horse”. Ok, fine. But we need a healthy, robust tech ecosystem where innovative AI applications and critique can coincide. Currently, this is at the hand of a few big companies. Are we ready to decide on the next course of AI technology that will impact humanity such a way? I hope not.

Meredith Whittaker @mer__edith

This problem is structural: 1. the ML community is all but entirely dependent on BigTech infra/data/$ 2. Reckoning w criticism means taking it seriously. Taking it seriously unravels the "AI" narrative, which in turn undermines the fortunes of Big Tech & the ML community w/it

Gary Marcus 🇺🇦 @GaryMarcus

@TaliaRinger The most serious problem in machine learning right now is its inability to reckon with serious criticism.

That is all for today.