How AI Reunited The Beatles for “Now and Then”

The Beatles, guided by producer George Martin, were famously pioneers of new technology in making seminal studio albums like Sgt. Pepper’s Lonely Hearts Club Band and The White Album; so using an AI tool to complete their final song should be seen as a natural evolution.

As John Lennon’s son Sean Lennon says in a making-of video, “My dad would’ve loved that because he was never shy to experiment with recording technology.”

“Now and Then” was built from a recording made by John Lennon shortly before his murder in 1980, using the same AI technology that director Peter Jackson used in his documentary Get Back to clean up and separate voices in archival recordings.

Co-produced by Paul McCartney and George Harrison’s son Giles Martin, the track features elements from all of the Fab Four — including a Lennon vocal track that was first recorded as a demo tape in the 1970s.

The track was included on a cassette labelled “For Paul” that Lennon’s widow, Yoko Ono, gave the three surviving Beatles in the 1990s as they were working on a retrospective project.

At the time the band members tried to complete Lennon’s demo but considered it unsuitable for release. It wasn’t until July 2022, Jackson told David Sanderson at the Sunday Times, that McCartney contacted him for his help in producing a new version.

READ MORE: I knew about secret Beatles song years ago, director reveals (The Times)

The audio software, called Mal (machine audio learning), allowed Lennon’s vocals to be separated from the demo. The track was then rebuilt with new performances from McCartney and Ringo Starr, along with Harrison’s guitar parts from their shelved recording session in the SC.

As is clear from the making of doc the chief difficulty with using the original cassette recording was that Lennon’s vocal was too indistinct in places with the piano and the sound of a TV playing in the background.

Weta’s AI, developed for Get Back, managed to cleanly isolate the vocals from the background allowing the mix to finally be made.

As Jackson explained to Rob LeDonne at Esquire, the tech originated from forensic investigation work developed by the New Zealand police.

“When it’s noisy and they want to hear a conversation between two crooks or something, they can isolate their voices. I thought that’d be incredible to use, but it’s not software that’s available to the public.

“So, we contacted the cops and we asked, “Do you mind if we brought some tape to the police station and we ran it through your confidential audio software?” So, we did a 10- or 15- minute test and the results were really bad. I mean, they did the best they could. But you realize that for law enforcement, the quality and fidelity of the audio doesn’t need to be good, so it was far, far short of what we could use.

Weta took the theory of it and made a tool capable of producing high quality audio. “To hear some of those early songs in a fully dynamic way… You realize what you’ve been hearing is quite a limited range of audio,” Jackson said. “You don’t realize how crude the mixing on some of the early songs were, how muddy they were.”

READ MORE: Peter Jackson Takes Us Inside the Music Video for Beatles’ Final Song (Esquire)

In the making-of video below, McCartney expresses his doubts about making full songs out of Lennon’s demos, out of respect for the late songwriter’s unfinished work.

“Is it something we shouldn’t do?” McCartney says. “Every time I thought like that I thought, wait a minute, let’s say I had a chance to ask John, ‘Hey John, would you like us to finish this last song of yours?’ I’m telling you, I know the answer would have been, ‘Yeah!’”

Martin says human creativity is still at the heart of the song. Even if AI was involved, it wasn’t used to create synthetic Lennon vocals. “It’s key for us to make sure that John’s performance is John’s performance, not a machine learning version of it,” he told David Salazar at Fast Company. “We did manage to improve on the frequency response of the cassette recording…but it’s critical that we are true to the spirit of the recording, otherwise it wouldn’t be John.”

READ MORE: How Beatles producer Giles Martin used AI to reunite the band for ‘Now and Then’ (Fast Company)

Jackson also directed the video for the track which contains a few precious seconds of the first-ever film ever shot of The Beatles on stage in Hamburg in the early 1960s.

“A Beatles music video must have great Beatles footage at its core,” he said as part of a lengthy statement about the project on The Beatles website. “There’s no way actors or CGI Beatles should be used. Every shot of The Beatles needed to be genuine. The 8mm film is owned by Pete Best, the band’s original drummer. Clare Olssen, who produced the video, contacted Best to get a few seconds of his film.”

READ MORE: Peter Jackson Talks About Making The Beatles’ Last Music Video (The Beatles)

Angela Watercutter at Wired suggests that “‘Now and Then’ signals, if anything, not just the last Beatles song but the first in what could be a long stream of work that’s salvaged or saved using artificial intelligence.”

READ MORE: ‘Now and Then,’ the Beatles’ Last Song, Is Here, Thanks to Peter Jackson’s AI (Wired)

Indeed, Jackson has hinted at the possibility, as The Guardian’s Ben Beaumont-Thomas reports, of more Beatles music to come culled from archival footage he went through when editing Get Back, the eight-hour docuseries about The Beatles.

“We can take a performance from Get Back, separate John and George, and then have Paul and Ringo add a chorus or harmonies. You might end up with a decent song but I haven’t had conversations with Paul about that,” he said.

“It’s fanboy stuff but certainly conceivable.”

READ MORE: ‘Earliest known film of the Beatles’ to feature in Peter Jackson-directed music video (The Guardian)