Your Eyes vs. Frame Rates: What You Can (and Can’t) See

READ MORE: What is the Frame Rate of the Human Eye? (Filmmaker IQ)

READ MORE: What Frame Rate is Needed to Simulate Reality? (Filmmaker IQ)

In recent years, frame rates for movies and television have become a hotly contested issue, with strong arguments from many quarters challenging the nearly century-old standard of 24fps for movies or up to 60fps for TV, suggesting that these represent anachronistic restrictions on the much faster frame rates (or refresh rates) possible in top-tier videogames running off powerful hardware.

Some propose that higher frame rates would look aesthetically more pleasing, others say that the look of material presented at higher frame rates looks more like “real life,” absent the artifacts of traditional refresh rates such as the motion blur and juddering effects that we’re used to seeing on screens but don’t perceive in life.

 While there is certainly a large contingent that feels that the traditional speed, particularly 24fps, offer the aesthetic look of what the world has come to think of as “cinematic,” and that faster frame rates suggest sports coverage, a “soap opera effect” or a videogame look, there is also a fierce group which argues that those things are only negatives to people with a sentimental attachment to the images that derived from technical limitations of the past.

Scientist and filmmaker John Hess of Filmmaker IQ produced a video in which he discussed all the arguments on all sides of this question and ultimately came out in support of the 24fps standard for narrative filmmaking primarily, because it is the look that has become part of the culture of cinematic storytelling over the years and partly because he finds many of the supposedly scientific arguments for much faster refresh rates flawed.

Assuming that YouTube’s algorithm brought that video to the attention of the large community with strong opinions on the topic, he notes that this video received a number of views (more than half a million) far beyond what he’s used to and posted far more comments (9,000 and counting) than his videos normally attract.

Since then, he made two videos where he seriously analyzes the scientific claims he’s seen in those comments and elsewhere, delving into some serious concepts related to human vision from a biological and psychoacoustic perspective.

With “What is the Frame Rate of the Human Eye?” and “What Frame Rate is Needed to Simulate Reality?” he unpacks what it really means to suggest that we see in frames and the way a camera works. He references scientific work suggesting that dependent on an array of factors, an argument could be made that we “see” at roughly 10fps while another equally persuasive set of facts suggest that for a motion picture to truly present movement completely devoid of the kinds of artifacts we perceive in a normal cinematic presentation, it would need roughly 20,000fps to get there.

“There are a lot of kids saying how they can spot some super high ridiculous framerate,” he notes. “But the truth is the human eye doesn’t work that way. Humans are analog creatures. We don’t sample or quantize or pixelate our vision.”

The idea that someone claims to see at 144 frames-per-second or even higher, he says, is about “the fetishization of technology. The eye and the connected human visual system have no real frame rate, at least not the way we understand in terms of video and motion picture film.”

He sets out to debunk what he says are some unfortunately popular myths: “Myth number one: Fighter pilots are able to identify enemy aircraft when exposed to only an image flashed at 1/220th of a second so therefore the eye can see at least 220fps.” Hess says this idea has been floating around internet forums on the subject for so long that he’s unable to trace if it comes from any real study and, if it did, exactly what the number is.

He readily acknowledges that fighter pilots, by selection and training, have exquisite vision and well-honed reflexes, but, he says, “Just because you can see a flash at a short duration, that doesn’t mean your eye has the ‘frame rate’ equal to the speed of the flash. Using a camera recording 24 frames-per-second, we can detect and record and 800th of a second flash. In fact, I can detect a flash using a camera running any frame rate per second so long as the shutter is open during the flash… So long as the photons are registered by the camera sensor when the shutter is open, the sampling rate or the frame rate doesn’t matter.

“Being able to see a flash does not anyway determine the frame rate of the recording medium or the frame rate of the human eye.”

“Myth #2: I can tell the difference between a 60 hertz display and 144 hertz display, therefore the eye is at least capable of seeing 144fps. But that says more about the way motion pictures are created on a screen than it does about human biology.”

He points out that the way people generally first notice the refresh rate of a monitor is by wiggling their mouse around. “But here’s the key point: when you move your cursor around the screen, nothing actually moves.” The motion of a screen is instead a rapid succession of still images, each a little different than the previous one. This is called apparent motion by psychologists.

When you wiggle a cursor around the screen very quickly, the images of the mouse create a phantom array of cursors, with 60Hz displaying 60 instances of the cursor generated every second, and 144Hz displaying 144 instances of the cursor created every second. You can absolutely tell the difference between 60 and 144 cursors-per-second. But before you congratulate yourself for such an observational feat, he says, “so can a camera shooting 24 frames-per-second.”

In short, he sums up: “The human eye does not have a framerate.” But there are some important numbers he wants to talk about.

The first one is 10Hz, or 10 frames-per-second. Even before movies became a thing, he explains, psychologists found that 10 to 12 frames is where the phenomenon of apparent motion begins. “After that, the succession of images becomes apparent motion, whether that’s 15 frames, 24 frames or 2,400 frames-per-second.”

After expanding on the psychologists’ work that arrived at that result and experimentation that showed that other effects of perception indicate that we tend to see multiple images as one continuous image at about 10 examples a second but not less, he admits that 10Hz is clearly not a desirable frame rate (given flicker and other artifacts) and he re-iterates that “human vision has no framerate” and notes that the effect of apparent motion, even kicking in at that rate, isn’t necessarily sufficient for making motion pictures.

“Unlike a camera,” he points out, “the cells in your eyes aren’t attached to a master clock that syncs and samples all the retinal cells all at one time.”

The rods and cones in our eyes, which simplified detect movement and color, respectively, do not all perceive visual information the same way or at the same speed. The fovea right at the center of a human’s field of vision absorbs far more detail, but more slowly, than the surrounding cells, which work more quickly but at far less resolution. To complicate matters more, the cones receive information more rapidly, but require more light than the rods.

Hess delves into some visual illusions such as strobing light sources that seem continuous to the eye and the famous “wagon wheel effect,” which can create the illusion we’ve all seen in movies of a spinning wheel that seems to be turning opposite from the direction it clearly must be moving.

He also he cites a number of experiments undertaken to attempt to explain the illusion that many have while on psychedelics that lights seem to be streaking, and how these and other effects relate to the unique way our visual systems actually work.

In the second video, he expands these ideas further. Here, instead of talking about apparent motion kicking in at 10Hz, he explores another point. Asking people to imagine a small object moving across a large screen, he explains, “In order to appear perfectly smooth and eliminate all stutter, each frame must move the image by no more than the smallest Angular change humans can perceive.

“Say you’re sitting in a movie theater near the front of the middle of the hall. So, the screen occupies 55 degrees of your field of view. If you had a small dot that travelled from one side of the screen to the other, you would need 3300 frames-per-second to make that dot appear to move perfectly smooth across the screen in one second, as 55 degrees divided by 1/60th of a degree. If you want to move it in half a second [perfectly smoothly], you’d have to double that to 6,600 frames-per-second.” There is no theoretical upper limit, he says, but he suggests that 20,000 frames-per-second would cover almost all possible scenarios.

“The visual acuity is only that fine,” he reminds viewers, “in the center portion of our vision, the fovea. But then again, we never know what part of the path of the animation will cross our fovea… Here’s the thing, if we did construct a screen and fed it 20,000 frames-per-second, what would that flying object actually look like? Assuming we’re not looking directly at it? Well, it would look like a blur. So why not lower the frame rate and add motion blur? Now in real life filmmaking, we don’t need to add motion blur, it’s part of the process of shooting an image.”

And this gets even more complicated because of another important aspect of human vision: We don’t actually pan our eyes linearly. For questions about movement, motion blur and similar phenomena, he refers viewers to the site Blur Busters.

“Every bad idea, every bad argument in this field comes from reductionism — the oversimplification of topics in order to make them consumable by the populace.

“Our eye sees all frame rates, and they all have their look and feel,” he concludes, “so we should approach frame rate for what it offers aesthetically. As such, cinema remains at 24 because [that frame rate] is aesthetically pleasing and culturally significant… video games can take whatever frame rate your graphics card can muster, because that too is aesthetically pleasing: different mediums, different forms of expression, different frame rates.”