Everything you ever wanted to know about HFR and The Hobbit

This article was originally published on Tim Dashwood’s blog
and is reprinted here with his kind permission.

The Hobbit has finally been released in 2D and 3D at the traditional 24 frames per second (fps) as well as 3D High Frame Rate (HFR) at 48 fps. If the posts on social media are a fair indication, it seems that audiences have not reacted as positively to the 3D HFR version as the industry had hoped. Why not? Industry leaders like James Cameron, Douglas Trumbull and Peter Jackson have been been telling us why HFR is a better choice for 3D (and 2D) by overcoming one particular technical issue that may cause viewer discomfort, and they are absolutely correct in that respect. However, HFR will not solve all the potential eye-strain problems associated with a 3D presentation, and the film-going public has become accustomed to the aesthetics of 24 fps as a key contributor to the cinematic look of their favourite films. To most, the hyper-realism of 48 fps seems like a strange intruder in the context of an epic fantasy film like The Hobbit. Battle-lines are being drawn between the technical improvement of HFR and the aesthetic “feel” of the traditional 24 fps film standard.

Why haven’t we always used a higher frame rate than 24 fps for cinema?

Historically, 24 fps came about at the dawn of “talkies,” when previously the usual frame rate of silent films was only 14 to 18 fps. 24fps was the minimum speed required to successfully play back sound tracks in sync. When you consider that a 1000 foot roll of 35mm film only represents about 10 minutes of screen time, is expensive to purchase and process, it makes logical sense that the studios would want to use as little as possible and therefore made the choice to standardize the slowest frame rate necessary for sound. Scientifically speaking, humans’ threshold for fps approximating real life is about 50 or 60 fps, so the unintentioned side effect of 24 fps is the ‘other worldly’ sense that a film is a cinematic experience and not real life.

The slow frame rate had no ill-effects in the infancy of film because the camera was typically wide and stationary. However, when filmmakers like G.A. Smith, Giovanni Pastrone, Allan Dwan and D.W. Griffith started moving the camera, it became apparent that, because of the flicker and low frame rate, slow controlled “push in” or “pull out” moves were far more pleasing to the viewer than fast pans or side-to-side dolly moves.

Above: A shot from Giovanni Pastrone’s “Cabiria” (1914), largely considered to be the first film to use controlled dolly moves, and a major inspiration for similar moves in D.W. Griffith’s epic “Intolerance.”

The bump up from 18 fps to 24 fps didn’t help very much with this issue and soon pan speed charts were published by organizations like the American Society of Cinematographers.

These charts are still published in the ASC manuals and professional cinematographers still refer to them. If I reference a typical example in the ASC chart I see that a 90 degree sweep of a static scene at 24 fps with a 35mm lens should not be performed in less than 18 seconds (whip-pans being the exception). However, the same pan with the same setup at 48 fps can be performed in half the time at 9 seconds.

There are always exceptions to these ‘rules’ but the point is that it makes logical sense to shoot and project at a higher frame because it frees the filmmaker to shoot faster pans with less “judder” and faster action with less motion blur.

Douglas Trumbull created an HFR 60 fps system a long time ago called Showscan, but it never took off for mainstream production because of the inhibitive costs associated with all that film. Filmmakers have always had to find ways to fit their creative requirements within the technical limitations of the medium. HFR digital acquisition is supposed to remove one of those limitations, but it can only truly do that if HFR is also used in digital projection.

The technical limitations of the medium are even more restrictive when working in stereoscopic 3D (S3D), especially when it comes to frame rates. The S3D illusion requires parallax points for each object in the scene to be visible to each eye, which is almost impossible in scenes with fast movement (like sword fights) where key parallax point practically disappear due to motion blur. When panning in S3D the brain must work even harder to link the images from the two views and the judder effect is compounded further by current S3D projection technology.

If you have the chance to watch both 24 fps and 48 fps HFR versions of The Hobbit you will see feel the eye strain in the large and fast sweeping shots at 24 fps, but it will feel smooth and effortless at 48 fps.

Knowing how higher frame rates can improve this important aspect of S3D and free filmmakers constraints, it makes sense from a technical perspective that the S3D industry would push for higher frame rates, especially now that digital acquisition and digital projection remove many of the financial detractors. But will audiences accept the radically different look and feel for narrative films like The Hobbit, or is it better suited for nature documentary, concerts, theatre, theme parks and sports? Frame rate plays a big role in the psychological perception of what we are watching.

Above: 24p (left) vs simulated HFR (48fps synthesized by Luke Letellier)

The “Video Look” and why most audiences dislike it for drama.

Even though the 2012 hype around HFR makes it seem like something new, anyone who has even watched television is already very familiar with the temporal quality and ‘feel’ of video acquired and broadcast at 50 or 60 interlaced fields per second. European HDTV viewers and American HDTV viewers of Fox or ESPN are already familiar with the look of 60 frames per second HD broadcasts. We have become accustomed to soap operas, news, sports, game shows, reality TV and low-budget drama being shot and broadcast at 50 or 60 fps. Films, sitcoms, music videos and TV drama are typically acquired at 24 fps are still broadcast at their original frame rate (via 2:3 pulldown in 60i or 25p in 50i).

At the time when television standards were being established, broadcast frame rates were chosen for practical purposes. Television systems needed some sort of timing clock and the simplest engineering solution at the time was to use the common AC power frequency. Therefore 60Hz for North America and 50Hz for Europe.

Since the early TV camera tubes had to scan the frame electronically over time (instead of exposing all parts simultaneously as with film) it was impossible to pan or capture fast movement accurately without apparent tilting or the “jello” effect, an issue modern digital filmmakers are all-too-familiar with in CMOS rolling shutters. The solution to the problem was to scan the even and odd lines in the frame at separate times, which minimized the effect. Hence 60 “interlaced” fields per second = 30 frames per second. The temporal ‘feel’ was still equivalent to 60 frames per second so there has always been a distinctive difference between 24 fps film and interlaced video.

Our association with certain types of programming and the frame rate is something we cannot easily shake. Aspiring filmmakers want their films to look like professionally produced films. There are many things that constitute the look of a professional film like lighting, camera moves, lenses, acting, and set design, but the overwhelming factor is always the frame rate. In the 90’s popular “Filmlook” processes and best-selling software like Magic Bullet approximated the look of 24fps from interlaced video.

Low budget productions who couldn’t afford to shoot on film were desperate to give their interlaced video the film look. Therefore it was no surprise to me when Panasonic first introduced the DVX100 camera (the first DV camcorder capable of acquiring in 24 progressive frames per second) it quickly became one of the best selling camcorders of all time. The “digital revolution” then followed with pro-consumer HD camcorders like JVC’s HD100 and then Canon’s 5DMkII, as well as the professional digital offerings with Super-35 sized sensors. In fact, 24p acquisition is now so popular that it is almost impossible to purchase a professional or consumer camcorder without the feature. This suggests that we really do love the look 24fps, especially for our narrative storytelling.

However, it is interesting that as soon as 24p was available to the masses via digital video, lots of complaints about “judder” started to surface. Suddenly an issue professional cinematographers had always effectively dealt with was thrust upon the amateur videographers who likely did not have access to panning speed charts. Unfortunately the term “judder” became the primary talking point for an anti-24p movement and before long all the major TV manufacturers were adding “smooth-motion” circuits to HDTVs, which created new morphed in-between frames to overcome the supposed “problem” of 24 fps.

Will audiences eventually accept HFR as a cinematic medium for narrative storytelling?

Just before Siggraph 2012 I was invited by SIRT to help plan a test shoot to compare 24, 48 and 60 fps acquisition at different shutter angles for S3D. This test footage was shown by Christie at Siggraph alongside James Cameron’s tests. During the planning stage we watched James Cameron’s HFR comparison tests where he convincingly made the argument for HFR as a solution to all of S3D’s problems. I personally wasn’t impressed with the look. The HFR dinner scenes felt to me like BBC teleplays from the 80’s and the sword fight felt slow, theatrical and rehearsed. However, my most interesting observation was that even though most of us “old guys” in the room disliked the temporal feel of the HFR, the film students in the room (most born in the 1990’s) all seemed to prefer the HFR. This sort of makes sense considering that generation spent its formative years in the age of disposable media and YouTube, where the art of the photography or lighting doesn’t matter as much as the immediacy of the content. So maybe it’s simply a generational thing and after some time HFR will be easily accepted by the masses? As i mentioned before, I personally think that the hyper-realism of HFR is perfectly suited to nature documentary, theatre, concerts, theme parks and sporting events, but not necessarily to narrative storytelling.

With this in mind, I can’t figure out why the proponents of this technology have taken such a huge risk of public backlash by choosing to feature The Hobbit as the worldwide introduction to the format instead of the James Cameron produced S3D film “Cirque du Soleil – Worlds Away 3D” coming out next week.

It was apparently acquired in HFR and with the dynamic fast motion of the Cirque performers it would have been the perfect application and introduction to HFR technology, but I can’t find a single HFR screening of the Cirque du Soleil film at any of my local theatres. They unfortunately are all in 24 fps.  I remember Cameron demonstrating footage from the film in HFR within the last year, so I can’t figure out why they didn’t jump on the opportunity to release it now in HFR?

I certainly hope Cameron’s Mariana Trench film will be easily accessible in HFR 3D because it will be the closest any of us will ever get to experiencing “being there” from the comfort of the cinema. Another perfect application for HFR.

The best approach for the future.

Higher frame rates are freeing for the filmmaker only if the projection is guaranteed to also be in HFR. In the case of The Hobbit, the film is still being released in 24fps 2D and 3D formats so it was still necessary for the filmmakers to consider the lower frame rate, somewhat defeating the purpose of the HFR. Unfortunately there are many fast sweeping shots in The Hobbit that are simply too fast for 24 fps, but look great in the HFR version. 48fps was chosen instead of 60 because it is simply double the normal frame and makes pulling a 24 fps master very easy, but it will not be easy to adapt it to 50i/p or 60i/p broadcasts. Also, using no shutter (360 degrees) in digital 48 fps acquisition is equivalent to the exposure time of standard 180 degree shutter in 24 fps, so the acquisition frame rate of the 24 fps master will be indistinguishable and the exposure unaffected.

Since the Christie HFR-capable projectors I’ve seen can dynamically switch between 24, 48 and 60 fps, in an ideal world I think I would use 24p wherever possible for narrative production and a higher frame rate DCP only when necessary. This method would also be easy to accomplish in almost any modern non-linear editor on a 60p timeline.  In fact, Douglas Trumbull has already been working out how this method would work by acquiring at 120 fps for a 60 fps master that can contain 24 fps with pulldown.

Above: Showscan Digital from Douglas Trumbull

It seems to be a good idea on paper, but of course the exposure time per frame becomes much shorter at 120 fps, which affects the lighting plan and apparent motion blur. Motion blur can be synthesized in post, but directors of photography never want to give up exposure, especially when working with 3D rigs. Imagine how fast a slow motion shot would need to be overcranked at 60 fps!

What about the 3D quality of The Hobbit?

The stereoscopic 3D execution in The Hobbit is not perfect, with numerous stereo window violations and some depth map/inversion errors on dimensionalized VFX shots. These are issues that left me with strained eye muscles (specifically in my right eye), and cannot be remedied by viewing the HFR version. However, overall the film has many good S3D moments and the visual effects (in 24 fps) are spectacular, but they start to show their seams at 48 fps. The stellar score ties it in well with LOTR and it has everything a Tolkien fan would expect from a Peter Jackson film set in Middle Earth.

The proponents of HFR for The Hobbit will say things like it creates a “window in to the real world”, but I don’t want my favourite fantasy hobbits, dwarves, orcs, trolls, goblins, wizards and elves to be in my reality, I want a window into the fantastical Middle Earth, which to me is always in 24 fps.

I personally recommend to all my friends to see it in IMAX 3D first because you will get the added bonus of the exclusive preview of the first 9 minutes of the next Star Trek 3D movie. Save HFR 3D for your second viewing. I’m glad I did.

About the Author

Tim Dashwood is the founder of Dashwood Cinema Solutions, a stereoscopic research, development & consultancy division of his Toronto-based production company Stereo3D Unlimited. Dashwood is an accomplished director/cinematographer & stereographer and a member of the Canadian Society of Cinematographers. His diverse range of credits include music videos, commercials, feature films and 3D productions for Fashion Week, CMT, Discovery Channel and the National Film Board of Canada. He also consults on and previsualizes fight/stunt action scenes for productions such as Kick-Ass, Scott Pilgrim vs the World and Pacific Rim. Dashwood is the creator of the award winning Stereo3D Toolbox plugin suite and Stereo3D CAT calibration and analysis system.

This article was originally published on Tim Dashwood’s blog
and is reprinted here with his kind permission.