Light Field Technology!
 

Have a look at light field technology it's going to change the way we take photos! Pretty amazing stuff...
Lytro's light field camera captures 'unprecedented' images, lets you choose focus later -- Engadget

Re: Light Field Technology!
 

Wow, finally coming to the consumer market. I've been wondering when this would hit since I first read about it in 2005.

Light Field Photography with a Hand-Held Plenoptic Camera

Can't wait to see the model that ships & even more can't wait until they get it into video cameras. I think this could even be great for green screening w/o a green screen. Just tell the software that everything more than "X" number of feet away, delete.

This has the potential of making the "DSLR revolution" a forgotten memory. Who wants shallow DOF recorded, when you can have any DOF in post?

Re: Light Field Technology!
 

I wonder if 2 different depth ranges could be in focus, while the areas in front, behind and between are out of focus.

Re: Light Field Technology!
 

Dang, that really IS a game changer...

Re: Light Field Technology!
 

As for the game changer, I think all the cameras they have for this are pretty low resolution, like 1-2mega pixels (I could be wrong). Not sure if it has to do with the lens array, or the amount of information that has to get passed through.

If it is the lens array, then making this for video seems not too far away. If it is the amount of information that keeps the resolution small, then video would be farther in the future, as they'd have to get the frame rate up.

Still the amount of R&D in cameras today & how quickly they're improving & getting cheaper, nothing is too far away to wait for.

I think this would be possible, as it would probably just making a good algorithm in the software to handle it.

I think that it would be great to adjust an over the should shot 2-shot, so the background stays locked at a constant out of focus; subject stays locked in focus; but the interviewer's focus can be adjusted.


I can't wait to get my hands on one of these cameras & start playing around with it. Game changer I don't think is the right phrase, it seems bigger than that if it can do half the things I'm thinking about.

Re: Light Field Technology!
 

Sorry to be a wet blanket, but you rarely get something for nothing - and that is the case here.

The principle has been around for a while - Plenoptic camera - Wikipedia, the free encyclopedia - and whilst perfectly sound, the flexibility (such as being able to refocus) comes at a price. In this case, it's fundamentally that for a given image resolution, the data etc corresponds to a far higher resolution image. From the wikipedia link:
Hence, for the foreseeable future, even with a high resolution base (and big file sizes), the output images are likely to be quite low-res by todays standards, and don't expect video at decent resolutions with current technology.

Re: Light Field Technology!
 

Read the info on their web site - it appears that some fancy mathematics have been applied to reduce the hardware significantly, and their samples don't look too bad... They specifically refer to reducing a roomful of cameras to a pocketable device, and they apparently have photographers "in the field" working with these right now...

Too early to know much, but it certainly could represent a breakthrough technology presuming resolution is within expectations. They do claim to be shipping in the relatively near future.

As for the amount of data, there have been several algorithms I've seen suggested that might drastically reduce the amount of data required to achive acceptable resolution - offhand, I can see several of the "academic" anouncements of experimental techniques I've heard of coming together to make this "work".

It will be interesting to see whether video is on their radar - at the moment it appears stills are the "focus", but it would be quite interesting to apply the principles to video... if they can apply the concepts across time instead of in a limited timeslice, it should be feasible, if not now, in the future with faster more capable DSPs...

Re: Light Field Technology!
 

As a lot of replies on their blog say, the site is very lacking on specs - and noticeably to do with resolution, or even whether zoom lenses would be possible. I'm not doubting that it will work - but strongly suspect that we may be looking at something geared far more towards the consumer point-and-shoot market than serious photography.

Which is exactly where the technology is likely to be most relevant. Take a 16 megapixel pixel image, and derive something like a 1 megapixel refocussable image (with very large file size), maybe?
Which again comes at a cost. The "roomful of cameras" refers to being able to produce a result whereby you can gain an effect of walking round (and over, and below) a scene, as well gaining a conventional 3D image, or a 2D iamge with variable depth of field - depending how the data is processed. Such techniques are not new - for example, a museum may use them to record a statue, say, in unprecedented detail.

This company is likely talking about a product with very limited view angle etc by comparison - but which may be just what some customers want..

The clue may be in the FAQs of the company:
Wikipedia refers to the Stanford research -
Now I don't doubt that this has progressed in the six years since then, but there are good technical reasons why the final product is far more likely to be something intended for relatively low-res point and shoot - not what most on this forum would call serious photography. But I wait to see more detailed information on their website.

In the meantime, you may be interested in the original Stanford University research paper from 2005 - http://graphics.stanford.edu/papers/...era-150dpi.pdf - one of the authors is the founder and CEO now listed for Lytro.

Re: Light Field Technology!
 

Taking this to the extreme...

Imagine a big, empty cube that you would bring into a room. It would have Light Field technology, HDR, RAW recording, and virtual camera positioning. Turn it on, hit REC, and yell, "action." The focus, exposure, color balance, and camera position/movement could all be chosen in post.

Some would say that this takes away from the art of filmmaking. I would disagree. It would decrease the skills needed to shoot, but could allow more artistic end results. It would also save a ton of time. There would be no more botched focus, exposure, colors or unintentionally jerky motions. We wouldn't have to move and level tracks, change lenses, and on and on. It would ONLY be about the performance of the actors.

This would be killer for composites as well. Camera motion tracking between layers would be perfect.

But for now, I'll focus, adjust the camera settings, and deal with dollies and jibs and imperfect movements. And I'll keep saying "cut! We need to do that again!"

Re: Light Field Technology!
 

David Heath, remember the Nikon D1? A "professional" camera with 2.7 megapixels, hardly usable resolution for any "professional" still photo.

But how long did it take after that to get DSLR cameras with respectable resolutions?

If I can get a 1megapixel light field camera this year for under $300, I'd go crazy & start playing around with it as much as I can. It would be a novelty at that resolution & not great for paying jobs, but still fun to play with.

And if it sells well enough, you can bet that all the major camera manufactures will be releasing better cameras as soon as they get the chance to take the money from buyers. (I'd be shocked if they weren't already doing R&D).

Re: Light Field Technology!
 

Yes, that's a good point. The point about the D1 was that it was targeted primarily at photojournalists, and for news the resolution was "good enough", but the time saving by no processing, and being instantly able to send the results electronically was key. But what the Stanford team produced was 0.09 megapixel output resolution (292x292), and that from a starting point of 16 megapixel. Tricks can obviously be done by trading off refocussing coarseness for resolution, but I think even 1 megapixel output resolution may be highly optimistic. We'll see, but something like 640x480 (0.3 megapixel) max might be more likely - and more than good enough for "point and shoot" and display on a smartphone screen.

The 16/0.09 is a ratio of nearly 200:1. That means to get an output resolution of even 2.7 megapixel would mean a sensor of about 500 megapixel - and video would mean reading and processing it 60 times each second. Never say never, but I don't see it happening soon.
[/QUOTE]If I can get a 1megapixel light field camera this year for under $300, I'd go crazy & start playing around with it as much as I can. It would be a novelty at that resolution & not great for paying jobs, but still fun to play with.[/QUOTE]
I don't disagree with any of that, just don't expect too much, too soon.

Re: Light Field Technology!
 

David -

Think on this for a moment. If I have grasped the fundamentals properly... the design uses MULTIPLE lenses with multiple "views" of an object from ever so slightly different angles... meaning there is an ABUNDANCE of data sets, which could easily be interpolated into a MUCH higher resolution image with some fancy math.

I recall an article (think it was linked here?) where they were using some "fancy math" to take relatively small file sizes and interpolate the details (something which still baffles me, as I don't see how you could get something from "nothing"!? Here there IS something though!) in order to recreate very high resolution, high detail images.

So let's take those two "technologies", mush them together, take a large number of ever so slightly different data sets, combine with algorithms that would analyze all those data sets at a high rate of speed, consider all the similarities and differences and "compute" a resulting image (looks sort of like what they are hinting at). You're not limited by "film", or sensor resolution, your only limits are data throughput and the efficiency of your algorithms in handling the data, and perhaps the quality of your lens(es).

In effect you get infinite focus (check), nearly infinite exposure (they claim very good low light), and nearly infinite theoretical resolution... gives me a headache thinking about it, and the computing horsepower in that camera would be pretty impressive, BUT methinks that's how they are doing it.

From a video standpoint, it may become rather problematic simply because scaling the calculations and horsepower required to take a single image enough to handle 30-60 images per second... well, it is mind boggling... but give it time, it's easy enough to do if the foundation is there, just have to wait for computing horsepower to catch up!

The more I contemplate this technology, the more intrigued I get - there is not a huge leap from this to "holocameras" capable of complete 3D capture, and all sorts of other crazy stuff!! Even crazier, what if you could "reverse" the algorithms to make a holo projector!? Eh, I'll take the camera since it looks like it's closed to "baked" already!


FWIW, the above is my "guess" based on pretty limited review of their website and that old technical paper, I could be completely off base, so don't take it as gospel or anything beyond a WAG (Wild A** Guess) at what is coming... but it's fun to contemplate!

Re: Light Field Technology!
 

Except that it is a little hard to define what would be meant as video processing, since much of the image processing with LFT is done in post. You could imagine shooting with just a huge amount of high speed storage, and then doing all the video processing in post.

Re: Light Field Technology!
 

All of these plenoptic cameras can be seen as a hack to do bokeh in software rather than
just using a wider aperture. The bonus of simulating a wide aperture in software is that you
can pick the focus plane after the fact.

It helps to understand where bokeh comes from but I think you can do a thought experiment
and get an idea for how this all works. I used the following explanation on
another website but maybe it will work better here:

Imagine you have a stereo rig with reasonably normal lenses that are almost zero
radial distortion and have the left eye and right eye files on two separate tracks in
your favorite video editor. Set the track on top to be 50% opaque so that your
output video is just both frames arithmetically averaged together.

In this new output video, most things will have two ghosts. Say you shift the
top frame left a few pixels until some of the objects loose their ghosts and
line up. You will probably find that the objects that it are possible to line up
(remove the doubled ghosts) are the same distance from the camera,
Congratulations, you have just focused your plenoptic camera at a particular depth.

You will notice though that everything in front of and behind the plane that is
in focus has two ghosts. This "blur" is basically the bokeh of your system. It
arises from the fact that the left edge of your new aperture views reality from
a different angle than the right edge of your aperture (the right eye camera).

Now this bokeh sucks. It's not a nice round diffuse ball, it is two points that tend
to make everything just look ghosted rather than actually blurred.

If you want a nice bokeh, use 20 cameras arranged on some sort of regular
two dimensional grid. It's more difficult to figure out how to translate each
view left and right so that the depth plane that you want is in focus, but it is
doable. Hard to think about but hopefully this makes sense.

If you want a whole lot of bokeh, add cameras that are farther from
the middle of the grid. If the bokeh is ugly (uneven/point like) add more cameras
between the cameras already on the grid.

You could also just make one big lens that was the size of the
frame holding the grid of cameras and get the same bokeh, but this could
be heavy and unwieldly. But the reason that the big aperture has big bokeh is
because the edges of the aperture are far apart they see different views of the
scene that don't line up right. Hence when they get averaged together, they
come out blurry and turned into bokeh.

One note:

No. These cameras have sacrificed spatial resolution in exchange for recording
angular information about how a light ray came into the camera. You can not generate
a high resolution 2 dimensional image. What you can do is generate a low resolution
3 dimensional image (or a surface mesh) more easily that you can with a stereo camera.

Re: Light Field Technology!
 

No, it doesn't work like that. It uses multiple lenses in front of the image plane to take multiple views, indeed, but they are all low resolution. The maths won't work to then reconstruct a high res image, it will work to reconstruct the effect of varying focus.
No - you get the ability to alter focus in post, but only over a finite range - here fixed by the number of sub images. Infinite exposure is not true - what you will be able to do is correlate the images to improve s/n, but only to a limit. I suspect it will be of the order (in the Stanford case) of what would be achievable if they had simply had a 270x270 sensor of the dimensions of the 16 megapixel one they used. (Hence each photosite being large.) And the resolution will be far from infinite, it will be equivalent in this design to the number of microlenses, and the number of pixels under each microlens will determine the extent of refocussing that can be done.

All this is illustrated very clearly in Figure 13 of the Stanford report, and the text beneath it.

If you doubt me, try e-mailing the company and asking directly what the max output resolution of the system for sale will be?
Even reading all the data off a 500 megapixel sensor at 60x times a second is a mammoth task, and, I suspect, way, way beyond current practical technology, and that's before we even start to think about processing. Applying this to high definition video may be theoretically possible - but practically is likely to be way, way off.

[EDIT - Just seen the last paragraph of Dustins reply above - exactly so, well put.]

Re: Light Field Technology!
 

It's more than just wide-aperture bokeh. It's the ability to pull focus perfectly every time in post.

Try a shot with a close focus lens and a dolly pulling back from an extreme closeup. Just try to maintain critical focus. After 20 to 30 takes, you'll re-design the shot. But with a plenoptic solution, you can nail the shot in one take. That's huge.

Of course, the technology isn't there yet, and may not be for decades, but never buzzing focus again has a lot of practical value.

Re: Light Field Technology!
 

It's now over six months since this was previously discussed, and I thought at the time the product was due to ship soon? It's still given as "Pre-order here" on the website - has anybody heard anything more? Or seen a prototype in the flesh?

Re: Light Field Technology!
 

Re: Light Field Technology!
 

Yes, but the last post in that thread (by me) is even longer ago (about 9 months). The product was supposed to hit the shops well before Christmas - but where is it? The manufacturers site still seems to have it as "pre-order" only. Anybody heard anything recent?

Re: Light Field Technology!
 

I think they shipped a batch out to "early adopters" on their waiting list - they only had MAC software, so I'd suspect they are waiting to have Windows software available for wider release. They post regularly on Facebook, and have been doing lots of "PR" and public showings, so I'd guess it's "coming".

I was surprised at how quickly they announced they would ship, maybe they had supply chain issues like many other companies last year? Still be faster than many products from announcement to release...

Lytro camera - cool stuff
 

This may have been posted in the past, but here's a really cool website to this new still camera. It shoots everythng in focus, then you can change the focal points afterward. The link will get you to some samples. It's pretty cool to actually change the focus in the pictures real time.
I know it's not a video camera, but maybe this technology can transfer to video soon.

https://www.lytro.com/living-picture...695?&_suid=829

Re: Lytro camera - cool stuff
 

Here: http://www.dvinfo.net/forum/digital-...echnology.html

And here: http://www.dvinfo.net/forum/digital-...ld-camera.html


When I saw the new thread, I was really hoping that someone at DVInfo had purchased one of these & started taking photos. I was so close on purchasing one, just didn't have the extra cash to justify it. If anyone does have one, please share.

Re: Lytro camera - cool stuff
 

And - as I've said in each of the previous threads - whilst what is claimed for it is perfectly true, it's what doesn't get said that is most relevant.

And that is the resolution of the final images that can be output from it. You trade off resolution for refocussing ability. It should be fine for images on the web, but not serious photography, and lytro have been very reluctant to discuss resolution.

The technology is already having industrial applications via other companies where depth information is more important than absolute resolution, such as in things like face recognition. (eg See Applications - Raytrix GmbH ) Don't expect the technology to transfer to video anytime soon. At normal frame rates, you'd be talking about a massive data rate, only to end up with a standard definition image - albeit one that can be refocussed.

Re: Lytro camera - cool stuff
 

Lytro has certainly had some great marketing these last few months.

Maybe if they tie up with a certain company whose name starts with an 'A', so they could beat a certain rival who has just released a '41'MP camera phone...who knows?

Re: Lytro camera - cool stuff
 

They need to make a 1080p capable lytro and a plugin for after effects for focus control keyframing, then I might bite. Resolution isn't much for photo, but for incorporating lytro stills into video/motion graphics it could be a creatively useful tool.

Re: Lytro camera - cool stuff
 

Be careful what you wish for, Nevin! Wouldn't want to be the guy sitting on a station tracking focus on a moving shot in Transformers XII.

Re: Lytro camera - cool stuff
 

Digital photography magazine has an excellent in-depth review of this camera on their website, which is something like dpreview or dpreviews dot com. This camera is really only a toy at this point, but may be the start of something useful. The camera is enclosed in a 1.5 by 1.5 inch rectangular shell which is 4.4 inches long, and captures 1080 by 1080 pixel equivalent (approx 1.2 Mpixel) photos. The software, at last reading, worked only on Mac computers, and it would appear that the company hopes to sell out to Apple in the near future. The data files for the 1.2 Mpixel photo are large, and upscaling to a useful photo size would require very large data files. Interesting, but not yet ready for prime time.

Re: Lytro camera - cool stuff
 

Thanks Mark. Well worth reading, and bears out much of what I've said - the refocussing ability comes primarily at the expense of resolution.

The article raises a point that I hadn't really thought about - what they call "refocussability" or the range over which the image can be refocussed. They refer to it as equivalent to depth of field in conventional photography, but instead of the depth within with everything is IN focus, it's the depth within with everything CAN be refocussed.

Point is it's not infinite, and (as with dof) the more you zoom in, the less it is. Hence, much of the time the camera still does need to focus to an extent more or less conventionally at the time of shooting.

The other point is that the refocussing "smoothness" is governed by the no of microlenses. More microlenses, smoother the refocussing abilities - but the lower the output resolution. So the "11 megarays" COULD be used to give a final 5.5 megapixel image - but then likely only be able to be refocussed over two planes! Point is that "refocussability" could give you the ability to focus just in front of or just behind the plane you really want - but possibly not actually on it! Of course, more microlenses equals more fineness, but then expect the resolution to drop away.

If this camera is "11megarays" and they quote 1.2 megapixels as final output resolution, it indicates strongly that the microlenses are in groups of 3x3. That similarly indicates that you can expect to move focus between 9 separate planes of focus in the post stage.

Re: Lytro camera - cool stuff
 

so it is already HD at 1080x1080, it could still have it's use in video production if we can control the focus over time on a timeline...

Re: Lytro camera - cool stuff
 

In a video sized frame (16:9) you're talking about the output equivalent to 1280x720 - and the implcations are that that the actual resolution is likely to be less. So 720 HD maybe, but not 1080.

But that's for stills. (And for a relatively coarse refocussing ability.) I don't think there's any suggestion yet that video is feasible. It's one thing to take the 16 "megarays" off the chip and record it as a still photograph - totally another to do that 24 times a second (let alone 50/60).

And even if it was possible, the ability is likely to come at the expense of other factors. The basic photosite size is small, and that's likely to affect dynamic range, for example.

Sorry, It's just not realistic, never say never, but I don't expect to see it in my lifetime, if ever.

Re: Lytro camera - cool stuff
 

I'm not talking about recording video. I mean animating still pictures in after effects for effect in a video production. For title/full-screen supers, diagram purposes, etc.

Say for a concert promo video/commercial. I can see using concert pictures taken with lytro, animating these still photos by rack focusing through thousands of people stopped in 1 moment in time.

Re: Lytro camera - cool stuff
 

You could do that now by taking an image with a very deep DoF and then selectively de-focussing the image to simulate out of focus or shallow DoF.

Lytro is still an interesting technology, but the flaw is that the camera is still using a lens. If there is a lens, then it's bringing light in to focus at certain points or over a certain range. I suspect the technology that will truly give a focus anywhere after capture ability won't use a lens as we know it.

Re: Lytro camera - cool stuff
 

Sorry for misunderstanding - I can see what you mean, I suppose a variation on the theme of tracking round a "frozen in time object".

It's an interesting thought, but I'm still not sure the lytro technology is even up to that, not for quality work anyway. You'd have to check it in practice, but I suspect it won't be quite high enough resolution, and give a steppiness to the refocussing. So with (say) 100 rows of seats the effect would be more like being able to have focus on every fifth row and mixing between these planes of focus - not a smooth rack. (Imagine having 16 cameras all focussed at different planes - that's equivalent to the raw material you have to work with.) I would be interested to see it in practice.

Re: Lytro camera - cool stuff
 

I'd think they could incorporate this ability into dSLRs in a manner akin to exposure bracketing. Would be nice to tweak focus after the fact when doing a timelapse. Canon???

Mark

Lytro images to get your teeth into.....
 

Lytro Camera Produces Amazing, Interactive Pictures (PHOTOS)

Interesting.


CS

Re: Lytro images to get your teeth into.....
 

Sort of, I guess.

I had one on pre-order, got it this spring. Tried it out for a day or two and sent it back.

It's an interesting concept and as the capabilities progress (3D) along with the resolution, it will likely be more formidable. For now, I found the image quality lacking (reminded me of the stills mode found in camcorders five or six years ago--not so good) and the ergonomics funky (novel and attractive package, but the touch sensitive zoom control was erratic).

I'll keep an eye on them but I can't recommend the first gen product; it's kind of a one trick pony. You go walking around searching for things to shoot on extreme planes to take advantage of the depth of field thing, which isn't really photography, more like a novelty.

My opinion of course.

Re: Lytro images to get your teeth into.....
 

I've commented on this a number of times in the past, and whilst "interesting" is indeed a good word to describe it, don't expect major advances anytime soon.

As Charles has found out, the main issue which the makers omit from their hype is resolution. The other point is that the focus point is not infinitely variable - rather that it allows you to step backwards and forwards in finite steps. Fine - unless the point you want to hit focus on lies between two steps!! :-)

And the key point is that the resolution and the number of planes of focus multiply together to give the number of photosites on the base chip. If this is a fixed number, better refocussing ability can only come at the expense of resolution, and vice versa.

I'd expect the number of planes to be *at least* 16 - and if we even only go for 8 megapixel final resolution that means a photosite count of well over 120 megapixels! You may argue that it will come in time, as with other technology improvements, but there are fundamental limits, not least those due to the wavelength of light. That may be overcome by moving to much larger chip sizes - but don't expect it to be cheap!

The light-field concept is already being exploited by other companies for industrial etc applications where resolution is not the issue. (Where about 0.5-1.0 megapixel are adequate for the task.) Lytro are trying to expand it to the consumer world and "normal" photography - frankly, what they are marketing just isn't up to it, and more and more people are now finding that out. Improving the resolution to meet user expctation will be a far, far more difficult task than it ever was with normal digital cameras.

One day there might be a Lytro video camera
 

Who would need more resolution? Take a look at this article on what they can now do with still images. Lytro isn't just a refocusable stills image gimmick any more.

Lytro changed photography. Now can it get anyone to care? | The Verge

From the article:

Andrew

Re: One day there might be a Lytro video camera
 

Well, I think resolution goes hand-in-hand with developments such as this.

Imagine in the future, you strategically set your cameras (possibly a selection of 4) on your set, all shooting very wide angles. They utilize this technology that Lytro is developing. On top of that, they are shooting 16K with an eye towards 4K distribution.

You would essentially just set the scene in motion and capture these 4 angles that cover pretty much everything given how wide they have been set to record. Then you take it to post, crop out to 4K the composition that you want, set your focus marks, and then place your lights (or really those three can be in any order you choose). Decide you need a dolly shot? You can add it. Want to change the way that shot rack-focused a mere 24 hours before release? Done.

This is where I see it ultimately going.

Re: One day there might be a Lytro video camera
 

Video autofocus technologies have been available for decades, but professional sets still employ manual lenses and focus pullers. We already don't even need cameras, sets, or face actors, we can just do entire films as CGI in post. They don't happen very quickly, though.

While light field cameras will surely someday push into video, there's still a substantial sector of video creation that needs the right image direct out of camera and straight to the broadcast switchers or same-day edit desk, places where nobody's going to want to futz with the focal point.