Well, in fact I've been in contact with the stuff but in some distant way, not exactly programming but more as some kind of advisor telling people where to look for code/ algorithms and which one to choose over another...
So I can't really say I "Know the stuff". I would really like doing it myself, but I don't have the required time for learning all the needed skills and become some kind of Superman, nobody can do everything by itself alone.
By now I'm just trying to experiment with some Microcontrollers from ATMEL for some other stuff.
Technology here in South America is not as easy to get as in USA, don't even think about getting a Verilog programmer who doesn't think he is Einstein!!!
My small test where done in simple C language on a Pentium 4 HT machine

I was thinking that if Elphel can compress Theora on the fly, this RAW Bayer compression should be a piece of cake.
Theora encoder took me several thousand hours to make - far from "a piece of cake". I need to find some time to port/improve it to the new 353 hardware - job not done yet.

And it is true - I don't see much sense of true lossless video compression, DCT-based JPEG or Theora with quantization coefficients of 1.0 could do the same virtually lossless (why do you need true lossless?) compression with the ratio of ~3x. I would rather try wavelets, but it would be a larger project for me.

I had been trying to get Andrey involved even earlier, but there was two things, the Elphel 333 and earlier models were not that good for the purpose, and Andrey had an lot on trying to earn an living out of security sales to spend time on this.
Yes we need to make living of the cameras - and currently it is rather far from the security business. Cinema projects are community driven and we would like them to stay that way (at least in the near future) - we can just support them with our hardware.

Technology here in South America is not as easy to get as in USA, don't even think about getting a Verilog programmer who doesn't think he is Einstein!!!

We are now working on Ubuntu-based distribution that includes FPGA development project. Unfortunately we can not include Xilinx software there - you have to download it separately from their site.

We now have a 10359 board ( http://wiki.elphel.com/index.php?title=10359 ) that is easier to experiment than with the camera FPGA itself.

Quite a good news!!!
Anyway I understand your position, I know the only way I have not being a company is making it work and showing if it is usefull or not.Only thing I want to clarify is that this encoder is not DCT, but integer transforms, and that it compresses the RAW bayer, not an RGB image, and that it is higher than 8 bit per pixel.(10 and up).
I thought that replacing some parts of Theora with these routines would be enough, but you prove me wrong.
see you. :)

(why do you need true lossless?) compression with the ratio of ~3x. I would rather try wavelets, but it would be a larger project for me.

Yes, I agree, visually lossless is good enough, but with Bayer, lossless would be preferable because of the knock on effects of color deviation during de-bayering. Otherwise, I think lossless is only of much use in movies for green screening, and computer graphics, even though I find it more pleasurable to look at than visually lossless.

Is anybody working on the choice of down converting 4:4:4 binned pixels to Bayer pixels before compression, to reduce compression and data rate?

Will the Dirac wavelet FPGA project be of any good to you, or is there any other wavelet FPGA codecs out there (2D-5D)?

There is no point in demosaicing before compression, since in this process you're going to "recreate" missing information. And you want to compress the bayer array because the errors introduced by the compression will be filtered by the demosaicing interpolation anyway...

/me who's working on implementing an high quality and almost not slow demosaicing algorithm.

I have the opportunity to work with the 353 already. I'll keep everyone posted on the results. I'll be ordering the power supplier, so it'll take some time to get started.

Besides that, I'm planning on rebuilding the 35mm wax adapter and perhaps building a Medium-Format adapter for the Elphel.
I'm thinking about testing a MF double lens camera, so I can focus through one lens and capture through the other one. This will solve the 'real-time' preview problem, but will rule out using different lenses. Still, I think it's worth looking in to.

the same idea of me!
I have now a set of mamiya lens 645 manual focus
they are incredibly sharp
I suggest you to take the 80mm f1.9 the fastest lens on the medium format market
and a 45mm f2.8 and /if you can take also the 35mm f3.5 you'll get an incredible variety of prime! :-)
take a look at this site do not go in e-bay! http://www.keh.com/onlinestore/home.aspx

for andrey: cool the ubuntu based distro ...did you get good time at linuxtag in berlin?

There is no point in demosaicing before compression, since in this process you're going to "recreate" missing information. And you want to compress the bayer array because the errors introduced by the compression will be filtered by the demosaicing interpolation anyway...

/me who's working on implementing an high quality and almost not slow demosaicing algorithm.

Steve, I meant that the video resolution is binned, with one red and blue and two green pixels per binned pixel. This maybe available as an 4:4:4 pixel from the sensor, but 4:4:4 requires too much data rate to compress with quality. Instead, we can use one of the color components to produce an equivalent Bayer pixel (straight, simple, fast, on the fly conversion, by simply dropping the other components) then compress that using one of the schemes here, or grey scale. This will give three times less compression for the data rate (or closer to 2:1 for 720p) but with an quality superior to 4:2:0.

Does this sound feasible?

(Matteo,)

The setup for a dual lens system should be very suitable for film, because the focus knob (on the side) would work the same way as a follow focus.
I doesn't have to be MF of course, I'll just test it because I have some camera's like these lying around to use: http://k53.pbase.com/o4/87/331787/1/53267450.RolleicordV2.jpg
I hope the lens on such a camera is bright enough for indoor use. Probably not.



I had a chance to talk to Andrey about the development of the new model and things like that. I have to say, my enthusiasm is only growing. It's wonderful to work on something that is in constant development. Thanks to 'open source' but most of all to the 'open mind' of Andrey.

Will the Dirac wavelet FPGA project be of any good to you, or is there any other wavelet FPGA codecs out there (2D-5D)?
Maybe, or maybe it can help somebody else to port it to our cameras.

There is no point in demosaicing before compression, since in this process you're going to "recreate" missing information. And you want to compress the bayer array because the errors introduced by the compression will be filtered by the demosaicing interpolation anyway...

For some application - yes, you are right. Somewhere earlier in this thread I explained the modification to JPEG we use to compress images where demosaic algorithms are implemented on the client PC. But for many applications it is not fast enough. So we use compromise - 4:2:0 where number of pixels to compress is just 50% (not 200%) more than in raw sensor data.

I have been able to record smooth 90% quality but it also depends on what resolution your shooting at. I prefer to use 2000x800 because I think that gives me the best results. At 90% quality there is a delay in the streaming video when recording at the same time, so I usually use 85.

some screen caps below from previous 333 test projects.
http://www.buysmartpc.com/333/1.png
http://www.buysmartpc.com/333/vlcsnap-1519185.jpg
http://www.buysmartpc.com/333/vlcsnap-1519992.jpg
http://www.buysmartpc.com/333/paint.jpg

Daniel, I finally got to review your images, I did find some compression problems and blocking, but seemed to be better than normal. But what % was each one recorded at, was it lower than 85%?

I've uploaded a full resolution version of the 'demo-documentary' I shot with the Elphel 333, the wax 35mm adapter and TV-lens. This short film will probably also be shown at LinuxTAG.
It's compressed with Xvid, but very acceptable to show. (200MB)

http://community.elphel.com/videos/RomainFULL.avi

I loved the look on this, so different, to really get a good idea of the motion I converted it to divx with the divx encoder (wouldn't play the xvid on my multi player) and put it on my regular 29 CRT tv , Beautiful! and it feels really special, not at all videoish, so far it's the most "different" look I've seen

what framerate did you shoot this in? because I was reading that it was hard to capture complex shots without frame drops and all that stuff and they were talking 19fps

Daniel, I finally got to review your images, I did find some compression problems and blocking, but seemed to be better than normal. But what % was each one recorded at, was it lower than 85%?

They were all shot at 85% quality and I was using a letus35 on all of them except 1.png which I believe to be the most sharp in the bunch. Im not comfortable using 90% yet because of the risk I may get a dropped frame.

It seems that when I use the 35mm adapter the video is not as sharp. I need to do some more tests, it may be that im not properly focused on the gg or possibly just because the lens is wider open.

I have the opportunity to work with the 353 already. I'll keep everyone posted on the results. I'll be ordering the power supplier, so it'll take some time to get started.
Oscar, current code uses 48MHz (like in older camera/sensor), we'll increase it to 96MHz as new sensor allows - that will twice decrease effect of the rolling shutter. This is something I'll be working in 2 weeks (I will also connect and synchronize 10 of the 5MPix cameras - that project requires 96MHz.

Hi,

I'm new here but I've been following this thread for a long time. I have a question regarding what Andrey just said.

If you shoot at 96Hz instead of 48Hz, wouldn't that ruin the cinematic motion blur you get with a 1/48 shutter?

And Oscar, did you shoot your Romain documentary at 1/48 or 1/96? Your video looks fantastic. I've never seen anything like it. It reminds me of old film cameras but IMO it lacks some more motion blur, that's why I'm asking for the shutter speed you used. If it was 1/48 then it means the Elphel cannot deliver a really cinematic mblur.

Appart from that, congratulations Andrey. Your camera is a real low cost solution for people trying to get something different and more cinematic than 90% of the other cameras. I'm a great fan of the Elphel and I plan on buying a 353 if my little project doesn't come up as expected.

I've been following this topic for over a year now, and I'm slowly understanding some of the elements and terminology involved, but still in the dark on much of it.

Oscar, the footage you posted looked fantastic. I was very impressed.

Here are a few questions (demonstrating some of my technical ignorance).

1) Oscar, your short project looked great. Is there a reason this setup isn't ready to be used on other short films?
2) (everyone) It sounds like the "bugs" or "roadblocks" remaining have more to do with software issues than hardware. Is this correct? And if so... is it a matter of colaborative manpower?
3) Could someone with a production (but not engineering or technical) background (like me) purchase some of the equipment and collaborate somehow without knowledge of the programming end of it?

Thanks, Daniel

If you shoot at 96Hz instead of 48Hz, wouldn't that ruin the cinematic motion blur you get with a 1/48 shutter?
And Oscar, did you shoot your Romain documentary at 1/48 or 1/96?

Jose,

Oscar could not use 96 MHz because 1) - he did not have 5MPix sensor capable of that pixel clock and 2) current FPGA code was not designed for such frequency and there is a software enforced limit of 48MHz to protect earlier sensors.

The pixel frequency determines frame time (it can not be less than (simplified)

(number_of_pixels_in_a row+extra_line_time) * (number_of_rows + extra_rows) / pixel_frequency
extra_line_time and extra_rows are sensor-specific parameters

With Electronic Rolling Shutter (used in virtually all CMOS image sensors) - http://wiki.elphel.com/index.php?title=Electronic_Rolling_Shutter the exposure time (long results in motion blur) and frame readout time both determine what you get in the images but in different ways. And so far I do not know of any value of ERS distortion so it is better to try to minimize it.

Ok, so with the Elphel is it possible to program a long exposure time to get more motion blur while keeping a short frame readout?

yes, you can
take in mind that the motion blur you can achieve is somethig different to the one you 're looking for
cmos sensor function like a paper scanner
take a look at
http://wiki.elphel.com/index.php?title=HD_cinema_camera_Development_FAQ#Background
maybe it clarify you some more about the elphel camera

what framerate did you shoot this in? because I was reading that it was hard to capture complex shots without frame drops and all that stuff and they were talking 19fps

I always shoot at 24Fps. The frame drops are mostly caused by the amount of compression. I can shoot perfectly at 85% 24fps with the Elphel 333.
And if I understand correctly, in the future it should be possible to record the stream of the 353 directly to a separate hard drive, leaving more power on the laptop, so you can crack up the compression quality even more. (Andrey correct me if I'm wrong)


It reminds me of old film cameras but IMO it lacks some more motion blur, that's why I'm asking for the shutter speed you used. If it was 1/48 then it means the Elphel cannot deliver a really cinematic mblur.

I prefer to reduce the rolling shutter artifact above motion blur. Maybe it's a stupid suggestion, but you can add pretty realistic motion blur in software like after effects, maybe even in VirtualDub.



1) Oscar, your short project looked great. Is there a reason this setup isn't ready to be used on other short films?
2) (everyone) It sounds like the "bugs" or "roadblocks" remaining have more to do with software issues than hardware. Is this correct? And if so... is it a matter of colaborative manpower?
3) Could someone with a production (but not engineering or technical) background (like me) purchase some of the equipment and collaborate somehow without knowledge of the programming end of it?

1)My current setup is fairly usable for short films. I don't have a short film planned right now. I just want to work on some things a bit. I'll post my plans for a dual lens adapter soon. And I'll be working with the 353 model.

2)Yes, and this forum is a wonderful place to get all ideas together. ...it's more a question of collaborative 'brain' power.

3)We tried to make the control interface of the 333 model more suitable for film making, but I hope the software for the 353 model will be easier to modify.

They were all shot at 85% quality and I was using a letus35 on all of them except 1.png which I believe to be the most sharp in the bunch. Im not comfortable using 90% yet because of the risk I may get a dropped frame.

It seems that when I use the 35mm adapter the video is not as sharp. I need to do some more tests, it may be that im not properly focused on the gg or possibly just because the lens is wider open.

Thanks Daniel.

35mm adaptors are notorious for not working well with HD cameras. I don't know what peoples are up to in the adaptor threads, but there were HD ones.

Just a question... Can the new 353 capture 4:2:2? What about raw bayer for later filtering?

Thanks.

Just a question... Can the new 353 capture 4:2:2?


No, just 4:2:0

What about raw bayer for later filtering?
Thanks.

Right now the raw Bayer code is broken, but I'll restore it in couple weeks (needed for a very urgent project). It is a special type of JPEG (where pixels are treated as monochrome, but rearranged in each 16x16 macroblock) that I described somewhere in this thread (should better put it in our wiki)

Hi Andrey,

I just found the posts talking about that bayer jpeg but I still have some questions.

Does "bayer jpeg" mean we can have full color info stored into a bayer pattern for later filtering but with jpeg file sizes?

Can you still get HD resolutions at 24fps with that kind of compression?

Thanks!

Hi Andrey,

I just found the posts talking about that bayer jpeg but I still have some questions.

Does "bayer jpeg" mean we can have full color info stored into a bayer pattern for later filtering but with jpeg file sizes?

Can you still get HD resolutions at 24fps with that kind of compression?

Thanks!

As full, as in the sensor itself. But, of course, 8bits/pixel (as in JPEG). That is also not a big loss as we use non-linear ("gamma") table on the input that makes the data to match noise performance of the sensor (see http://www.elphel.com/actualbits.html).

Current code in the camera (ported from 333) runs compressor at 120MHz, pixel rate is 1/2 of that and with 4:2:0 it corresponds to 40MPix/sec sensor pixels).

In the 353 the components are faster, so it can run (and is tested) at 160 MHz (53MPix/sec with 4:2:0). Our special "raw bayer" JPEG might run faster (80 MPix/sec @ 160 MHz), but the current FPGA code still waits for non-existent color components, so the compression speed is the same.

Great, thanks Andrey.

Thanks Andrey.

Some people are interested in 2 mpixel shooting, I think that 1/1.3mpixel shooting might be more practical in terms of quality. As there is no Bayer below full resolution, is there any chance to use the binned color components, as I described before, to form an 1mpixel Bayer pattern?

At what average compression ratios does the Bayer codec achieve lossless, and visually lossless, Andrey.

I thought that 4:2:2 and 4:4:4 was automatically available with higher quality? Was ti always 4:2:0.


Thanks

Wayne.

hi to all yesterday I've taken a compact flash extreme 3 2gb card for cheap
....it is a 20mb/s read and write card amazing maybe it will be enough so we haven't any power issue and no need of a hd!
one idea I've in mind looking at oscar short film and other hd 24p shoots!
cause to the design of the ccd or the cmos sensor comparing digital to film shoots you can notice an evident flickering in the digital one due to the minor motion blur so, to fix this , in high light shooting why don't shoot in 48p mode and overlap frame one with 50% trasparent frame 2 and so on to achieve a smooth 24p footage...did you think it is possible diectly in camera or it is best to make it in post!?

oh and I've seen on the net that there are 8gb and more
extreme 4 card (85 euro) and are 40MB/sec read and write!

hi to all yesterday I've taken a compact flash extreme 3 2gb card for cheap
....it is a 20mb/s read and write card amazing maybe it will be enough so we haven't any power issue and no need of a hd!
one idea I've in mind looking at oscar short film and other hd 24p shoots!
cause to the design of the ccd or the cmos sensor comparing digital to film shoots you can notice an evident flickering in the digital one due to the minor motion blur so, to fix this , in high light shooting why don't shoot in 48p mode and overlap frame one with 50% trasparent frame 2 and so on to achieve a smooth 24p footage...did you think it is possible diectly in camera or it is best to make it in post!?

Another future idea would be too shoot 50p full shutter for TV and digital cinema, slow video and audio by 4%, and drop every second frame to get 180 degree 24p?

Why not shoot at 1/48 sec exposure time while keeping frame readout time low? Andrey said it's possible.

Why not shoot at 1/48 sec exposure time while keeping frame readout time low? Andrey said it's possible.

yes you can but I want to increase the motion blur! with your method you'll get very sharp image that result in a flickering video like "28 days later" movies

Isn't real film shot at 1/48 shutter? If the sensor reads the frame fast but keeps "adding light" for 1/48 of a second wouldn't it result in more motion blur?

I'm also very interested in getting filmlike motion blur with the Elphel cause I'm using the same micron sensor for my project.

The method I mentioned allows everything to be done. Dropping every second frame gives the conventional film look for 25fps HDTV, slowing 4% gives conventional film look, ever drop framed can be used to smooth out the motion at 24/25fps. Using all the frames gives an ultra-smooth 50p HDTV or digital Cinema, or interlace frames can be easily extracted, allowing for distribution across all formats with the minimum, of image recalculations. The issue is, what is future proof? At 720p50 it can be done in 100mb/s, 1080p50 would need an bit more. At the moment, he industry is waking up to 50p/60p (and 60p HDTV should be able to display higher quality 50p).

An month without an post, you guys are never that quiet, what's happening?

maybe all of you are on holiday!

or maybe someone is gone to another project? ....New DIY HD Cinema Camera Project??

for me I'm here and I'm looking on the net for good medium format lens to use with my next adapter :-)

Hey, I still take a look at this thread from time to time. The Elphel's the only one that can do what we can't do yet. Deliver an already compressed stream to the computer. Data rate's not as large as uncompressed raw so recording to disk is possible.

They're also using the same Micron sensor we were testing, so we know image and motion are just great. Very filmlike.

I hope Andrey's still working on it.

Last activity in the CVS was 7 days ago, so yeah they are basically updating the code every week...

I've found on the net this sensor ...is a strange one is square global shutter and is very big http://download.cypress.com.edgesuite.net/design_resources/datasheets/contents/lupa_4000_8.pdf
if it is not too expansive maybe can be used for the next elphel

I think that's the one Cypress offered me. The guy didn't say the reference but he said it was a big global shutter sensor that was able to do 2k 2.39:1 at 24fps. He said it was a bit pricey too.

http://parts.digikey.com/1/parts/946134-ic-sensor-image-4mp-127pga-lupa4000m.html

1750$ :whistle:
I don't know if it's the nude sensor because it's in the evaluation kit category but yeah, "a bit pricey" ;)

That's just the sensor. I've read somewhere the price for the whole evaluation package is like $5000. Anyway the guy from Cypress will send me a price list on monday.

wow 1750 dollars only the sensor
compared to the micron that is 33.19$
....if in some year it will be taken for cheap
I think that this is what we need...
no adaptor 35mm sensor size and the possibility
to "shift the image" like some very expansive lens
....full to control vertical architecture line distorsion :-)
(this can be made already but with less benefit cause the size of the sensor)
and all in camera maybe it can be adapted to the elphel 353
and with these spec. :-) it will rock

Hey, I still take a look at this thread from time to time. The Elphel's the only one that can do what we can't do yet. Deliver an already compressed stream to the computer. Data rate's not as large as uncompressed raw so recording to disk is possible.

They're also using the same Micron sensor we were testing, so we know image and motion are just great. Very filmlike.

I hope Andrey's still working on it.

It is also much cheaper and smaller than what is being proposed over there, more comprehensive. You are looking at the a size very close to the existing small box (handheld).

The Micron, I don't like the quality, an few years back it was obviously inferior to what else was available (but not an FF Ibis5a relying on internal ADC) but we can only hope that the modern micron binned to 720p can accomplish something.

The Lupa sensor is probably not available at an cheap price, it was monochrome only requiring an expensive for color. They mention it is available with color mask, maybe it has changed. Altasens is an better choice I think, they have an increased range.

Altasens doesn't support small projects. They were my first option.

Yes, this thread is a bit quiet lately.
I have the 353, but I just want to get started on it when I 'really' have the time.
I think within a month I can spend all my free time on this project again.

Andrey was (or still is) in China for the development of some parts, so it seems quiet, but there's probably allot going on. I'll be shooting another test in the middle of France the coming weeks, but I'll probably use the 333, because I don't have portable power or a power transformer for a car battery yet for the 535. I don't think I have an internet connection in France, but if I do, I'll post what I'm doing.
I think the Elphel still is the most promising camera for us 'alternative imaging' filmmakers.

Altasens doesn't support small projects. They were my first option.

Yes, I know, I tried to talk to them ages ago, but you can go through their agents, like hanvision. They are producing an number of different chips in the background, apparently, and so you can find cameras based on them. Foveon should be getting an move on, to go big and license more, before their patent worth runs down.

Oscar

The Elphel is still the most promising, but converting an still or webcam, or doing HDMI on something cheap, is still desirable.