Hello,

Thanks for your question Wayne, I'm currently developing the video editor plugin for the "codec".

But I must ask you that please don't exaggerate: as I stated earlier the encoder will only do the minimum to fit into the 353's disk write speed, nothing more. If that means only 2:1 compression then that will be. I wouldn't even call it a "codec" at all as it will be dumb as possible. And if I have to do a really complex encoder then I won't do it.

Also, I only make a verilog encoder and an importer for a video editor software so the term "codec" isn't appropriate, again. So if Rob is looking for a general purpose codec then this might not be his choice.

And as I still don't have a camera I cannot test the encoder so I don't know that it will produce the desired results or not. Let's wait with this.

Zsolt

Big Question is:

Why not giving to Elphel camera a PCMCIA interface?
It shouldnt be so difficult ,there is already Verilog/VHDL code for a PCI interface at Opencores......, just add a device driver and a grain of salt.

BTW, is there any way for me to get uncompressed 10 or 12 bit bayer video????

I mean It was impossible for me to get stuff like that to experiment with.

Zsolt, that sounds like a very good approach. Will it be open source?

BTW, do you know if the 353 will support writing to disk and streaming Ogg Theora video simultaneously?

Thanks!

Zolt, this is sucking time out of my tight schedule, but where did I exaggerate? I've asked you questions in times past, about your claims of much more than 2:1 performance from your experiments, but received very little reply. I merely stated that you could, on your large claims, out do the lossless codec that was worked on for Obin's camera, which was simple and 2:1 or so, the cineform codec is visually lossless (not really lossless, not so much of interest) so is a different matter to which I was talking about (but from your previous claims it sounds like you could have a go at that). The need for only a codec that could , at a minimum (2:1) to squeeze raw through, was originally my proposal anyway.
So, please, I don't need anybody else needlessly coming down on me, I am aware of the limitations and the possibilities, which is usually more than we can say for most.

You and Rob have fun together. As you can see, I am mostly away from this, deliberately.

Thanks for your efforts.

I imagine he only wants something that will fit the data through and then can transcode to whatever he wants for editing.


External PC units for display etc:

If anybody wants to hook a computer for display/control, then 5 inch UMPC's (mini tablet Window XP computers) have just been shown off for next year, and Apple is hoped to come out with one of these next year. The Intel G965 chipset, with unified shader direct X 10 GPU, I suspect they will sort out the problems of, and the mobile version is expected to come Q1 07. Eventually, I expect this will come to the UMPC. But for anybody wanting a small PC hooked to the unit, without graphic card, the G965 (G versions only) might be well positioned.

If it works as expected, then it will be open.

I think we can manage that, that's what I'm also planning to do. But we have to keep in mind the processor's bandwith issues when dealing with two streams simultaneously.

Well, I'm sure we'll have 2:1 but not sure it will be enough for actual recording. I'm curious just as you are, probably, as I've never seen a raw bayer stream in my whole life...

Didn't mean to offend you. Actually I see nothing wrong in what you say but I fear that others might overestimate the things that I do.

Zsolt

Excellent. If there is any way I can help with this, I will.
Yep, I'm really hoping it has enough horsepower to do both. Do you happen to know how big of a hard drive it will support? One of those 750GB drives would be cool, or perhaps a 1TB drive when they come out in a few months :-)
From my calculations, 2:1 should be just fine, unless it overloads the CPU. Also, I can provide you with some raw bayer images, though it may take me a few days to pull it together.

Thanks!

I've uploaded a single raw frame here: https://sourceforge.net/project/show...ease_id=452036

Thanks Zolt.

I think testing the effectiveness of the JPEG compression on an grayscale of the bayer mask should be tested to see how well bayer interpolation holds together, with adjustments made to the compressors parameters. The main parameter to test is the quantisation phase (? I forget, the phase when they divide down the pixels to even them out to make them more compressible). A maximum of 3:1 compression, would probably be best. it would probably be near lossless at the most, and nowhere near as good as a bayer compressor. But ti would be better for existing camera owners. Very simple intra frame compression then could be added, as was being designed elsewhere.

Rob, a lot of those drives are power hungry and big, so select carefully.

I would like to move in different directions than doing anything camera/codec development related, but I am interested in your bayer footage. if you could post 10 second scenes and bayer format information that would be useful? A simple scene, simple shooting, an complex movement but simple scene, and a complex scene and complex movement. This would provide good test subjects for codec performance. I'm itching, but have too much else I should be doing.

Sure, I was just thinking that the 16-bit mode of JPEG compression would help to preserve the high bit depth, despite the quantization step.
Sure, but a drive is something you could easy swap depending on the particular application. For a studio camera on a tripod, a full-sized 3.5" 750GB drive is no big deal, as long as the noise could be kept down.
That would be useful for me as well; I'll see if I can fit it into my schedule.
I hear you! :-)

It has linux running on it so I don't think there's any limit.

What calculations? I had some calculations earlier (they're on this thread somewhere) but Andrey instantly proved some of them wrong so I decided not to calculate but wait for the test results instead...

Thanks. Actually I've seen single raw images before, while I was building my own camera but was unable to get a continuous stream of images.

There are some simple but good ideas here: http://www.silentpcreview.com/

...and for me and probably for Juan here as well (altough we don't know his intentions with those raws). The three types of videos Wayne mentioned would do just fine. Hope you find some time to record them.

Zsolt

I saw the figure 15 MB/sec, and calculated that a 2:1 compression ratio applied the raw frame (1280x720, 10 bit @ 24fps) would fit. Of course, if this ratio is an average, then we might have problems during particularly detailed or noisy scenes.
Thanks!
I'm sure I will. (Of course, they will probably have some nasty rolling shutter artifacts.)

Where? The theora streamer's output rate on ethernet is 70Mb/s=8.75MB/s.

Zsolt

I was referring to hard drive (IDE) performance, not Ethernet. I thought I saw the 15 MB/sec figure in this thread but I guess it was this post, which states the theoretical maximum for the ETRAX 100LX as 16.67 MB/s. I must have rounded it down to 15.

I did a bit of Googling and found this document which states that the FS supports UltraDMA mode 2 (33 MB/s).

The bandwidth required for 1280x720, 10 bits @ 24fps is 26.4 MB/second, so it may be possible for the FS to write the raw data to disk with no compression at all, or with only slight compression.

But the guy also writes that 12MB/s is the practical maximum and another guy writes that he measured 7MB/s. So I guess the reality can be between the two.

The new micron sensor has 12 bit output so it's slightly larger than that.

Andrey stated in this thread somewhere that we shouldn't expect much improvement if we write to disk instead of ethernet.

Based on all of the above, I expect something around 10MB/s and that requires 3:1 compression ratio. Anyhow, we're going to be on the edge, that's for sure.


Zsolt

True, but that's for the 100LX. The FS supports UltraDMA mode 2 (33 MB/s), so presumably the maximum would be somewhere between 14 and 24 MB/s.
I was assuming a LUT to reduce it to 10 bit. But if it turns out to be possible to handle the full 12 bits, that would be better of course.

BTW, have you looked at some of the components available on OpenCores such as xmatchpro lossless data compressor or Video compression systems?

Thanks!

And the 353 will use the FS? Anyway, Andrey knows the system better than us, so if he says "not much improvement can be expected" then I believe that. My encoder will be somewhat faster than the theora stuff, but it could well be that some bottleneck comes from the fpga itself so the processor won't be able drive the IDE interface to a maximum.

No. I don't know how easy it would be to insert a third party encoder into the Elphel design.

Zsolt

That's what I understand. You may be right about the increase in performance (or lack thereof) but I guess we'll see.

Edit -- Andrey mentions using the ETRAX FS in this post and this post and indicates that the CPU should be roughly twice as fast.

I see that Etrax FS has PCI support built in so, does it mean that you just can connect a PCI board to it or what?

Juan, I believe you'd have to design a PCI bus into your custom board; the Elphel won't have this.

Back on track
 

I had do deal with some events of a personal matter - that caused slowdown with the 353 project. But now I'm again working on putting life into the first unit, CPU+network+memory are tested, now we have to write NAND flash boot loader for ETRAX FS - it turned out that it is not yet available from Axis.

For those who are interested, I can email circuit diagrams - I don't want to post it until I'm sure it does not have errors (2 already found/fixed so far).

We also made some noise measurements with Micron 3 and 5 megapixel sensors and plan to make this software as part of the camera firmware. 3 MPix has about 22ke- full well capacity, 5Mpix - about 10ke- with rather low readout noise - just above a couple electrons (with high settings of analog gain).

Andrey, if I was able to find funding and time to help with this project, what would I need to purchase and roughly how much would it cost?

Thanks!

Rob, when somebody wants to help with the project we usually provide hardware free of charge, but I don't have any extra 353 boards right now. The one I'm playing with is (by tradition) mostly built by myself - only BGA chips are mounted professionally - I don't have an oven.

As soon as I'll test the board electrically (with minimal software running) we'll be able to order a small batch assembled - then it will go to developers.

I would love to help, if I possibly can. In the meantime I will get familiar with the 333 source code. What else should I do/read up on?

Thanks!

Andrey,

What did you find for noise average db and latitude, for 3 mpixel, and 5 mpixel binned to 720p?


Have you seen this brand, they have 170db latitude, and 0.001 lux sensitivity?:

http://www.hdrc.com/sensors.htm
http://www.hdrc.com/hdrctech.htm

Some of the demo pictures are pretty interesting, and I wonder when they will go HD.


Thanks

Wayne.

Guys, is there any way for me to get a small video clip of the bayer kind of 10 bit bitdepth?

Andrey, I know I can sound really stupid sometimes but I will make this question anyway.

Is there any advantage about using an ATMEL AVR 32 bit RISC instead of the actual ETRAX FS?

Second stupid question:

http://www.latticesemi.com/products/fpga/

Lattice has some interesting and inexpensive FPGAS with gigabit output/input, is there any use for them inside the Camera framework?

It's on my list of things to do :-)

There is virtually no information there, but I am not impressed. The logarithmic transfer function is an old idea easily implemented in CMOS sensors. Sensitivity - it is not so easy to fool the Nature - quantum efficiency/amplifier noise levels of the modern CMOS sensors (like Micron's) are not that far from the theoretical limit.

0.001lx? - easy, any regular sensor can do it with long enough exposure.

Rob, what is exactly do you want to develop for the camera?

I am open to helping in any way I can, but in particular I am interested in implementing cinema-related features. Andrey mentioned that the board would have an IDE connector; I would like to help implement writing to disk.

Zsolt, you mentioned working on a lossless compressor for the FPGA (to reduce the size of the the raw image data) and I'd love to help with that as well. I've never had a change to do any FPGA work and it sounds like an interesting challenge.

I'd also like to write front-end (or is that back-end?) software to provide a viewfinder/heads-up interface. Perhaps this would involve a modification/plugin for ObscuraCap; I'm not sure yet.

I just saw an announcement for Hitachi's new CinemaStar hard drives. They could be good choices to use with the 353.

I've seen a distinct lack of this outside of SD CCD's. It is also a matter of noise and gain, which CMOS is not as good at.

The QE of the Microns is about 40% or lower, isn't it (not to mention low fill factors)? I've seen a sensor with upto 90% QE, and experiments in recent years have succeeded in getting one photon to move two electrons (QE*2). Then there is loss from the color filter, and another advantage that the FF sensor had, was that because of the 100% fill factor, a microlens was not required, so very fast aperture lens could be used. Rai, that designed the Drake camera, could get F0.85 or something lens. So, there is a lot of latitude over what the Micron can currently do. If HDRC go HD can get some of this advantage I think it could be a sweet deal. I can't get a reply from them yet, I am interested in one of these cameras for my own project, and some research purposes.


Thanks

Wayne.

Implementing my encoder is a one-man job so I'd like to do it myself. But if you can do the disk recording part (can be tough if the processor is too weak to handle two streams), that's okay for me.

The elphel already has the client which can be used as viewfinder, I suppose. It's not easy to write a streamer and our stream will be huge so, for starters, we should try to use the current theora stuff.

The problem is that the fpga will have to create two streams simultaneously and I don't know that the architecture and software will support that or not. Andrey?

And if we somehow manage to create two streams, the Theora will be very bad quality - I don't know if it will be enough for precise focusing. I'm afraid we'll have to modify the theora encoder too (maybe with losing all the color information) and that's also not very easy.

Zsolt

Sounds good.
I agree, no reason to reinvent the wheel. Perhaps a custom version of VLC? I'll have to look into the options.
I was thinking perhaps two modes --

• Focus mode: No disk writes, high-quality Theora @ low frame rate (if necessary)
• Capture mode: Disk writes, lower-quality Theora @ full frame rate

What do you think?

You can develop the disk recording part with theora while I'm working on my encoder.

Follow focus would be nice but if there's no other way then we must do it like you've said.

Zsolt

Good idea -- and perhaps the raw frames to see how many FPS it can handle without any compression at all.
Given that the CPU and FPGA will be quite a bit more powerful, it may be possible to do both. I guess we'll have to see how close to the edge we are.

Good ideas guys.


Andrey, I know that you are somewhat into sensor design, I am not, but I had a few ideas today, and some older ones.

I was thinking that the sampling mechanism between four or more Bayer sensor pads could be shared, reducing fill factor. Using a multiplexer, the inputs of different pixels could be sampled sequentially or combined into different combinations producing an new value for binning and even debayering (4Mp binned 2*2 to either red green or blue for 720p bayer for that position). Because of the speed of internal sample the delay between pixels should be minimal. A larger number of pixels could be shared depending on noise induced on trace length, to further reduce fill factor loss. A large amount of data could be fed out in parallel because of the serial digital data nature. If you used a serial like A/D conversion, the fill factor loss could be reduced to a few percent. I turned to the thought of storing charge, and realised that the chip manufacture process is compatible with extra Vapour layering (or magnetic level layering). To stop over charge effects, the sampling method gives some resistance of course, but then continual sampling and dumping/discharge of over charge would give further resistance, but an over charge sink can be made to earth it (I have a unique process to do this (to handle large amounts of over charge) but it has greater potential and I have this in mind for other things). The circuits can be protected and the pads can be covered with layers promoting light gathering and capacitance. This allows for very much capacitance.

I realise that some of this sounds suspiciously a bit like a CCD and existing cmos methods, but I think it is an alternative method that can easily be added to a cmos assembly line without having to substantial change existing components, and keeping it open for production runs of other normal chip items.

What do you think about this process, your comments?


Thanks

Wayne.

Wayne, I'm sorry - I never worked on sensor design myself - only used them. So I can not provide you with educated opinion on your ideas.

That's O.K.

I saw your paper on some sensor function some time ago, and thought you might be into the senor hardware side.


Thanks

Wayne.

Dear Wayne
 

Do you have a glue about the Playstation 3 and the linux disk
in France maybe, maybe middle next year ????