Development Platform for DIY cameras

[Rob Lohman]

How drake exactly did it is uknown at this point in time, but I have two guesses:

1) a computer running linux, like some embedded system

2) an FPGA or other processor programmed by them

Both could handle what they are doing if the right hardware and programming
skills where put together.

Depending on how they store the data it is actually quite feasible to do with
current harddisks, remember, they are storing 8 bit data, not 10 or 12!! Also,
if they store the original bayer data (more efficient) it will require this datarate
for 1280 x 720 @ 24 fps:

22,118,400 bytes per second, or 21.1 MB/s.

If it is not bayered it goes up with a factor of 3 to get 63.3 MB/s (which is
a problem).

Perhaps they are using two harddisks with a (software like) RAID solution,
not that hard to do.

[Michael Maier]

True. But as you add HDDs you also increase power consumption. I think what makes more sense is what you said, they store the original bayer data. (As If I knew what's that LOL).

Linux has some great HD appliacations by the way.

[Rob Lohman]

Read up on Bayer at this URL: http://www.siliconimaging.com/RGB%20Bayer.htm

Since there is only a single chip in these camera's you have a problem to
get color (since basically all these chip are mono [B&W], except for one
design which is expensive). What they have done is add a tiny color filter
to each pixel on the chip, so each pixel only filters a certain color.

Since the human eye is more sensitive to green the green pixel is used the
most often, so 50% of the pixels on a chip are greenm, 25% are red and the
last 25% are blue. Now to get the full color information you need to re-sample
this information to RGB (red, green & blue), or 3 bytes per pixel. So your
storage requirements increase 3 times.

It's much easier/better (perhaps) to do this after capture on a dedicated
computer, this dramatically decreases processing power, storage space and
bandwidth.

The "downside" to having this bayer chips is that you can debate if they are
really HD or not (since for a 1280 x 720 sensor you would have a resolution
of 640 x 360 for green and 320 x 180 for both blue and red). However,
depending on the quality of the de-bayering algorithm (to get RGB) you can
get some excellent results.

The quality of this algorithm also determines if you have unwanted effects
like fixed pattern noise (where you can "see" the holes in the color ranges
due to missing pixels etc.).

[Wayne Morellini]

Originally Posted by Keith Wakeham

Their is a lot to try and consider when trying to compare cmos and ccd's.

The full well capacity may be higher on the cmos, but you will normally only get full well in super bright conditions, and for that amount of charge to build

Stepping down/or up the lens, or using an ND, would allow you to move/compress the image into this extra range for extra latitude. CCD's and sensors in general have latitude problems compared to Film. Wouldn't the actual dynamic range be a function of well capacity and noise (QE, fill factor, pad size, a lot less so as they more affect where sensitivity starts, and where the range window is). I'm curious how the broken down noise values of the IBIS effects the true range, and what SN they translate into. I had a try but the SN/Range figures turn up too big, or too small.

But what you also need to look at is really how much light is getting to the sensor, This will determine a lot. The kodak has larger imaging sites which means they can gather more electrons. So lets do a comparision and say the ibis5 gathers 10000 electrons @ 30% qe

The Ibis uses a new mechanism around the photo site to gather and calculate the photons missing the primary site, give it near 100% fill factor. Other sensors use microlens to gather extra light to the photo site, they can not gather 100%, so some light misses. The Micro lens restricts maximum aperture (1.4 for acceptable image I understand) while the Ibis allows super-wide apertures allowing a number of stops (double) more light, and 35mm like DOF (0.75 aperture). Of greater concern is the size difference between the sensors.

The QE of the Ibis is about the average of the 3 different QE values for the Kodak.

Also, did you look at the ibis colour filter , the ibis colour filter response is poor at best. When blue filter picks up a green light wave on ibis it is about 1/2 a blue so will affect the blue brightness even though it is green that is striking the sensor, but with kodak is about 1/9. The kodak filters are clearly superior - unfortunately for most people.

Now this is interesting, I suspected there could be something in the filters, but received no confirmation of this. We are still getting more photons (and I believe that one of the film camera companies has a way to use this spill over to increase accuracy in green and calculate it out in the blue, but still very massive in size). Diohcotic?? colour filtering is pretty standardised technology, you would think they would have had similar performance.

I'm not ragging on the ibis, its a good chip that when used properly and it can really give a nice picture but the colour filters remain poor at best and this is where the IBIS gets it poor colour. In a 3 cmos setup the ibis would be amazing because you could control the filters.

Actually Sumix is hoping to attempt one.

If i took an ibis right now i could literally jump to fpga programming and not have to worry about the issues that i do now like timing, but when the numbers show that the kodak will give more accurate colours and a better s/n ratio because it needs less light that was where i decided to start.

I have not seen an SN for the Ibis, what is it?

I did see that imperx camera before along with one from red lake. I'd love to get my hands on one but I just don't have the money. The redlake one was about 3500 euros, so that would be about 5k in canada plus a cameralink card. I'd assume the same for the imprex one also and i'm really trying to avoid cameralink.

Red Lake is expensive, I think you will see 2 times difference in price of some of these manufacturers. I mention the Lynx, because there has been much interest in on head programmable computer/FPGA. In a GigaE or Firewire camera there is the possibility of controlling an hard disk from the head (cutting out computer). If you can somehow route external controls into the box (say through serial port) and have preview port (on some cameras) and lens control, you completely eliminate the computer and make the whole system development very simple. If you haven't bought the Kodak development kit yet, I would factor in development costs compared to buying the cheapest comparable all in one camera.

I'll be honest, a production KAI-2093 is 1200 USD, while a ibis is somewhere around 300 USD. It might cost just as much to build a 3 cmos setup or be cheaper than what i have designed so far. That really makes me sad now that i think on it.

Actually, we have heard of 100euro for the Ibis (maybe lower), Micron is cheaper again. But these are quantity orders (not sure wherever it was small quantity (less than 1000 to 100, or above 1000-10K)).

Wonder if their are and 3 cmos ibis cameras around, that would be cool.

Hmm, your onto something, I wonder if somebody already has them?

On Juan's suggestion (wasn't I rubbished for suggesting doing a three chip prism before) that equipment seems a good price. If you google D.I.Y/home made clean room and vacuum chambers etc I'm sure you will find some (there was even one for making a CRT tube).

A prism should be available cheap, they are mostly just a chunk of glass. For prototyping you could (theoretically) remove one from an compatible defunct camera. Now Panasonic do cheap prism (3M do cheap projector prism technology so might also have CCD prism). There are a number of optic manufactures around the world, including cheap ones in third world countries (like India) that may even supply to bigger companies. They might be a avenue to get cheap prism once you have finished prototyping. But I think 3 chip will be very expensive, and much work, for small operation like yours (you have to learn optics and problems with HD prisms etc).

It is good to see you getting support here, I wish you luck. I am needing to rationalise my time into more constructive areas at the moment, so look after Keith guys.

[Keith Wakeham]

Wayne, I think your right about all the factors that influence lattitude but i was unaware that ibis are using some sort of new method other than microlens.

All I know is that it clearly states in the IBIS spec doc that Peak QE for IBIS is >30% but averages around 25-30% and all the colour filter response is given as a relative number.

As for colour filters, well before i compared i assumed that they would be similar, but it really appeared that the blue filter was really affected by green which totally surprised me.

S/N ratios are just used to describe the relationship between the amount of signal and amount of noise. This has been the big problem with cmos for the last several years and is why it hasn't been used much in any type of high end camera, the S/N ratios were hovering around the 40-45 db mark while an interline ccd would be around 55 db. Now, considering that these are logarithmic, that is a huge amount of noise in cmos, but recently cmos has had a nice bump that brougth them up to high end quality. A few years ago nobody would have taken cmos seriously, but thats technology for you. As for the actual numbers, well if they don't give it out than it is really hard to say, but i would suspect that ibis is close to 60db which is good.

I'm seriously thinking about the 3 cmos thing now, and I have factored in the development kit for kodak, but I'm unwilling to make a big purchase of several thousand dollars and might end up with nothing at all so that is why i'm hung up on development now. If i can get it to work perfectly in a simulation, then i don't have to worry so much about designing as I go.

I wish you luck Wayne in all your real world endeavours, i'm gonna live in twillight zone this summer, it ought to be fun and i just might make it out with a camera.

[Rai Orz]

Keith,

you are on a good way, but debayering on a mc/fpga unit? It work, yes, but how good? A bad or poor debayer is like a poor translation software. It translate words, but you dont understand sentences. A poor debayer can produced artefacts and bad colors. And you got only a low resolution. This is what Rob Lohman wrote. But with a good debayer, adjusted at the sensor color filter values, it produced also a high resolution picture. But it is a hard way to write a good debayer and sometimes we god new ideas. Thats why with Drake we do the final debayering in post. For preview, we use a fast, but not the best bebayer.
Keith, for others, please email me.

[Wayne Morellini]

Originally Posted by Keith Wakeham

Even the firewire cameras will not work with nle's because the aren't using any variation of minidv, they are using dcam (IIDC) which is a protocol for uncompressed raw data transport. Same goes for usb.

And beyond that their is the camera issues, but that is just a whole different discussion.

There are NLE software that do RAW/uncompressed capturing, I am not sure which ports they use. One is the freeware Linux Cinelerra from http://www.heroinewarrior.com. A driver could be written to recognise cameralink/GigaE/Firewire (if not already supported) capture and translate the camera format to native internal NLE preferred format, and translate camera controls. This requires a lot of skill/experience and is best handled by an established driver writer (though it might be possible for a good programmers to intelligently glean the best methods from the existing code (still difficult).

The sumix altasens based camera is going through testing at the end of this month, so maybe in a couple of months they might ship a couple and then we could have a camera that could be easily put together and rival stuff like kinetta.

You realise that this is probably going to be an cheap Altasens programmable head with compression, makes me wonder if hard drives and preview monitor canbe attached to it with the camera micro-controller replacing the computer.

[Keith Wakeham]

I was just expressing that IIDC is not a widely reconized format for raw over firewire and most nle's won't see the camera. With programming anything is possible.

What sumix has told me is uncompressed via Gige and if people are really interested they might do compression in the head, but they seamed very doubtful and more that Gige would be standard and completely necessary.

I barely have some timing code written for the fpga yet and i'm not sure if i'm going to even use it. My fpga skills are not what i need yet so i need more practice and to get my spartan 3 development kit (I can't get a price on virtex II dev kits, so i'm doing coding for spartan). So I'm not even 100% sure how i want to get the data to disk, and i understand completly that the quality of the picture is related to the debayer algorithm.

It's like i'm almost back to square one on the planning.

[Keith Wakeham]

Well, some interesting stuff is going on, but suffice to say I started this project because i was interested in building a camera and possibly using it for a workterm for my engineering faculty.

Well, I got a workterm today, (assuming when the company does the numbers they can pay me)

This by no means is a negative to the project, but really a positive, because now i can take the money I'm making working 9-5 and work on the project on evenings and weekends.

Most of my time was taken up lately by finding a job anyway, so progress should proceed as much as it did before. Still paper for now, but at least i have and easier way of funding the project.

I'm in transit right now and i have to settle so not much will be done for a few days

Here is what I'm doing now, i'm programming the spartan for timing of the ccd, and multiple modes (all progressive) at 24, 25, 30 fps. So that is started but I haven't decided about how to get it to storage. Its going to need a preview so some sort of debayer will need to be done in the fpga, so I am working on data flow charts for appending the two outputs together and debayer. I'm not doing high end debayer on chip, only enough for preview, maybe some combine thing so you end up with output of 960x540, so this pretty much means vga, but it might be able to be cropped and stretched to SD.

Well, thats what new with me.

[Keith Wakeham]

I'm looking for a couple of things and hope someone can help

I am looking for a raw bayer from a dslr, doesn't matter as long as i can open it, a tiff or something would be fine, i just need to do some tests

Also, anyone have any smpte documents, like white papers on smpte 292, 274, 296 or any others pertaining to hd, i don't want to buy them at $30 each so if someone has any it would be a great help

Thanx in advance

[Michael Maier]

I have contacted some idustrial camera makers and it seems many come already with capturing applications. Why aren't people using these applications which comes with the cameras to capture rather than trying to right software to capture?
I have found a camera from a japanese supplier which looks very interesting. It's a color 2/3" which does 25fps at 1080 lines and it's a firewire camera. It also comes with a capture application. It has Global shutter too. I think the Global is the best and the way to go rather than rolloing, right?

[Joshua Starnes]

Originally Posted by Michael Maier

I have contacted some idustrial camera makers and it seems many come already with capturing applications. Why aren't people using these applications which comes with the cameras to capture rather than trying to right software to capture?

Because the stuff that comes with the cameras doesn't really work for a cinema application. There's no good way to capture the image information in a system that an editor will understand, at a consistent speed. It has to be created from the ground up.

[Michael Maier]

I was more talking about capturing the footage to a HDD. After it's in a drive, then you would have to find a way to transfer it to a NLE for editing I guess. But I was talking about the camera capturing part. Sorry for the confusion.

[Wolfgang Neun]

Originally Posted by Michael Maier

I have found a camera from a japanese supplier which looks very interesting. It's a color 2/3" which does 25fps at 1080 lines and it's a firewire camera. It also comes with a capture application. It has Global shutter too. I think the Global is the best and the way to go rather than rolloing, right?

Michael,
Will you please give us a link to this camera?

[Michael Maier]

Unfortunately no link. Just a phone number. They would actually custom make a camera to specifications. I'm currently trying to gather information as what would be the perfect camera for a DIY HD cine camera. Then I will contact him and see if it is possible and how much it would cost.