Mirage Recorder at DVinfo.net

Go Back   DV Info Net > Special Interest Areas > Alternative Imaging Methods

Alternative Imaging Methods
DV Info Net is the birthplace of all 35mm adapters.


Reply
 
Thread Tools Search this Thread
Old March 24th, 2007, 08:56 PM   #1
Major Player
 
Join Date: Mar 2007
Location: Amsterdam The Netherlands
Posts: 200
Mirage Recorder

Hello everyone,

This is the thread about my new Mac OS X software that I am writing to record 16 bit raw bayer data from an IIDC camera. In this case I am planning to use the Pike 210C from www.alliedvisiontec.com.

At this moment I can already see a live image from a stand-in camera, the Fire-i from unibrain, it does not do bayer but I wanted to start cheap.

I use libdc1394 for the camera feed. The 8 bit images are converted to 32 bit floating point using the vImage (accelerated image processing) library. These 32 bit images are then send to OpenGL as a luminance texture map.

I am converting to 32 bit floating point images as OpenGL is extremely slow when working with 16 bit integers. There are also more accelerated functions that can work on 32 bit floating point number than there are for 16 bit integer.

I am now planning on to compress the images before they will be added to the ring buffer. Compressing the images before hand would allow for more room in the ring buffer.

I have not yet decided on the compression algorithm to use. I will probably make a prediction algorithm that uses left and top neighbor pixels. The error values (real - predicted) are then converted using a sort of huffman code.

I have this weird idea of creating a set of static huffman dictionaries based on normal distributions. For each line or block of pixels the average error value is calculated and this average will select the dictionary to use for compression. It would be better to calculate the median, but an average is much faster to calculate (doesn't need a sort operation).

Cheers,
Take Vos
developer of Boom Recorder
Take Vos is offline   Reply With Quote
Old March 25th, 2007, 05:42 PM   #2
Major Player
 
Join Date: Mar 2007
Location: Amsterdam The Netherlands
Posts: 200
Hello everyone,

I just designed and implemented the compression algorithm.
It first predicts the value of a pixel by taking a sort of average of its four neighbor pixels, of the same color, positioned above and to its left.

The actual value is subtracted from the predicted value. When there is not to much high resolution contrast in the image the error values should on average be low.

These values are than written down using the exponential-golomb encoding, this encoding writes down small values in less bits than high values. With my 640x480 camera with lots of noise I mostly get a compression ratio between 1:1 and 1:2.

I've decided against implementing most of the compression pre-processing into OpenGL and write it in straight C. The amount of CPU time used is still within limits, but tight.

Cheers,
Take
Take Vos is offline   Reply With Quote
Old March 29th, 2007, 03:29 PM   #3
Major Player
 
Join Date: Mar 2007
Location: Amsterdam The Netherlands
Posts: 200
Hi again.

I've made the compression algorithm simpler, it now just looks at the pixel two positions (because of bayer) to the left of the current pixel and subtract. I still use exponential-golomb encoding. The cpu usage for 640x480@15fps is now between 10% and 11% for the conversion from RGB to Bayer (for testing with a RGB camera), compression and displaying the video.

I have just created a fragment program in OpenGL that does realtime demosaic with linear interpolation. The linear interpolation is not only used for debayering, but also for getting a smooth image when the image is displayed on the screen using a non-integer zoom factor.

Next step, recording.

Cheers,
Take
Take Vos is offline   Reply With Quote
Old March 30th, 2007, 04:41 AM   #4
Inner Circle
 
Join Date: May 2003
Location: Australia
Posts: 2,761
Congradulations, are there any HD webcams for the mac this would work mon. If you are using the portable or mini, the Intel chipset emaulates many GPU fucntions, next chipset (GM965) hopefully should not.

Good luck.
Wayne Morellini is offline   Reply With Quote
Old March 30th, 2007, 05:52 AM   #5
Major Player
 
Join Date: Mar 2007
Location: Amsterdam The Netherlands
Posts: 200
Hello Wayne,

Thank you, I am pretty happy with the results right now.

The software now is specifically written for the Fire-i camera, which is a 640x480 RGB webcam using the IIDC Firewire protocol.

An HD camera that uses IIDC needs a lot of bandwidth and a Firewire 800 controller is required, which currently is only available on the MacBook Pro.
The sustained transfer rate required for the disks is around 60 MB/sec, this can not be done using a Firewire interface as it is already in use by the camera. Thus a eSATA ExpressCard is need, ExpressCard is only available on the MacBook Pro.

Cheers,
Take
Take Vos is offline   Reply With Quote
Old April 2nd, 2007, 04:17 PM   #6
Major Player
 
Join Date: Jun 2004
Location: Buenos Aires , Argentina
Posts: 444
It sounds to me quite a lot simillar to just how Huffyuv compresses video.
Juan M. M. Fiebelkorn is offline   Reply With Quote
Old April 2nd, 2007, 04:22 PM   #7
Major Player
 
Join Date: Mar 2007
Location: Amsterdam The Netherlands
Posts: 200
Indeed, it isn't exactly rocket science.

I am changing my mind slightly though.
To make it myself easy I am thinking of actually letting my recorder write OpenEXR files directly, probably without compression :-(

Cheers,
Take
Take Vos is offline   Reply With Quote
Old April 8th, 2007, 01:30 PM   #8
Major Player
 
Join Date: Mar 2007
Location: Amsterdam The Netherlands
Posts: 200
Good and bad news.

I've been able to record the images to disk in OpenEXR format. Bad news, OpenEXR is quite slow, for something that should be a simple copy operation it uses a tad to much CPU power.

So it is back to the exponential golomb coding and writing my own type of file.

Cheers,
Take
Take Vos is offline   Reply With Quote
Old April 11th, 2007, 12:27 PM   #9
Major Player
 
Join Date: Mar 2007
Location: Amsterdam The Netherlands
Posts: 200
Hello everyone,

The recorder is working, it is creating my own digital negative format.
I will publish specifications on this file format at release to source code to read and write these files.

Of course I can not yet actually see if it is recording correctly, that is my next task.

Cheers,
Take
Take Vos is offline   Reply With Quote
Old April 11th, 2007, 08:08 PM   #10
Regular Crew
 
Join Date: Jan 2006
Location: Woodstock, Georgia
Posts: 154
Cool, keep posting updates :)
Solomon Chase is offline   Reply With Quote
Old April 25th, 2007, 01:36 PM   #11
Major Player
 
Join Date: Mar 2007
Location: Amsterdam The Netherlands
Posts: 200
Boom Recorder

Hello everyone,

Another update, I have ordered the Pike 210C as I am really interested to see if it will all work.

Also I will build the video recorder into Boom Recorder instead of a separate program. This will allow you to enter metadata, such as scene and take numbers. It will also allow timecode signal via an audio input and a lot more features.

Cheers,
Take
Take Vos is offline   Reply With Quote
Old May 3rd, 2007, 06:40 PM   #12
Major Player
 
Join Date: Mar 2007
Location: Amsterdam The Netherlands
Posts: 200
I received the camera, it took me some time to get a picture from it, but it is working now. I currently have it set up to send quite a large 2.35:1 picture at 14 bits depth.

It seems that the camera by default does not do any compensation as I hoped it would not, so the picture is quite a mess. It should not be much of a problem to get all that sensor unevenness removed.

My current problem is that I had an old camera with a 1/3" CS mount lens, and I am using that lens on this camera. That does not make for a sharp picture without vignetting (although the inside of the lens and the three diaphragm blades are in focus).

As for computer performance, I found out that my compression routine was using quite a lot of CPU time and it did not actually compress the data much (actually it became larger). So instead of Delta-Golomb compression it now simply packs the 14 bits tightly.

Next step is finding a good lens and start working on writing to disk.

Cheers,
Take
Take Vos is offline   Reply With Quote
Old May 8th, 2007, 01:46 PM   #13
Major Player
 
Join Date: Mar 2007
Location: Amsterdam The Netherlands
Posts: 200
An other update.

Boom Recorder can now write the sensor data to disk, I just did a full scale test: 1800 x 750 (1:2.40 ratio), 16bits @ 24fps. I recorded for 9 minutes onto an external SATA disk, smooth as an android's bottom.

I only configured a video ring buffer of 128 MByte (I should put more memory in my computer), but it kept up rather well as the maximum was around 50% filled.

It was also pretty light on the CPU, the computer did not even turn on the fans.

Next step: QuickTime component so that you can natively edit the footage on Final Cut Pro, it must do real-time debayer and compensate for fixed sensor noise. I also have to make an intermediate encoder, so that you can render the footage in Final Cut Pro to a 16 bits depth.

QuickTime does have 16 bit depth capability, but it is not linear, rather it uses a gamma correction of 1.8. I guess Final Cut Pro converts it to linear floating point before doing any calculations. To bad QuickTime does not support linear floating point pixel types.

Cheers,
Take
Take Vos is offline   Reply With Quote
Old May 11th, 2007, 07:53 AM   #14
Major Player
 
Join Date: Mar 2007
Location: Amsterdam The Netherlands
Posts: 200
First light with pictures

Hello everyone,

I finally have a decent lens, it is an zoom lens from an old Minolta 35mm photo camera. I had to do some back focus adjustments (the pike supports this by rotating the C-mount in its body-mount) to get it to focus through zoom. Strangely enough the focus marks on the lens are off.

It zooms between 35 and 70 mm and the iris is between f1/22 - f1/4.5, so it is rather slow.

Anyway I made three screenshots of my application. The pictures are not corrected for sensor anomalies or color. the only corrections applied are demosaic using a simple linear algorithm, gamma correction of 1.8 and resized to the smaller screen size.

The first image is the park from my open window, it was raining outside. f1/11, 35mm, infinite focus.
http://www.vosgames.nl/products/MirageRecorder/park.png

The second image is of some bicycles in the park, same settings.
http://www.vosgames.nl/products/Mira...er/bicycle.png

And everyone likes to see a depth of field shot. F1/4.5, 35mm, focused on the blue perfume sampler around 1 meter away. The green perfume sample on the left is 20 cm nearer and a mobile phone is 20 cm nearer than that. On the right the orange lens converter box is 20 cm further and the small C/S mount lens on the far right is 20 cm further than that.
The background wallpaper is around 3 meters from the lens. The photo camera bag is about 1.5 meters from the lens.
http://www.vosgames.nl/products/MirageRecorder/DOF.png

Cheers,
Take
Take Vos is offline   Reply With Quote
Old May 27th, 2007, 02:02 AM   #15
Major Player
 
Join Date: Mar 2007
Location: Amsterdam The Netherlands
Posts: 200
Hi there,

I have another update for you. The QuickTime component is working, it can read the raw digital negative and demosaic it during playing. It asks for a 16 bit per-component drawable, so with a bit of luck Final Cut Pro can use all the 14 bits from the camera.

I found in the Final Cut Pro manual that you only get fast (extreme) performance when the timeline uses the same codec and codec-settings as the clips that are imported. So I will make this QuickTime component to also be used on the timeline. Footage that needs to be coded by Final Cut Pro will use 48 bit RGB and not bayer, in other word it will silently use other settings without Final Cut Pro knowing about it.

At this time the QuickTime component is rather slow as all the demosaicing and other processing is done in software. The plan is to use OpenGL for processing the footage before it enters Final Cut Pro. With a bit of luck you will be able to edit the footage in real-time.

But the next programming challenge is an application that can make calibration data to be used by the QuickTime component and Mirage Recorder. The calibration data will consist of a dark-image offset and slope (based on the exposure time), and a light image offset and slope, and lastly a pixel repair list.

light-image calibration image can be done with any "white"-light that has enough intensity in each red, green and blue component, as long as the same light is smeared equally over the whole sensor. Some kind of diffuser would be handy, or take the lens off the camera. As the calibration is color agnostic, I can use the color conversion matrix for this camera without change. White balance is then the job of Final Cut Pro (or Color if that works).
Take Vos is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

B&H Photo Video
(866) 521-7381
New York, NY

Z.G.C.
(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > Special Interest Areas > Alternative Imaging Methods

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



Google
 

All times are GMT -6. The time now is 03:31 PM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2017 The Digital Video Information Network