Interest in open-source capture software for Sumix M73
 

Is there any interest out there in using the Sumix SMX-M73 camera for low-budget filmmaking? I have an opportunity to get one of these cameras fairly inexpensively, and I'm trying to decide if it's worth the money and time to add support to the open-source capture software I am developing.

If anyone out there has an M73 and would be interested in beta-testing the software, please let me know. Replying to this post is fine, of course.

Thanks!
Rob

I'm interested... a DIY Silicon Imaging setup?

The website indicates that it shoots at 12fps and 27fps (and more), but no mention of 24fps. Also, there is no mention of lens mounts.

Yes, but not the "real" cinema-quality SI-1920. The SI-1300 and SI-3300 are industrial cameras and have some drawbacks when it comes to cinema. But they are a good deal cheaper than the "real thing."
Yeah, those specs are for the machine vision people, not for cinema. It does support 24 fps, but I have not yet determined how precisely.
It's C-mount, which means you can use Bolex lenses on it.

The software currently supports the SI-1300, but as the initial post mentions, I am also looking at the Sumix camera, which is less capable, but even less expensive as well.

Thanks for your interest! Let me know if you want more details.

Rob -- I'm trying to make a short film with the Sumix M73 as a feasibility exercise. I'm going to start a thread here about it when I have a bit more experience with the camera to report with some credibility. Since I'm no programmer I'm using the supplied software for RAM-recording uncompressed Bayer video to a laptop. Would welcome the opportunity though to use other software options with this camera; happy to be a beta tester.

Sam -- I'm experimenting with different 16:9 frame sizes at the moment, all at 25 fps. You have the freedom to enter your own fps speed (asuming the computer can handle the frame size you set for the fps speed you want). RE: C-mount lenses -- I started a thread here (many pages back now) titled "Call for C-mount lens info". Many of these lenses (Angenieux, Schneider, etc) were used on 16mm movie cameras in the 1960's and 70's and certainly good enough for our purposes today. Because C-mount had been taken up by and continues to be used by the scientifc/industrial/security camera people, you can also get new lenses and 35mm SLR adaptors (to use lenses you may already own), though be warned the field of view will be fairly telephoto even with say a 28mm SLR lens. Also, some security camera lenses are not good enough for high resolution video.

John.

That sounds interesting, John. I'll try to get my hands on an M73 and I'll keep you posted.

25fps is close enough (NLEs can handle it). I'm assuming you'll post footage/grabs when you can.

How hard would it be to add on-board audio? That's the one thing that I don't like about the SI camera. Even though I also record to DAT, it's much more convenient in post.

Good question. I know there is some open-source code out there to do this (Audacity comes to mind), but I've never written code to do that, so I have no idea how hard it would be. I expect it wouldn't be that hard. (Famous last words!)

Actually, "on board" in this case means using the PC's sound card. The configuration of one of these DIY cameras is a PC (desktop or laptop) connected via cable (CameraLink or USB) to a "box" camera. So there is no microphone as part of the camera box itself, unless you made a custom case for it.

BTW, the price difference between these two DIY systems is pretty large -- the SI1300 plus 64-bit frame grabber is probably around $3000, whereas the M73 is $800. There are additional pluses and minuses involved as well, if you're interested. I should probably write this up in the ObscuraCam wiki.

Edit: Fixed a typo.

Rob -- I'm a bit busy at the moment but I'll read your wiki stuff to get up to speed on what you've done with the SI camera.

Sam -- "25fps is close enough" -- sorry, I didn't explain it very well: you can set 24 fps if you want to. I'm in a PAL country and since there's little chance of a film-out for my short film I decided to shoot 25 fps, but there are many options. The two main issues being frame size versus fps speed to stay within the datarate (this is a USB 2.0 camera); bigger frame = slower fps speed. "I'm assuming you'll post footage/grabs when you can" -- yeh, things are a bit mad at the moment at work (and I'm trying to find time to experiemnt with this project), but I'm hoping to get a simple website up along with some notes on my experiences and sample frames; will keep the discussion on threads here in Alt Imaging.

RE: sound with mute cameras -- my current tests involve using a DV camcorder to shoot alongside the M73. I import the DV footage and export a wav sound file, to sync up with a low res Quicktime movie copy of the M73 clips for off-line editing. Perhaps there's a better way, though it's nice having 16-bit stereo sound using something I already own.

Looks like it should be pretty simple. I dug around in Audacity and found that it uses the Portable Audio library.
I'm afraid the wiki is pretty slim and badly organized, John. (For one thing, it's hard to work with -- I'm not impressed with the DotNetNuke wiki.)
The other issue, of course, is the rolling shutter. It seems to me that the best compromise with the M73 is to "bin" it both horizontally and vertically, resulting in a resolution of 1024x576 (16:9) or 1024x768 (4:3) at ~48 fps. By then skipping alternate frames, you get 24 fps, 1/48 second exposure and much reduced rolling shutter artifacts.

Downsides:

• it's only a "nearly-high-def" camera
• 8 bit depth

Upsides:

• 4:4:4
• define your own 10-to-8-bit LUT
• maximum use of sensor area

Rob -- I looked at the your wiki pages. Was wondering if the Bayer recording is a still image sequence or a single video file? The M73 is a 10-bit camera which uses 8-bit for Bayer video; would your software be able to utilise the native 10-bit depth of the camera?
John.

It is a "video" file, but it basically stores a sequence of bayer still frames in a single file. It's a format called IHD that Rob Lohman and I worked up -- basically a simple and efficient "digital negative" format.
Yes, it supports 8-bit, 10-bit and 12-bit depths (though I haven't had the h/w to test 12 bits). It will be configurable to use whatever combination of frame rate, frame format (binning/windowing) and bit depth that you wish to use.

Rob -- thanks for replying so quickly. If you can recode your app for the Sumix then it would dramatically increase the flexibility of this affordable camera. I wondered if your file format was a still sequence because the Wiki mentioned DNG as one of the output options (which I believe is a still image format?). This is interesting to me since I was planing on having my ultimate output to be a tif sequence with a seperate WAV sound file (to take to a facilities house and be layed back to HD tape).

I have 2GB of RAM in my laptop which was bought for RAM recording, so I guess I could set quite a high figure for accumulating frames in the RAM buffer? This might ensure no dropped frames (even for larger frame sizes) despite the laptop HDD only being a 5,400 rpm SATA drive.

You're welcome!
The "capture" app writes only to IHD. I am working on a separate command-line-only "transcoding/processing" app called ObscuraConvert, which currently bayer-filters and generates a series of 16-bit TIFFs from the IHD file. I am hoping to be able to output a series of DNG files in case someone would prefer to bayer-filter the images in a different application.
Yes, absolutely. You can configure it to pre-allocate as much memory as you like; it will accumulate the frames in RAM and write them out to disk as fast as possible. If the hard drive is slow, this should still allow for more recording time than RAM alone.

Just a quick update -- I have ordered the Sumix M73 and will begin adding support as soon as I receive it. In the meantime I have been refactoring the code significantly and have made the following improvements to ObscuraCap:

• Migrating UI to wxWidgets to improve portability.
• Rewrote several sections in assembly/MMX/SSE
• Added support for 3 lossless compression libraries -- miniLZO, libLZF and QuickZ.
• Added some unit tests using CxxTest
• Migrating threads to zthread library to improve portability.
• Beginning to add audio record support using portaudio and libsndfile libraries.

All the code is in the SourceForge project, but it's a bit broken because the migrations are not complete yet.

Rob -- big effort. It's good to know you're out there, working it! I'm still doing my tests to get a feel for the camera (going to write it up for a little website).

Rob, for some time I have been meaning to ask you if you would like to make your app record raw from webcams. There are a umber of models out there that do video, and in times past SD models have been able to record raw with off the shelf software. I had approached a company about getting their software to do this, but I don't like the chances. But the number of companies doing application makes me believe that manufacturers are willing to listen to developers. I also was lining up a deal with a webcam parts manufacturer to become a developer for their webcam design, but the deal went nowhere, for the moment (maybe with an partner I might resurrect, but have other potential commitments that have come up in the meantime).

I don't mention any of this in the Elphel thread, because I don't want to discourage them, they have product that does descent job at a potential cheap price, compared to webcam + portable pc. I think Micron is a bit of a let down for now (they are picking up). It is not descent Ibis5a + external ADC and circuits implementation, or a Altasens, or something else that has over 52db S/N and descent well capacity for latitude. There is probably a few other possible good sensor options out there.

I'm enjoying myself! :-) Hopefully I'll have something for you to try out fairly soon.
I would certainly be interested, if it would be useful for filmmaking. Do you know of any models that currently return raw Bayer data?

I did a couple of Google searches, and discovered that there are a set of Firewire cameras that use the IIDC / DCAM standard. I also found Coriander, a Linux GUI for controlling these cameras. This site lists a number of compatible cameras.

Edit: I found several open-source control libraries: 1394-based DC Control Library for Linux and DSVideoLib for Windows. The ARToolKit project apparently contains a DSVideoLib package as well.

Thanks for the extra links Rob. The old firewire camera chart contains information on a Foveon based machine sensor, I have just emailed the manufacturer to find out about it.

I am unsure which one this is, but there is one listed as 3 Layer:
http://www.net-gmbh.com/e/3LayersCMOS.htm


The list has 3Layer listed under Toshiba Teli though:
http://damien.douxchamps.net/ieee139...ras/search.php

SD Firewire webcams with RAW recording are listed, along with the usual machine vision cameras. The Apple Isight is one, but the cheapest Unibrain's and Irez are also there. I forget which cameras have RAW enabled, under $140 I think, maybe $80. I don't know of any cheap HD firewire cameras yet (maybe Apple will start) the HD cameras are USB, I have not investigated which has RAW ability (something you are likely going to have to be activated after market) because I am otherwise busy for sometime.

The most you could expect out of Raw Webcam is to make footage better then Sanyo HD1 or SR1, or HC1 if a good sensor is available. I had been investigating a possibility to make a camera perform as good as the Sanyo HD1 without Raw for less than a few hundred dollars.

The Astro cam groups even modify cameras, write software and replace sensors/components with better ones for better performance. But I could never get an email back from them, so I just gave up on them.

The problem with finding people to work with is that many people are content to work on a level of "disruptive co-operation" if anything at all.

The market is changing, as prices tumble and HDMI recording is a reality low priced, webcams go HD, video camera corporations are worried about video capabilities in Digital Cameras and Cell Phones (despite what they think, in denial, over at camcorderinfo, the rising ability of sensors will make video, even HD acceptable enough to a standard comparable to Sanyo or current AVCHD cameras, even the microns are effectively the things you would use in low to mid end Digital cameras, and Mobile phones). This is an end of market scenario, it would be best if the video corporations concentrated on much better quality HD for the low end and the next level beyond HD, before mobiles get there. For us, it means that machine vision can be much better quality than the low end, but not much cheaper etc. This limits how much we can practically achieve dollar for dollar over low end HD camera with HDMI recording. So webcams can give a low cost solution similar to the quality you could expect from low end cams with HDMI for much less.

I have other concerns on my platter, even if I become available again, doing it all myself is a low priority.

Now, I am not saying, as I always did, that there is not a market for a sub $5K camera, just that it has to be a lot better quality per dollar now because of the competition. The new Sony V1 with a HDMI uncompressed recording solution will likely deliver a picture so good, most of the time, that it will be good enough for most low end people.

Got email back, this is a Foveon based camera 1/3rd inch, Hitachi based product. It is not available and no more information is available yet. I suspect it won't be cheap, and if it really si restricted to 400Mbps it won't have much of a frame rate in HD, and that firewire spec basically restricts you to 8-bits in RGB, but it is a start.

Forgot to mention: If it is based on recent Foveon X3 design, it might have performance that greatly exceeds the Microns.

I received my Sumix M73 camera today, so I hope to start adding support soon. I have been slowed down a bit though -- we had a death in the family and my kids got food poisoning, at least one stomach bug and possibly strep. It's been a fun week so far.

Rob, it looks more certain now I will be busy on another project for an undetermined amount of time. Please feel free to investigate the webcam thing yourself.

Best wishes on the project, Wayne!

Basic support for M73
 

I have implemented basic (read: very basic) support for the Sumix SMX-M73 camera in ObscuraCap. It's far from perfect, naturally -- I have no idea how accurate the frame rate is, the preview colors are screwy, etc.

Rob -- I've been waiting for better weather to shoot some brighter-looking test clips to illustrate the little website I'm writing which is about my experiences with the Sumix. I do like the uncompressed images it makes. How you getting on with your M73? Anything new with your software?

Regards,
John.

I have had limited time recently to work on it, but I am able to capture images. the biggest problem right now is that I'm not able to open/close the connection properly all the time. Sometimes the system just won't recognize the camera and I have to reboot. Making progress, though.

Rob,
I am intrested in your project. I have looking at SMX73 for an indie film project where this camera will be attached to helmet for first person view.
Your capture program will be a great toll for this kind of projects.
Did anyone try to capture with SMX73 in VirtualDub or StremPix(they have freedownload trilal)?

VirtualDub has exelent Capture utility and is famous for precise audio sinc to video. It has lots of real time info during capture: File size, Frames captured, CPU usage, Data rate, Compression ratio, Frame dropped, VU meter, Histogram, realtime preview. I think is open sourced and it will be maybe easier to try to adopt this or expand it for use with SMX73. If you guys that have this gadget can just try to capture and see if it works. It will be a good info. ( its free, and has no installation, simple drag&drop) BTW: It works with my BMD HD extreme to BMD Mjpeg codec...

Igor -- I believe VirtualDub will only capture an RGB or YUV video source to hard drive (?) which would limit the frame size you can record. To get the best out of a camera like the M73 you would want to capture HD-sized frames, though in RGB or YUV mode large frames would exceed the datarate of USB 2.0, and the hard drive would not be fast enough to capture them at our preferred speeds (such as 24 or 25 fps). What's needed is a capture program which will solve these problems by taking in Bayer video (one-third the size/datarate of RGB), and to record it temporarily to RAM (avoid the hard drive speeds, particularly relevant with laptops: many are usually still 5,400 rpm speed instead of 7,200 rpm found on desktops). The Sumix cameras come supplied with software for RGB/HDD and Bayer/RAM recording, so you can use them straight away. Rob is working on his capture software (originally coded for an SI camera) which may provide more Bayer/RAM recording enhancements for the Sumix cameras. Perhaps large frame RGB/HDD recording is possible with compression, though personally I like the uncompressed look of the images...

John:

The Stream Pix record RAW video, wich I prefer for the reasons that you've mentioned, and also because you can correct the video in post for better tuning, that sometimes when you are recording no are able to do for time constraints, or for not having the adequate equipment to do so on the field.

You can try Stream Pix for free, if you like it...don't forget to e-mail me please...you might actually like it, and it offers FREE LIFETIME UPGRADES!

Thanks,
Cesar Rubio.

John, I also agree with you about uncompressed, but none of those capture apps capture audio ( as I can see ). Cesar, correct me if I am wrong, but I am willing to lose some video quality for good and in sinc audio. It is importnant for my projects.

Igor:

With Stream Pix you DON'T loose any video quality recording in RAW mode, on the contrary you can SAVE all the information that the sensor CCD or CMOS is able to offer, and you can get the BEST video quality in post later on. With SP you also can record audio in sync with your video...give it a try, it's good.

Cesar Rubio.

Igor -- the M73 (like many other box cameras) does not record sound in any way, consequently there is no requirement for the capture software to record sound. My own plan for sound is to record it with a DV camcorder (16-bit stereo) running at the same time -- marry up sound with low res proxy later for off-line editing. This sort of set-up (and the need for a sound synching method) may not be suitable for your project if you need high levels of mobility and informal shooting with long takes.

Cesar -- it's great to see someone championing 3D! You mentioned StreamPix capture software; I looked at this about two years ago but was put off by the cost. It seemed too expensive for no budget projects like mine; has it come down in price lately?

Rob -- can you tell me if you have been able to capture 10-bit images from the M73 with your software? That is an area which really interests me, because it takes the camera even further away from an HDV camcorder.

Happy New Year, everyone.

I believe so, but I've take a few weeks off the project, so I'll have to go back to my notes and verify that. The biggest problem with 10-bit (vs. 8-bit) is that it cuts the bandwidth in half, making the the rolling shutter artifacts very pronounced. From what I can tell, to reduce artifacts enough with the M73 you'd need to stick with 8 bits or use 10 bits with a mechanical shutter.

Just FYI, at some point I am going to try capturing simultaneous audio in ObscuraCap.

Sounds interesting!
I believe the biggest issues are (1) limited to 8 bit depth and (2) we'd have to do realtime Bayer filtering.

Rob -- thanks for the latest info: "To reduce artifacts enough with the M73 you'd need to stick with 8 bits" -- shame, 10-bit would have been the cherry on top; I should be more grateful though for what we do have with 8-bit! Rob, do you have any recording durations yet; I was wondering how much longer shooting duration might be compared to the supplied software? Also interested in your opinion of the deBayer you're using compared to the Laplacian one with the supplied software.

"I am going to try capturing simultaneous audio in ObscuraCap" -- sounds good Rob; perceived wisdom is it's best not to do anything else with the computer since it's probably maxed out taking in the video (and you don't want any background interrupts to that), though if it can be done (instead of using a separate device), that's obviously a good thing for portability. Thanks once again for your effort on this...

With the M73 in 1024x576 (binned) mode, I am expecting to be able to be able to capture indefinitely (or until the drive is full).

Certainly that's true in most cases, but with the sheer CPU power available in newer systems it's worth a try. It can always be turned off if there is a problem with it.

You're welcome! I hope to get some time to make further progress soon.

Rob, I am very glad that you are considering sound in your app and I agree with you that new motherboards have enough power to do that.
Here is some links to a system components that I want to put together:

http://www.spectra.de/produktliste.c...dummy=1#114781

http://www.spectra.de/produktliste.c...dummy=1#113878

http://www.seagate.com/docs/pdf/data...mentus7200.pdf

http://www.presonus.com/firebox.html

Rob, John can you put some footage online where we can see rolling shutter artefacts? How much motion blur disapear when you max settings against rolling shutter artefacts?

Cesar, have you receive footage from AVT Marlin F131C? That little FW thing also look promising and have global shutter with 2/3"cmos, and it is in our "price range"...

Igor:

I am still waiting, like everybody else. The Marlin F-131 has a 1/2 CMOS sensor, what do you mean by "FW"?

Thanks,
Cesar Rubio.

Probably you meant FireWire...