We are making an assumption, that the XL2 has 960x720 chips to begin with....but if it does, yes.

The info is coming...i am working on this pretty much full time, it's just been an incredible amount of work. There is a lot of stuff being worked on in parallel, website, software, hardware, circuit board layout, etc. All updates will be posted here.

Thanks for your patience... :)

PS:It now has an LCD + Menu buttons!

"PS:It now has an LCD + Menu buttons!"

PS: You now are a total badass!

Can't wait to see it in action.

-Luis

Question: If the footage is captured in raw format, but I want the end result to be 24p, would I then put the footage through Magic Bullet or equivalent? How would I go about this and retain the 24p look of the DVX with the raw footage?

Eduardo,

Juans Mod is taking the images straight off the chips.
The chips are shooting at 24 frames per second, so you are getting the RAW files at 24 frames per second.

There is no need to use magic bullet or anything else.

In fact, this is a cleaner way to get the footage.
There is no pulldown to remove or anything.

It is true 24 frame per second progressive capturing.

-Luis

Eduardo:

The DVX's CCDs can capture in progressive mode. The only reason why you have to know about pulldowns, etc. is because of the DV format. The DV format was designed to encode 60i, so in order to put the 24 progressive frames onto a DV tape they have to come up with some ingenious method, while still complying to the DV standard.

With my device, you do not have to worry about standards. If the DVX is in either 24P or 24PA, you get 24 complete frames a second. If the camera is in 30p, you get 30 complete frames a second.

What you get is similar to just scanning a film strip. If you record for a second, you endup with 24 uncompressed image files(TIFF,BMP,etc). You get true 24P, uncompressed, at 36-bit RGB 773x494 resolution with 10 F-stops of dynamic range.

Hope this helps!
Juan

Juan, I'm probably more excited about the range than anything else. That is fantastic, 10 f-stops. It's too bad that a practical homemade DOF adapter hasn't been created for the DVX yet. I hope that will be next.
Best- Frank

Wow. That's music to my ears. Thanks for your quick response and all you do.



es

Hello.

I've not posted into this forum for quite a lot of time and just come by to see what is happening. Glad things go straight forward and glad also Juan that you gained a lot of knowledge on digital videography and that you do well.

I have just one suggestion. You go straight into commercial implementation of it and that is obviously ok. But if you intend to earn money on this project I think it should be vital to make all of the RAW benefits available. Of course, one part of it is proper hardware - and that is what you are doing now. But on the other hand in my opinion you should put stress on that part, that will miss (and very good that will) in your equipment - the bypassed colour correction done in the camera, as this stage is obviously just vital to obtain final good-looking footage.

None of us is a professional colourist, and concerning that the image-processing module in dvx does a very good job (at least in terms of DV), the most vital thing for now is the proper handling of the RAW 12 bit image and finding a good way to proper downsampling it to standard 8 bit. I think this is the right time to start thinking of it as a second crucial part of your project, if I may suggest, apart from the hardware. I'm afraid of the situation that the final user will have perfect RAW footage and from some reasons be unable to correctly process it, making all the work of bypassing DV not so reasonable.

I think you could just hire somebody to write a short manual how to handle this properly or just prepare some presets for most popular applications for handling the footage. Maybe you could take advantage of the DVX color management menu, and base some solututions on it?

And, finally, maybe this is right time to establish an official web site of the project?

regards and good luck

Milosz:

I think i should post some images after i learned that the WB setting affects the CCD output. I am getting images that look great, even without color correction.

Another point to consider, and i'd like everyone else's opinion on this, is that i was under the impression that anyone who is concerned about getting RAW uncompressed 36-bit 4:4:4 out of the DVX, already does their own color correction. Am i wrong? The whole point of getting RAW out of the camera is such that you have the most pristine data to start with the post process...? I could write software that puts a predetermined film-look on the footage and has some parameters the user can modify, but honestly i think that would be wasted time.

I don't know, is there anyone here who plans to just use RAW footage as is and doesn't already do his/her own color correction?

Juan

Well...maybe just a *couple* presets might be nice.... : )


but i'll take what i can get, no complaints...

I think that those of us who dream of sending you a Canon XL2 and getting back a baby Viper are well aware of the hardware, software, talent and workflow considerations of the type of files your mod will be handing off to the hard drive. And we silently pray each night that you don’t go on vacation or otherwise become distracted from your task at hand.

However I do respect that some are looking for a 4:4:4 version of what their gear already pumps out. Here is where the tutorials and faq on your website comes into play and I know that you’re already working on that.

If you need any help on the site many of us would jump at the opportunity to contribute.

Gordon

I would second Gordon's post.
Most of us looking for this type of mod are well aware of what will be required in post.

I look forward to the RAW files.

however, if you still plan on implementing SDI out for 10 bit 4:2:2, that would be incredibly welcomed as well. It woudl be nice to have the option to out to a deck every now and again.

Overall, seeing as the white balance is still intact, I don't see what other camera functions I would want access to. THe less image manipulation the better.

I would also second Gordon's offer to help out. Count me in if there is any way I can help.

-Luis

PS.
Gordon, your mention of a "baby Viper" made me drool.
:)

Ok, that makes sense...

Actually i could use your help with this: import some frames of a static scene from your DVX in normal gamma mode and cinelook mode. Then, see what color correction and most importantly, gamma curve you have to apply to the normal footage to get it to resemble the cine look footage. This will allow me to program corrections that resemble the look that we expect from the DVX....

In any case, I think Milosz's idea of posting tutorials on the webpage is a great one.

Juan

Juan,

John Beale did a DVX100 Gamma Comparison Chart between the Normal and Cine-Like gamma settings.

http://www.bealecorner.com/dvx100/color/

And Michael Bergeron at Panasonic Broadcast also did a document on this.

http://66.102.7.104/search?q=cache:k7wpMUn6kLUJ:www.hpaonline.com/files/public/Begeron.doc+DVX100+Gamma+Curve&hl=en

Gordon

Juan, how many manual adjustments to the image(skin detail, chroma level, mast pedestal, etc.) can u still do with the DVX in camera?

The idea of being able to manipulate with total autonomy in RAW format is almost surreal! As long as the image is clear, without artifacts, and having a latitude of 10 f-stops, I sincerely think that even the way we light a set will change significantly. Can you flash foward into the future where cameras will be able to capture latitude similiar to the human eye? It's kind of sad because I think it will mark the end of a lot of established cinematography techniques in favor for post. But even now, emulsions from film are treated and altered. Maybe a lot of DPs want a film stock that just displays a pristine image that they can bleach-bypass for example and play with rather than a stock that has inherent qualities that make it unique. So as far as software for a more filmic look is concerned, I do not think it is a necessity when going RAW. Look at the digital SLR the SIGMA S10. It captures RAW format with a Foveon chip and there are no "in-camera" adjustments such as sharpness or contrast.
Best- Frank

Mark:

So far the only controls that affect the raw output are all the necessary optical controls(focus, zoom, exposure, shutter, etc) and the White Balance adjustments.

I understand why the WB affects it now, since it alters slightly what the 'white' level out of each CCD is to match the white in the image.

Here is an example of the colors I am getting out of the camera, now that I am doing a WB adjustment. The only correction I did was to apply a lazy S gamma curve to give it more contrast, but that's it. As you can see, the color balance is pretty much perfect.

http://expert.cc.purdue.edu/~pertierr/cap15_small.tif

Juan, that looks freakin amazing. Blown up to 1280x720, that's definitely passable as HD... You should shoot another resolution chart -- this looks better than ever. Make sure you've got enough light, fill the frame, and give us a nice clean res chart. This thing looks like it's going past 800 lines...

- ben

Really,really good results!!!!
I work recently in a feature film made with this camera (unmodified I mean :) ) and yes, its quality is amazing, I can't understand why some people still say that Sony's camcorders have the best image quality.
Juan, are you Mexican?
In my heart I'm still waiting to see what you could do with the HD Camera project we are trying to push at the other threads :D.

Juan,

the color looks fantastic.
If you get a chance, I would love to see a resolution chart as
well, and possibly a clip chart if you can get your hands on one.

Do you still want us to do the gamma vs. normal curve
comparison? I know Gordon posted some links for you, but I
wasn't sure if they answered your questions. Just let us know
if you need it.

Although, my 2 cents would be, why manipulate the gamma curve
on capture just to mimic something we can do easily enough in
post?

I take it that all the gamma settings on the DVX menu are
disabled through the mod? That shouldn't be an issue for most
people I think.


Lastly, I can't help but ask, is that a large urine sample on top of
your TV or what?
:)

jk

-Luis

Quote:

Frank Roberts Said:

"Can you flash foward into the future where cameras will be able to capture latitude similiar to the human eye?"


Erm.., the humam eye is a vastly complex organ, and the eye and brain (Vision Centre) operate togther to "percieve" images, and does raise philosopcal questions that have troubled man from the dawn of history ---------"To Whom does the eye inside belong"

But that aside as a crude compasion the resolution of the eye is aprrox 72 Mega Pixel

from http://clarkvision.com/imagedetail/eye-resolution.html

Human eyes have about a 1,000,000,000:1 absolute range from fully adapted dark vision to fully adapted snow conditions at noon on the equator. But when we are adapted to a normal working range we see about a 30,000:1 range of brightness values.

from http://www.digitalsecrets.net/secrets/DynamicRanger.html

Juan,

The picture really made me want to see a moving clip.
Is there any chance of posting a 5 - 10 second clip?

Thanks,

Luis

Sure, i'll take one tonight.

Note that the output is still 10-bit RGB because my software is flaking out on me :P. I'll also have to make the frame size small like with this last frame to save space, my account is almost full.

Can't imagine what it will be like at 12 bit.
I just put the frame grap into after effects to even out the gamma curve (seeing as it seems so dark initially).

I was able to pull all the detail out the shadows without any visible noise addition.

The incredible thing to me was, when viewing such a shot on the histogram, that after pushing it as much as I did, I saw no signs of banding.

Usually, with 4:1:1 DV footage, you would see large gaps of missing information on the histogram after pushing the levels to much.

I can't wait to see a sample clip.

Very exciting.
Keep it up Juan,

-Luis

Juan, when you do the clip, any chance the footage could involve panning in a really bright, contrasty area? Just curious. Best- Frank

Sure, that last frame was taken indoors with two work lights. It's not optimal lighting, but it gets relatively high contrast images.

And luis, no that is not a urine sample <g> that's where I keep all the DVX screws and small parts while i'm working on it :)

Juan,

Do you happen to remember what fstop you were at when you took that still?

And what kind of wattage are those worklights?

I'm trying to get an idea of the sensitivity of these chips, and what kind of light we'll need compared to shooting DV with the DVX.

Thanks,

Luis

Sorry to post again so quick.

I was looking over the still frame again (still shocked as to how well it reacts to some color work) and it looks like you might still have some noisy pixels.

I see a small group of them above your laptop monitor, near the top in the center of the frame. They weren't completely apparent at first, but as I played with the hue in the frame, they became pretty obvious.

-Luis


EDIT:

I realize now that the easiest way to see the pixels is to turn the color saturation all the way down, making the image black and white.

You should see the small group of black pixels against the white wall above the laptop in the photo.

I just found this thread late last night and read the whole thing. This is great! The results are amazing! I thought it wouldn't be possible from disscusions that started when the DVX came out. Then when I thought I heard Jan say it only did 4:1:1 off the DSP I lost all hope. Guess I heard wrong.

I am really excited at all the possibilites of the new DVX filmstream. If those errant pixel problems are taken care of, it will be perfect! 2:35.1 will look so much better too.

Now I've started my plans to build a box that will hold support rails, a hard drive, PCB and an adaptor to power it from another Panasonic battery.

Luis,

My software was bugging out so that frame was captured using the old method, that's why there are some speckles present...i didn't notice them when i captured it though...

Juan

That's good to hear. For a minute I thought you were back to
square one.

I didn't notice them right away either, probably because they
were up against the white wall. I've scrutinized the frame, and
haven't found any other speckles in there.


One other small request. When you have a moment (which I'm
sure you don't have many) could you give us a quick summary of
the different capture options that will be available. So many have
been batted around, I just want to make sure I know which are
staying in the design and which are going.

(i.e. will SDI still be implemented? etc.)

I think a quick list would help a lot of people.
In fact, we may want to start it out in a new thread, seeing as
the camera prototype is now completed. Perhaps the new
thread could cover the contruction of the box, and capture of the
footage.

This thread seems to be getting kind of hard to search, and it
seems people (myself included) keep asking questions that have
already been covered.


-Luis

Juan,

About 3 weeks ago, you said that you were going to redo the 35mm to DV to RAW comparison. Any progress on that?

Speaking of questions that may or may not have already been asked heres one I think is VERY important for this project.

Can we see screen grabs both from a normal DVX100 and a uncompressed DVX100 AFTER its been thru DVD's standard MPEG2 compression?

Dont get me wrong, Im a perfectionist so uncompressed screen grabs are great to look at but what I would really be interested in is see what the difference is THE REAL world application that most of use will use it for - DVD.

Film outs are totally different story and no doubt uncompressed footage would look much better...but Im also a realist and know most people arent doing film outs all the time. All numbers and specs aside Im very interested in seeing what is can do for 99.9% of our projects that are going to DVD.

Brett,

I don't know that that test is really necessary.
Most people would already agree that uncompressed 10 bit
footage looks better going to MPG2 than DV footage does. I've
done both and I can tell you that DV looks worse everytime. Of
course it all depends on the compression program you are using,
and the skill of the person doing the encoding, but you'd be hard
pressed to get a cleaner encoding from DV.

You can try it yourself with the stills that have already been
posted from Juan.

There is a reason most people master onto D1, or Digibeta at the
lowest end, when encoding movies shot on 35mm for DVD.

Either way, if Juan posts a 5-10 second clip in the near future, as
he said he would, perhaps he can also roll a tape in the camera
so we can compare the DV and uncompressed footage. That
would be good to test how the footage stands up to the entire
post process. I guess it can't hurt.

-Luis

Brett Erskine said:

"Can we see screen grabs both from a normal DVX100 and a uncompressed DVX100 AFTER its been thru DVD's standard MPEG2 compression?"

If you only apply MPEG2 compression onto the uncompressed DVX100 frame and not the compressed frame, then it wouldn't be a fair comparison.

But it doesn't matter anyways, because the uncompressed version will look better no matter what, since it has only been through one compression (MPEG2) rather than two compressions (DV and MPEG2)

This is why it's better if we see the actual uncompressed frames.

Okay, some quick questions and then some hypotheticals.

I looked over the boards quickly and couldn't find this asked before, but am I correct in assuming, since your only taping video straight from the chip, there is no audio included in the mod, thenceforth you will always have to record to MiniDV/Firwire and resynce the PCM audio from that, into your Mod-Captured footage?

Also, I was reading some recent posts concerning loosing the in-camera effects, which is part of what concerned me previously, though it's great to hear the white balancing is still intact. I've been recently pondering over this in regards to the XL2, seeing all those neato in camera features in posses. One thing I've considered is you could always keep a record of all your settings and/or record to MiniDV for comparison.. to help your match any in-camera settings you want to replicate in post. It's not the most convenient thing in the world... although not too high a price to pay for uncompressed 4:4:4. Oh and that also brings up my question.. what activates the recording process on the mod, assuming it's separate from the cameras record activation? And if separate, can the two be synced?

And lastly, to appease my hypothetical brain... if you –could- tap into the pre-DV YUV signal and say.. put it on a wire or whatnot.. then you had a method to convert it to RGB.. say like a standard DVI output... your mod then could be applied from there, correct?

Hey Juan, I haven't posted in a while, but I'm still keenly interested in your mod. Looking forward to your clips =)

To answer Michael's questions:

1.Correct, there is no audio handled in the mod.(I gotta stop calling this a mod!) But it is trivial to simply make sure the DV tape is going, and then match the DV audio to the RAW frames. Remember, they are the exact same frames, so you don't even need a snap board(or whatever it's called), just any kind of quick motion will allow you to find two matching frames and synch them. Then delete the DV video and you are left with the audio.

2.Loosing the in-camera processing is the (LOW) price to pay for pristine data straight from the A/D's. Same thing with the Viper.

3.Not sure what you mean, but we wouldn't want to tap into the YUV data on the DVX, because it is lower-precision and a stage after the clean RGB data. My device does have a DVI output for monitoring purposes.

As far as the MPEG-2 compression comparison, if you are interested you can just grab any of the DV-RAW comparisons I've posted on this thread and compress BOTH with your favorite MPEG-2 compressor.

Like it has been said by Kin, the correct comparison is to compress both frames.

I'll try and take that clip tonight! :)

I dont want to buy a camera and then loose all the features that I paid for, 4:2:2 uncompressed is still better than DV and its better than just dropping samples. 4:4:4 uncompressed may give film-like latitude but it also means that postproduction will take longer and it will take up more disk space.

Filming 4:4:4 uncompressed is only an option.

Yea, it's true that you're going to lose some features and in-camera effects, but you can always shoot in regular DV if you want.

If I had a camera with potentials like that, I'd kick myself for not using it to the max. (even if it means I lose a few lousy in-cam effects)

As far as editing cpu speed and hard disk space, you could always use the DV tape as an offline edit source that will later be replaced by the uncompressed footage. Of course this would have to be rough edits with no color correction, but would save time and space none the less.

Also, a majority of the camera effects and features could be duplicated in post. Sure that will result in more render time in the edit bay, but sometimes you have to give a little to get a little. Personally, I feel the gain of uncompressed video captured straight to hard disk is far more significant than loosing the in camera features. So you're giving a little to get a whole lot.

Juan has done an excellent job of "unlocking" these amazing abilities of the DVX while at the same time preserving those included stock. Like Kin said, with a camera capable of this much, it would be disappointing to use it for anything less than its highest potential.

This is really amazing stuff, Juan. Would somebody pay this guy? He needs some assistance!!

I wonder whether Panny should put him on the payroll. If this guy had a budget he might be further along.

Rock on bro!

"As far as editing cpu speed and hard disk space, you could always use the DV tape as an offline edit source that will later be replaced by the uncompressed footage. Of course this would have to be rough edits with no color correction, but would save time and space none the less."

Nicholi,
That sounds like a great idea, but what are you keeping the uncompressed footage on? D1 tape?

Usually offline edits are done to save hard drive space in post.
There is no point in doing offline editing if all your source material is captured to hard drive to begin with.

This is why I think the SDI out solution is ideal. Granted, I still want the 4:4:4 uncompressed option, as it would be great for some special effects shots, when compositing or greenscreen is planned. But otherwise, going SDI out into a computer would be much more feasible with long form projects.

-Luis

Yes, you're right that initial hard drive space wouldn't be saved since the information is being recorded directly to hard drive to begin with. However, please correct me if i'm misunderstanding, after the individual .TIF frames are recorded to the camcorder's hard drive, you still have to process those frames into video, thus taking up space on the computer's hard drive. By rough editing with DV first, you will get an idea of what RAW footage isn't necessary and won't have to process or store that "extra" uncompressed video, thus saving both time and drive space.

No matter what process you take, you will always need to capture the DV tape in order to get your audio. I figure its a good idea to take advantage of this less-drive-intensive format since you have to capture it anyway.

Of course, a lot of what I'm saying is entirely dependant on Juan's program and how it functions. If there is a way to pick and choose which .TIF image "clips" to process, then this offline system could work well.

I agree, SDI would be great with long form projects, especially studio shots and chroma keying. You would only keep the acceptable clips and wouldn't have to deal with processing or capturing later. The only downside I see here is having to carry a computer along with the camcorder. Again, this is all about what setup best suits your production.

"I agree, SDI would be great with long form projects, especially studio shots and chroma keying. You would only keep the acceptable clips and wouldn't have to deal with processing or capturing later. The only downside I see here is having to carry a computer along with the camcorder."


Actually, what I was saying was that 4:4:4 12 bit uncompressed would be best for chroma keying. You could record the rest of your footage through SDI if you didn't mind the loss of color information. But, for select shots you would still have the option of capturing 4:4:4.

And, SDI would not necessitate a computer (although I wouldn't mind carrying one around). If you didn't mind the 2:1 compression you could always go out to digibeta.

I realize this still isn't as ideal as uncompressed, but let's remember we're still talking about digibeta quality out of DVX. That is still amazing to me.

This is what I like the best about the modification. It will allow for various options which can fit numerous different production environments and situations. One capture solution obviously won't work for every scenario. Hell, who knows, there may still be times I'd want to use my DVX as simply a DV camera.

It's great to be able to get anything from DV to 4:4:4 uncompressed out of the same camera, and chances are I would use every single option at one point or another.

-luis

I know this isn't related to the dvx100a but going back to the XL2. Even if we can only get 960 x 480 with a mod that is a 2:1 aspect ratio. If we want a 2.35:1 cinematic aspect ratio we now would only need to crop 72 pixels from the resolution. That would be 36 from the top and 36 from the bottom. This gives us 960 x 408 or a 2.35 x 1 aspect ratio. This would be great if we plan on going to film with that type of aspect ratio because we keep most of the resolution without cropping to 720 x 272 like we would with regular 4x3 DV.

Thomas I get the idea of what you're saying but I think there might be a slight problem with your math, er... your formula.

I think that the pixels are not exactly 1x1 but 1x.9 which means that when you shoot in 4:3 mode you're shooting in 480x720 which gives an effective resolution of 480x (720x.9) or 480 x 648 (approx. 640) which reduces way the heck down to 4x3 (or 4:3)

Now, do the same math with 480x960 you get 480x(960x.9) or 480x864 which is 1X 1.8 which is almost 16x9 (16x9 being 480x853 which is 1x 1.77)

So in order to get cinema scope (1x2.35) or cinema flat (1x1.85) you'd have to remove 112 pixels from the top (864/2.35= 368 and 480-368=112) for scope and 13 pixels for cinema flat (864/1.85=467 and 480-467=13)

That being said it's still interesting because the dv format can't hold anymore then 720x480 (or 648x480) so in order for the xl2 to record it on to minidv tape then there must be some sort of algorithm that compresses the footage on to the tape and effectively (we hope) perserves the total image quality. However, if someone were to capture the entire CCD image (or at least the ones that are firing in the xl2's case) without using ANY compression or algorithms, they would be super duper cool, and would hopefully finish up with this silly dvx100 business and get onto working on one for the xl2, before everyone decides to go HDV and we need a RAID array to capture uncompressed information off the chips.

I hope that wasn't too technical, and more so I hope that my math was right, if anyone else want's to correct me please do.

I'll correct you. The business with the DVX100 isn't silly. He's working his tail off and a lot of us appreciate it. I don't own an XL2.

Frank is right, it's not silly, just an attempt at humor, I'd venture to say that there are VERY few of us that own an XL2 right now as well.

Sorry, I think I overreacted. You know what's crazy, it's funny how sometimes we all get so overwhelmed that we can't even sense when someone is just trying to be humorous. Nick, keep up the humor bro, its me who should be corrected. Best- Frank