Hey Juan could you capture a still frame of an object in front of a solid blue or solid green background. It could just be a small piece of green or blue paper that is fairly saturated. Maybe you could just put your hand in front of the paper. It would be nice if we could test one of the most important reasons of having a 4:4:4 mod which would be keying. We may also have an easier time to figure out any alignment issues of color channels this way.

If you could just point one light at the object and color background. Shadows shouldn't matter because we will not be trying to get a perfect key. We just need enough to test the edging.

resample
 

Ben,

According to what Jaun proposed earlier about uprezing, he seems to believe that an interpolator such as bicubic or spline could be implemented in his software. Take a look at some of Juan's last posts.

As for image quality loss, I did a resample in photoshop of the green channel using bicubic and my eye didn't notice any quality loss (I only resized by 1 pixel)... what my eye did notice was an improvement in the green channel's croma shift... and I don't know about you, but that lack of alignment is what I consider the poor quality. You gotta give a little to get a little, you know.

re:sample
 

John,

Any quality interpolation will not happen in realtime on modern computers. It's just a mathematical reality -- at 24fps, you're looking at over 7 million pixels a second to pass through the interpolator, and each pixel needs to see a neighborhood of at least 16 pixels.

But maybe Juan has some awesome algorithms he should patent immediately. :)

I would hope that if he did implement interpolation, he'd use Spline rather than Bicubic. Juan, if you want any help with the interpolation code, let me know -- I've got some laying around for really nice 16 and 36 pixel spline interpolation...

- ben

Randall(waaay up there):
I just wanted to make this clear: ALL the raw captures i have posted so far are 10-BIT. Nothing I have posted so far is 12-bit because my test gear doesn't handle the extra 6 bits. So everything i have posted is handicapped in color precision and a little latitude from the final prototype output.

Now, about the interpolation...I think it is best to allow the user to decide, and if Ben you can email some Spline interpolation algrithm that would greatly help. I'm sure it's not that difficult to implement, specially on the PC.

It would get really difficult to do it on an FPGA, specially since it will be running at around 100Mhz or so. I don't think real-time is an option.

About the alignment, my current code from which the clip came out of does not align the channels, right now just puts them on top of each other as they are...it's a matter of three variables but I haven't added that yet.

I did notice however what others have pointed out...that the green channel seems to be offset from one side of the frame to the other, but i think this is due to just the level of manufacturing and alignment of the camera...we might just have to live with this.

Only solution I can think of, is to uprez the R,G,B channels separately and then use the extra resolution to do a fractional alignment in the original image space.

Green/Blue Screen capture: Sure! I just don't have a blue/green screen so what do you guys suggest? Can i get something at lowe's or just get a green piece of construction paper/poster board?

That's a good idea...probably one of the most important tests. I imagine lighting is also very critical for a good key?

Green screen
 

For the last film I did, we used green fluorescent poster board from the local Walgreens drug store for one of our days of pickups when we didn't have the green screen stage anymore. It worked just as well as the stage we worked on with the exception of the seam lines between pieces of boards.

Also, I've looked at it several times, and it's not a fractional offset, but rather a slightly different image proportionally... everything on the left side of the frame is much better aligned than on the right... which tells me that the images themselves have are too different in FOV for them to ever align right... I already tried uprezzing the footage by 400% to do fractional alignment and it did not work. I agree that it's probably just an issue with the manufacturing, but when I did a resizing of the green channel in Photoshop to compensate (enlarged it by 1 pixel), the alignment got noticeably better. I agree that using the algorithm for uprezzing the footage to HD would merely be a perk, but using the algorithm to compensate for this manufacturing defect (not an offset defect but rather a FOV defect) may be a crucial part of getting a good green screen key with the camera. Having the green channel gradually shifting like that across the image will cause problems with green screen, I promise.

I wonder if this problem is also evident in the DV frame i posted? i.e. how does the camera originally deal with it? I'm thinking they just got it as close as possible...

DV footage
 

I just took a look at it... from what I see, there's no difference between any of the three channels in the DV frame. The camera has to be doing something to compensate what's happening in the green channel. I'm gonna keep studying the raw version and the DV version to see if there are more clues as to what's happening.

John

Strange
 

When I place the DV still on a layer above the Raw version, they don't match up at all. The DV version is narrower on its horizontal axis than the raw version. I had to enlarge it by about 106% horizontally for the balcony posts to line up on the two.

Why is that? Could it be a clue? And what does that mean the correct pixel aspect for the RAW footage you're pulling off the chips is? I guess I assumed that it was the same as DVs .9

Am I missing something?

John

Green flourescent posterboard is actually a near-perfect color for greenscreen.

Juan, you shouldn't have to uprez to correct the problem -- you should be able to just use interpolation to do a sub-pixel distort or scale or whatever.

I'll dig up those interpolation functions and send them over with my notes!

- ben

DV vs RAW
 

When I say narrower on the horizontal axis what I mean is it's more compressed horizontally than the RAW version.

John

John,

The chips in the DVX are NTSC so the image will look wide unless you apply the correct NTSC 0.9 pixel aspect ratio.

Also, I know for a fact that there are some columns and maybe horizontal lines which are cropped off by the camera for the DV footage, and what's left is resized.

However, now that i think about it, even if the only difference in the green channel is some optical FOV, we should still be able to match the pixels by doing a transformation...afaik it should be close to linear. We can take samples at extreme points in the image and then adjust, shouldn't be that much more different than a resize if it is linear.

Juan, you can also buy some green paint at Lowes, I used that to paint our cyc wall and it works very well..make sure it's the most pure green they have..

0.9 pixel aspect
 

Juan,

I don't think you understand what I mean. The DV clip is 720 x 480 right?... which will appear a bit wider unless you apply the 0.9 pixel aspect ratio to it. That will reshape it to something proportionate to 640 x 480 square pixel... and thus it will look normal on a monitor... But what I'm seeing is a difference between the the DV frame you uploaded (720 x 480... which has not had the 0.9 applied and is therefore a bit wider than normal, agreed) and the RAW frame (773 x 495... which has not had the 0.9 applied either AND YET is even more "stretched" wide than the DV frame.) Why wider than the DV frame?... the DV frame is stretched wide enough... applying the 0.9 pixel aspect to DV 720x480 will correct it to the normal propotion, but if the RAW frame is even more "stretched" then it needs an even lower pixel aspect than 0.9 to correct itself. Am I crazy... does anyone else understand what I'm seeing?

John

Does anybody know about this new Final Cut Pro HD? It claims 10 bit 4:2:2 uncompressed 1080p at 90-160 MBps via PCI transport . . . and from SOFTWARE alone (No additional hardware)?

I'm not sure I buy this. I will say that if it does what it claims, it would limit the 4:4:4 idea, but sure gives us enough for Hollywood level acquisition.

Somebody tell me what I'm missing here:

http://www.apple.com/finalcutpro

Don't spend a lot of time trying to light the blue/green screen setup. Just point one light at it and the subject. I know it will give harsh shadows on one side but that isn't important right now. We just need to test the side the has no shadows. Besides it would be nice to test how well the mod performs with a very bad keying setup. I assume even with bad lighting a 4:4:4 signal would be much easier to key than a DV signal would. If it was a perfect screen with perfect lighting we might not be able to tell as easily how much better the mod is. Clearly it would be better but we would get the full effect by having a badly lit setup.

If you can't get to the store you could always print out a piece of paper full of a solid 255 green image. This of course would be a huge waste of ink.

John,

Ok, here's what i see. I do understand what you are saying but I am not experiencing the same problem.

The RAW frame DOES cover a wider field than what is on the DV footage. The camera crops that off but we can see that area in the RAW footage, so right away the image is dimensionally wider. However, as for 'stretching', as long as I apply the 0.9 pixel aspect ratio on the RAW footage, it looks fine. I can especially see it on some shots of resolution charts, you can judge by the circles. The circles are ellipses without the pixel aspect ratio correction, and become circles with it applied.

What has everyone else noticed?

John,

I think i found what you are talking about..in the DVComp2, the DV version covers a slightly wider field of view, and i just noticed this...however i'm pretty sure that the camera must've been moved when i pressed the record button, it's a pretty shaky setup.

I'm going to try and take another set without movement.

DV vs Raw
 

Juan, is part of the 773 x 495 image being cropped or masked by the camera in order to fit the 720 x 480 DV dimensions? or is the whole thing being resized? If it's being cropped as I had assumed from previous postings, then while yes it does seem that there's a different field of view in the DV (like you accidentally moved the camera), I wouldn't say the FOV is SLIGHTLY wider. In the DV you see almost an entire plank of wood more than in the raw version. If there was even more area of the DV image that was cropped than that extra plank then that setup must have been shaky indeed... you would have had to really knock the camera in order to change the FOV that much. Anyway, those two clips are too different to compare and analyze correctly. If you could capture another two that would be great. This time hit the record button first and then start the capture. And if you can, it would be great if you could take it off letterbox mode... that would help the comparison process... I want to figure out what's going on in that Green Channel and why the DV footage doesn't seem to have it.

Thanks,
John

Will do...you're right, the only way is if i hit the zoom by mistake a bit...but we all know how touchy the zoom is when it is set to manual.

I will take another outdoor set tomorrow, and hookup the joystick so I can set the letterbox off.

Juan, I figured something out
 

I went back and took a look at that test chart you posted a couple weeks ago... very interesting.

First I scaled the image horizonally to match a 0.90 pixel aspect. And yes the circles appeared more like circles... but then I compared them to a perfect circle I created using the elipse tool while holding down shift. The circles are not perfectly round, they're a bit wider than the perfect circle I made. Of course if you've been looking at the wider version for a while then the 0.9 version certainly looks more natural, but it's still not totally propotionate.

Then I just tried resizing the 773 x 495 image to 640 x 480 (which is the size that a 720 x 480 w/pixel aspect 9.0 applied scales to.) and the circles were perrrrrrrrrfect. There's no doubt that you knocked the camera on the last comparison shots, but there's also no doubt in my mind that the raw image is more stretched out than the DV images. It actually appears that what's happening is that the camera is simply reshaping the 773 x 495 image to 720 x 480 before or durring the DV compression. So there is actually no extra part of the image we're seeing... we're just getting a higher resolution image... which is still fantastic.

Thoughts? Am I missing something? Getting somehting wrong?... I'm no expert, but this is what I'm seeing.

John

Very interesting...so it's sort of an inverse anamorphic adapter? Fits a 4:3 image into a very wide CCD? Interesting trick, probably why the DVX footage looks so sharp.

If this is correct, then it would greatly help to first up-rez the raw data and then resize to the correct aspect ratio...or even better, leave it in the original aspect ratio and interpret the correct aspect ratio at play/render time?

Juan, you definitely don't want to up-rez and then downsize to the correct AR. You always want to do all your transformations in one step. So if you're going to DV, it's best to just do one transformation to DV size. If you're going to HD, you do one transform to that size. As soon as you do two in a row, you've done two interpolations, and you'll start to see generational problems

- ben

capture stills
 

It would actually be better I think if you not do the DV RAW capture comparisons outside like you did last time. Something there in your room is better. Something totally still and with enough detail to do indepth comparison of the channels in both DV and RAW. That outside shot was good for latitude, but if there's even the slightest bit of wind then you'll have to search tirelessly for the exact same two frames on the RAW and DV cause the tree leaves will be different... If you just shoot another Test Chart for example, you can pick any two frames of either and they should be exactly the same... that is if you don't hit the camera again ;)

John

Also
 

Juan,

Can your test grab include some sort of detail very close to the right and left edges of the frame so that I can compare the FOV issue I'm seeing with the green channel. I want to compare the green channel in the RAW with the green channel of the DV.

John

idea on anamorphic adapter
 

juan - do you have access to the anamorphic adapter for the dvx? in these discussions about the stretched image, you said something interesting, about it being "like a reverse anamorphic adapter" - i wonder if this was done intentionally, so that when you use the panasonic anamorphic adapter, it actually matches somehow with the pixel count of the CCD? it seems like the camera was designed from the start to be used with that adapter...

perhaps if you put the anamorphic adapter on, you could have a much higher resolution anamorphic image than the dvx would give you otherwise. it might be worth borrowing (or renting) one? perhaps there's someone in your area that could loan you theirs for a test.

-benjamin

i realized i didn't finish my thought... the pixels on the CCD are square, yes? so the camera, as it processes, is squishing the image horizontally, but perhaps when you have the anamorphic adapter on, the math will work out so you take those square pixels and expand them slightly to make them rectangular ntsc pixels, and we could have a perfect 16:9 camera on top of everything else!

i don't know how to test this, mathematically, without footage shot through an anamorphic adapter though. any thoughts?

-benjamin

there's a math way to figure this out
 

I agree that anamophic was build especially for the DVX, but that's more because of the 72mm diameter thread I'd think.

I'll try to figure it out... my gut says no though.

John

registration problems; HD-SDI; EZhook?
 

Juan,

I hope you can resolve the rgb registration problems.

Adding the anamorphic adapter would be a good idea if that is the ideal way to use this system (and I think it is). As I recall from the literature the Anamorphic adapter is intended to let us use "all the pixels" in the sensor rather than crop to get 16:9.
Unfortunately these adapters cost $700-$900.

HD-SDI transmitter:
Xilinx has some interesting application notes of possible interest.
an HD-SDI transmitter using the virtex II FPGA:

http://direct.xilinx.com/bvdocs/appnotes/xapp680.pdf

I think this might have some data on the HD-SDI spec.

There are a no. of relevant app notes on this site if you plan to use this part. There is also a ref. design for a SD-sdi transmitter.

I would still like to know what you are using to clip on to the altera chip? I would like to experiment with my DVX100. I would also like to try hooking up to the chip in my Sony DXC750 and DXC950 (these camera's make 700 lines resolution interlaced).

I have just finished shooting a feature with the DVX-100 with the anamorphic adapter on it. Will it help you guys at all for me to upload a clip?

SDI info
 

Juan,

You'll find additional info on the SDI spec at:

http://www.xilinx.com/xlnx/xweb/xil_...lication+Notes under SDI.

Anamophic
 

Laurence,

do you have a raw frame grab as DV 720x480? That's all we'd need to see.

John

Well, you're going to have to pardon that I'm not as technically proficient as you guys are. After reading around this site for a bit, I've come to the conclusion that my version of a frame grab may or may not be what you need. When you say you need a RAW capture, I truthfully don't know if I am capable of doing that or not. I recorded the movie footage with the DVX-100 & the anamorphic adapter attached, and the camera mode in 24p advanced. The footage I have was recorded via the DVX onboard recording system (so that's what we're starting with. My method of accessing that footage is a typical dv camera (sony vx-1000) that I can either output a signal via s-vhs or of course standard 1394 small firewire. I bought a non-linear editing system back in the day (6 years ago) from matrox (MATROX DIGISUITE LE) that had s-vhs and BNC component I/O, but no firewire, as firewire was not as popular then (used that system to edit a feature with 16mm film footage transfered to beta-sp). I'm using a PYRO 1394 Drive Kit that says it transfers rates up to 400 Mbits/sec. It came with some cheesey software called SONIC DVD. So I connect the camera to the external Pyro drive via a firewire cable that I assumed was standard small firewire (Hey, but maybe since it says 400 Mbits/sec, it's actually firewire A or B whatever? - - - hope I learned something already). And then I capture with SONIC DVD software. Then maybe send you guys the file here? Is this the right direction or is this useless to you guys? Maybe I can capture with some other software that would be better? Not sure what you guys would recommend. Let me know and I'll be happy to do what I can.

RAW
 

Actually I didn't mean to confuse you. When I said RAW I just meant a frame of the DV as it cam off of the tape and out of the camera through firewire. I realize in a discussion like this one the word RAW can get confusing.

Anyway, when you capture to Sonic DVD is there a way to export a single frame as like a Tiff or PNG or even a JPG.

John

Hey guys,

Sorry for the short 'dissappearance' :)

I'm well on my way working on the prototype, and already have all the FPGA's and programming stuff, along with the other components. I have a design for the casing which I will probably make in my milling machine/lathe but that comes later.

I DID find a green posterboard but I left it at work today. So, i'm going to hook everthing back up and do some tests using DV tape and RAW capture of a resolution chart and maybe some other indoor shots.

I'll make sure that i don't bump the camera this time.

Also, to answer Benjamin's question, i am almost 100% that the pixels in the CCD's are NTSC, (or some other aspect ratio) but not square. This is because the RAW images clearly look stretched when viewed with square pixels on a computer.

I do not have an anamorphic adapter, and even though it is something that i've always wanted to purchase i can't really do it right now since i'm working on this project. But i guess if anyone lives somewhere in/adjacent to Indiana and wants to help that might work. :) I drive all over the state all the time anyway.

Juan

Another quick question...

About the SDI implementation, what use could it have? If you want to use it, you're going to need a computer anyway, in which case you can just plug-in a FW800 cable and watch the preview/capture with that right?

In either case, the video coming out the SDI port would not be the full RAW data.

I'm just curious if this would really be a useful feature. As for a monitor, analog S-video seems to be the best option, or do most people actually use SDI monitors???

Analog
 

I think analog is a better choice for a monitor.

John

I can second that. Analogue S-video monitors are much more commonly used than the ultra-expensive SDI or component monitors.

<<<-- Originally posted by Juan P. Pertierra : Another quick question...

About the SDI implementation, what use could it have? If you want to use it, you're going to need a computer anyway, in which case you can just plug-in a FW800 cable and watch the preview/capture with that right?

In either case, the video coming out the SDI port would not be the full RAW data.

I'm just curious if this would really be a useful feature. As for a monitor, analog S-video seems to be the best option, or do most people actually use SDI monitors??? -->>>

Randall Larsen's reply:

AJA makes the Kona and Kona2 cards that allow input of SDI (and HD-SDI) to the MAC. SDI is also a standard transport protocol in current TV and EFX facilities.

SDI monitors are common in high scale facilities and on high budget video shoots. The only possible objection to S-video is cable length. How long can an S-video cable be? Do we ever need to feed a video village from your mod?

I think firewire 800 is fine. I think gigabit ethernet would be even better--gigabit ethernet cards are cheap for PCs.

If you ever need to run a 300 meter cable to a computer for capture UTP Cat 5 or Cat 6 cable would be a lot cheaper than a long S-video or firewire cable.

MAC G4 and G5's already have gigabit ethernet. However, sounds like you've already committed to Firewire 800 so thats OK. Chances are there will be new parts available if you later do a mod for the new HD prosumer cameras that are coming out. I am just worried Firewire 800 won't have enough bandwidth at higher resolutions (it should be fine for the DVX100).

EBU has some standards and tech references on line. Some of these are similar to SMPTE standards that one has to pay to access. This does explain TRS and a few things FOR SDI (625) see:

http://www.ebu.ch/CMSimages/en/tec_d...tcm6-10543.pdf

this is accessed from the tech ref home page 3000 series docs.

Juan, if we are talking about monitoring for reference only, you should have analogue out capabilities (meaning not SDI out to an SDI monitor which costs more than the DVX100). Do you have to disable the existing analogue-out on the camera or something? Why not just keep that intact? For reference only, there's no need for 4:4:4.

Nope, unless something comes up, the DVX will work completely as normal, and all outputs including the S-video on the camera will still work. I'm introducing some D/A circuitry to provide an s-video output.

If I have enough gate elements and output pins left, i might be able to implement SDI anyway, since the FPGA i'm using can handle it. The S-video is the first priority though.

I'm also thinking about making it scalable or different versions, one with Gigabit ethernet and the other with Firewire800, depending on what the buyer wants.

Juan