|
Obin's right -- I think there's only one person here still invested in this discussion, so maybe he can continue it with himself...
|
I think it's unfortunate that Ben chose personal attacks over healthy discussion of the issues.
I realize Juan is hard at work on getting this to work - I've been in this thread from the start. However, I still think the drive/recording issue is a vital issue to the success of this unit, especially since it should be able to work on other cameras and a reliable field recording setup will prevent it from being something that requires a bunch gear ro record with. However, since the vocal opposition is to drop it, I will. I also will remove all the drive discussion from the thread between Ben and I if Juan wants to us to clean it up to make the thread easier to follow. |
Stephen, I didn't mean for any of my comments to seem like personal attacks; I was indeed trying to have a healthy discussion of the issues, however I didn't understand the context you were discussing SATA II in, and you didn't make that context clear until your last couple of messages. That was frustrating.
I (like a few others here) mistakenly thought we had all moved past the idea of a dedicated drive and tether, when clearly this is still a big issue for you. If I had understood that from the beginning, I would have politely ignored you. Instead, I thought you were arguing for a SATA II port on Juan's box, which is neither what you wanted nor a good idea. I apologize that I became frustrated -- it's clear now that we weren't even arguing about the same thing... - ben |
Ben:
That's one problem with this thread - it's so long and has been going on for so many months it's hard to keep all the tangents up to speed. The Agus35 suffers from the same issues. I would like to see some type of branching thread for these heavy topics here - I'll mention something to Chris as it would make it far easier for people coming into long-standing threads easier. Anyway, no hard feelings and sorry for the confusion. |
No hard feelings here either, and my apologies.
Here's an idea: could we make this topic its own forum under the "Special Interest Areas" section? |
Its probably impossible to do but I was just wondering. Would you be able to make a memory card reader to make custom gamma curves for the DVX100? If so, it would be great if you make one after ur done with this project.
Sorry for being a little off topic. |
Stephen, Ben, and the rest <g>:
Regardless of the thread getting a little 'heated', i think just about every post contains useful information and good , well informed opinions, even if we feel passionate about our points. I wouldn't expect any of us not to. :) To answer Stephen's question, yes, the start/stop can be handled over firewire, just like if the capture box circuitry where a PC itself. The circuitry in the box accesses the drive just like a PC and when the recording button is pushed, starts writing files to the drive. However, i think there's a point we are overlooking here that eliminates SATA directly, and that is that right now the FW800 port on the device can also be connected to a computer directly, for recording on the machine's hard drive. This will be done through a device driver on the machine side, which will allow the camera to access the drive on the PC. This is simply not an option with SATA because it is simply not as flexible as firewire. I can write a device driver on the PC side to do just about anything, but if we go straight to SATA, all we can do is hook directly to hard drives. SATA is definitely a better hard drive interface, because that's what it is designed to do..however FW800 is not primarily a hard drive interface, but rather a flexible interface that can be used for just about anything...since this app involves more than just writing to a drive, firewire is a good choice. Let's hear it! :) Juan |
Quote:
For this thread it should be "4:4:4 12-bit Uncompressed DVX100 - Part II" -Rodger |
There's no official moderator here, so I've been holding off, though Chris had talked about me doing it here. I sent him an email earlier.
|
<<<-- Originally posted by Stephen van Vuuren : There's no official moderator here, so I've been holding off, though Chris had talked about me doing it here. I sent him an email earlier. -->>>
So I see... Make sure to link to the original thread on the opening post if you guys do this. We have a few threads for our products on some different boards that went over 2000 replies if I remember right. It also helps indexing and search engines if you limit thread sizes. Setting an arbitrary post number seems to work well. -Rodger |
Hi,
fist I have to say thankyou to Juan for his brilliant virtuoso work on this project, which is not only very exciting but is also giving all of us a rare glimpse 'behind the curtain' on our camcorders. What we pay for is not always what we 'get' with these units... As I understand it, the normal lattitude rating on prosumer camcorders is only a few stops (5-8?) with the higher end cameras pushing 8-11 stops (correct me if I'm wrong on the numbers here). But it's apparent that the latitude coming off the DVX100 chips is far greater than anything that we've been accustomed to in the past with our cameras. Does anyone know how to measure this difference in range? Even an educated estimation would be enlightening since it would give us a better indication of what our cameras are capable of, rather than what we're used to getting from them. Again, great work Juan. Best, DW |
One of the tests I am planning to do today or tomorrow, is to take outdoor and indoor snapshots of the same scene with my 35mm camera, and the DVX raw. This will give us a better idea of what the latitude is and i for one can't wait to play withe footage and see if I can get it anywhere close to the look of film.
I am going with ISO400 film because i read somewhere this is what most CCD's correspond to roughly... I will have the film images scanned off from the negatives at 2K resolution and put on CD at the lab so i can post them... Juan |
Juan,
Most people seem to agree that the DVX100 is roughly the equivelant of 320ASA (when shooting at 24P with 0db and a 1/48 shutter). http://www.dv.com/jive3/thread.jspa?...000694&start=0 http://www.dvxuser.com/cgi-bin/DVX/Y...num=1068605987 I did see one post on Dvinfo which claimed a 640ASA equivelance, but the majority of posts claimed 320. I think the 640ASA may be true when shooting at 60i with a 1/60th shutter speed, which is not the case here. I realize you can't get 320ASA film for your still camera, but 400 should be close enough. -Luis |
latitude
Juan,
Did you ever figure out why the darker areas of the DV clip you uploaded are considerably brighter than the darker areas of your raw capture? The explanation about the distribution of 24bits across 8bit codec doesn't seem to explain why those areas would have to be brightened by the codec as well. Only why information in the brightest areas of the frame might be sacrificed. It would make more sense that panasonic brightens the footage at the same stage that color correction functions occur in the camera in order to help out overall with low lighting. I don't know... I may be way off... but if you captured a raw clip and then captued the same thing with a ND filter on enough to match the darks and midtones of your raw footage, we'd get a better idea of exactly how much information in that sky is actually being lost... please explain what I might be missing if you can. John |
John,
All DV/RAW comparisons where frames shot under the exact same conditions, so you can see what the DV output for the specific RAW data is. I'm not sure about the darks either, but do note that the adjustment from 12-bits to 8-bits is not the only correction done by the camera. There are tons of other modifications done to the video which could be causing this...it could also be my settings on the camera which i haven't checked in a while since the joystick is disconnected. It is possible, however, that since the color is not paletted, they moved the entire limited 8-bit range (which doesn't cover the entire latitude) to somewhere in the middle, sacrificing a little low end, a lot of the high end, but compressing the 8-bits in the brightnesses that they thought you would work under...this would give more color precision in that specific range, while sacrificing high end and low end latitude. Just a theory... Juan |
Oh, and an update:
I started on the code work yesterday, but finally gave in to a couple of days without sleep <g> However, i did snap pictures of the outdoor shot with my 35mm camera and the DVX at the same time...i got em developmed but even though i was using the on-camera spot meter, they came out a bit over exposed, and to all honesty the DVX captured a way better scene but is not a good measure. I will try to do it again today or tomorrow, and try some slightly underexposed shots by the camera's spot meter. I do have a couple of indoor shots of my lamp which makes a good test of the range in brightness, so I am going to take some shots with the DVX of that as well, and see what it looks like... Juan |
Color Corrected
I'd like to post my try of the colorcorrected cap9_RAW:
http://www.filmerforum-koeln.de/user...cap9_RAWcc.tif the colors now seem more like the DV (although - of course - more natural) und I also lightend it up a bit... I'd LOVE to see your 35mm / RAW / DV comparison! But why scanning the 35mm on 2K? A simple NTSC-Scan would be better to compare. BTW: Do you already have a name for your system? I think it's time :) |
Juan,
Would it be possible to write your code so that the hard drive records a flipped image, or even a way to toggle back and forth from normal to flipped modes? Like many others on this board, I think your 4:4:4 system will coordinate perfectly with a 35mm adapter. Recording the image with correct orientation would helpful in the longrun. Although, if its too much of a hassle, flipping in post is pretty easy too. |
Nicholi,
That's not a hassle at all, and in fact already implementing it...the function i wrote to deal with the inverted blue channel can invert images in any orientation, and thus you can save them inverted vertically if you'd like. |
Just to make sure everyone is saying the same thing:
The 35mm adapters flip the image vertically AND horizontally, right? so the image would have to be flipped not only vertically (like the blue channel you mention Juan) but horizontally as well. (or rotated 180 degrees, same thing) As was already mentioned, this would be simple enough to do in post, but it would be a nice addition if possible. -luis |
Yes Luis, you are correct.
The image must be flipped both horizontally and vertically. Just for further reference, this is also referred to as inverting (flip vertically about the x-axis) and reverting (flip horizontally about the y-axis) or, like Luis said, rotated 180 degrees. Juan, Am I understanding correctly that the user will be able to switch between the normal and flipped modes? If so, this would be excellent for use with switching between 35mm adapter projects and other projects recorded with just the native camcorder. Either way, Juan, you're doing excellent work. |
Yep, that's correct, the user will be able to decide the orientation completely, that is there are check boxes for flipping horizontally and vertically, as well as rotation up to 270 degrees.
In other news, i know nothing about color correction so this is mostly for fun. It took a similar shot of the lamp and attempted to make it look as much as i could as the film output. Now, the main problem I had was that since I do not have control over exposure because it is disconnected, all i have is the ND filters, and one of them was too bright and the other too dark. I started with the dark one and had to push it A LOT. I basically clipped a lot of the low end in the levels and it shows in how saturated part of the image is...still if i got this far with this dark an image, once i have the camera closed and i can vary the exposure, i can get adjust the light going into the camera to be in the correct range... http://expert.cc.purdue.edu/~pertierr/sample1.tif http://expert.cc.purdue.edu/~pertierr/sample2.tif Juan |
A noise free 4:4:4 10-bit uncompressed -> HD frame, with some of my own color correction...
http://expert.cc.purdue.edu/~pertierr/cap10_RAW.tif |
I don't mean to be annoying, but I've noticed a bit of chromatic abberration in cap10 RAW and sample2, but sample1 has no chromatic abberration at all. It looks perfect.
Could this be due to your color corrections? I'm sure your initial recording is perfect considering sample2 looks so great, but I figured this was worth mentioning. EDIT: It just dawned on me that you are capturing the red, green, and blue channels seperately and then layering them in post. I'm guessing the "abberration" is probably being just a hair off on the alignment of one of the layers. This seems like a valid subject though, so I'll leave the whole post here. |
what kind of chromatic aberration is it? if you're speaking of something present in the histogram, it wouldn't suprise me, because the frame was put through S-Spline pro for up-rezzing which could have some unexpected effects.
All I did on cap10 other than uprezzing was some level adjustment. |
I'll use sample2 as an example. I've opened the file up in Photoshop. Even at 100%, I notice a very thin green line around the base of the lampshade. If I zoom in to 200%, it shows about a 1 pixel "shift" of green to the bottom, and red to the top. After seeing that one part, its apparent throughout the rest of the image. I figured the RGB layers just aren't aligned right. Could this be correct?
As far as uprezzing, both cap10 (uprezzed) and sample2 show the same abberration effect. I guess whatever these two have in common, but different from sample1 will be the culprit. |
Yup, that's alignment...try one of the other Photoshop PSD files that i have uploaded, and see what alignment you can come up with by moving the R,G,B layers...it's tricky! :)
Juan |
Yeah, that is tricky. Good thing you can program the computer to do all the precision detail work, because I know I couldn't :).
|
Juan,
I noticed a few stray noise specks in Cap10_raw. I was adjusting the levels and they cropped up when I brightened the file. The specks are green and appear in the sky. Do you see them? |
Yup! I haven't gotten the new cap yet, i've been working on finishing the software to put out a clip...
Juan |
Update..
Intersting things first...i have good reason to think that the CCD's are actually 811x495, however several columns seem to be covered up for some reason,(black level adjustment?). Given the barrel distortion of the lenses with the current configuration, it probably would work well even if there was a wider image available, but it'd be nice.
I am sooo close...i've got the program spitting out clips perfectly, except for one thing. There is a somewhat periodic shift in the raw output frames of 0 or +1 rows, such that right now the video seems to be shaking slightly...i'm trying to figure out how to recognize the shifted frames so I can compensate for this, and then i'll be done. Except for the slight shakes, the video looks awesome! Pretty CPU intensive though! |
4:4:4 12-bit Uncompressed DVX100 - shakes
Could the shakes be due to vertical subsampling?
Dalsa has an application note on this AN07. This is my first post to DVi; however, I have been following this thread for a while. I would like to add my name to the waiting list for Juan's modification and software. It will be great to escape the limitations of the DV format. I don't think these sort of limitations are what Orson Welles had in mind when he said "The enemy of art is the absence of limitations." [thanks to Isaac Brody for quoting this]. I worked with Orson on some projects from 1978 on. He often remarked that Black and White movies were better than Color movies and Radio was better than movies. Art is often in what is left to the audiences imagination. Jaggies and poor chroma due to the limitations of DV are not what Orson had in mind to promote art. He shot his TV projects on 35mm film because he didn't like the limitations of 16mm or video. Excuse my rambling here (somewhat off topic): Now that Juan has almost solved the problem of outputing raw files to disk from the DVX100. There are still some remaining opportunities for research. JVC will release a 3 chip High Def box camera the 870 in July. It is said to have dual SDI outputs. However for effects and compositing this camera would benefit from a raw output. Juan if you are willing to take up the gauntlet here may JVC would even loan you a camera! There is talk that JVC will furnish raw outputs on their mid-price pro HD camera to be released in 2005. The new JVC cameras use a 2/3" cmos chip from a Rockwell spinoff. [Development kits cost $10,000]. What is the size of the DVX100 chip 1/2"? This brings up another annoying problem with DV camera's they don't have 35mm depth of field characteristics. PS+technik has an expensive solution previously mentioned in this thread. I used to have 1960s RCA TK-41 image orthicon cameras that had a 35mm image area. They used motion picture lenses. There was no moving ground glass. They did make use of a field lense to bring the image (snorkel fashion) back to the image orthicon tubes. Perhaps I don't know optics well enough but I would think a field lenses could incorporate image reduction and perhaps an increase in "speed." by concentrating the light. Now maybe that wouldn't preserve the depth of field relationships? The problem with mini-35 is that you lose two stops. DALSA has a 35mm image area CMOS sensor. To me using a sensor that has 35mm image area would be ideal. Now if different parts of the sensor had to be driven separately and the image assembled in a frame buffer that would be OK. I also understand that ARRI is working on a 35mm image area sensor. These solutions will cost big bucks! In the meantime Juan has enabled HD capable results from a $2500 camera! Bravo! Incidently my retro solution to avoid DV jaggies has been to shoot 600 horizontal res. analog NTSC on a sony BVH500 1" machine and to record the 9bit output from an ampex Zeus TBC uncompressed to the G5. This doesn't avoid the cross color and dot crawl problems of NTSC but i don't get Jaggies. I am thinking of recording rgb on three machines to outdo betacam sp. I can't wait to get my DVX100 modified. This is definitely the way cool solution to avoiding DV artifacts! |
Quote:
|
Lots in interesting stuff:
Great idea about adding the image flipping problem of the 35mm adaptors to this. I agree that a combo of 35mm adaptors and 4:4:4 (there's got to be a good name is their somewhere) would provide a very non-DV looking image. And it would make far more sense to drop the high price for buying or renting the prebuilt 35mm adaptors using Juan's cam then compressing the image back through DV codecs. Juan: I recall reading somewhere (perhaps in Steve Mullens article on the DVX100) that pixel shift was employed. Unfortunately, I don't remember how or why, but it was a variation on the pixel shift in the XL1 CCDs. Randall: Welcome aboard. I just recently got the restored Citizen Kane and how can you not love the cinematography? Also, our local filmhouse is actually doing a Welles retrospective though no word on what prints etc.. Welles could be a whole topic from this thread - but I agree with your point that he would have not been a fan of the DV codec or any artificial limitation. The DV codec was great to break off from the generational, non-digital aspects of previous video. But we've rapidly gone from the days of having to carefully tweak a machine to capture DV data to easily being able to capture HD or uncompressed. I'm hoping Juan's efforts will help hasten the end of all these lossy, artifact inducing aquistion codecs. |
Stephen and Listmembers,
Thanks for the welcome. Juan's hope of providing a $50 mod for the DVX100A to allow raw output or 12 bit processed output is awesome. We all know we would be willing to pay $1000 for this. Juan are you still using the Adlink PCI-7300A. I am thinking of trying to duplicate your experiment. However the PCI-7300A carries a list price of $880.00 without cables or probes. This device has a maximum clock rate of 20MHZ. So it won't handle the data rate of 75MHZ on the JVC camera I am interested in tapping in the future. I assume that you started with Tektronix klip-chip probes. Did you use the ADLINK Scsi terminal block to connect to the Adlink? Altera has a logic analyser probe for the Apex 20K but I am sure the kit is pricey. http://www.altera.com/literature/ds/dssignal.pdf On the other hand it might allow some reverse engineering of Panasonic's program if that were necessary. This probe probably is too big to fit inside the camera? I guess I would like you to tell us where you are getting your APEX20K probe made. I can understand why you might not want to reveal that at this stage. However, I am not too shy to ask. It might be interesting to make a "open source" project out of this. You might actually make more money writing books and furnishing support to users than you will selling mods. Just a thought. As someone pointed out earlier. Manufacturers are not going to be too happy when they see $3000 cameras out performing their $100,000 SDI, DVCpro and HDV models. For recording, Blu-ray DVD disk recorders might be a good option to hard disk recording (archive copy). Of course 2 gig and 4 gig solid state cards are now available from panasonic. A 2 gig/sec Fibre Channel interface might be nice if it wasn't so expensive to implement. I would be interested in making a dual head 3D rig that might require higher than firewire 800 bandwidth to connect. I guess two firewire 800s would work. off topic comments: Stephen, Hope you enjoy Citizen Kane. Welles probably takes too much credit as auteur on that film. He had some great collaborators. I believe it was Greg Toland who shot and lit most of that picture. Welles was a perfectionist though so if he didn't like the lighting he would have sent everyone home till it was fixed. He certainly wouldn't have put up with DV artifacts! |
Juan and Listmembers,
Can you tell us which model of the Altera Apex 20K is used? Is it the 208 pin package. What is the package type? Ironwood electronics makes probing adapters and prototyping adapters for altera devices but you have to know exactly what device and what package to order. http://www.ironwoodelectronics.com/catalog/indexes/main.cfm?ID=30&ProcessID=11 I assume the probing type adaptor is what is needed. I am also wondering if the Altera Device is in the same package in my DVX100A vs. DVX100? Can the 10bit/12 bit processed video be accessed from the same clip on adaptor or are those signals on another device? |
Just an update,
I've found a way to correct for the shift, i'm now trying to fix a bug which causes the corrections to cease after a few frames at the beginning of the clip. I think i know where the problem is, but the code is getting pretty large so it just takes debugging... Randall: If i remember correctly it is the 208 pin type. However there are unsurpassable problems to using that probe, as you will find... Cheers, Juan |
Juan,
Sounds like you are making rapid progress on the software side. Are you coding in C? I looked around a little more for probes. What do you think of http://www.winslowadaptics.com/uk/clip1c.asp Where would the capacitor be placed? The picture makes it look like this probe is too big to fit in the camera case. Perhaps a cable could be made that connects to clip pin assembly. Tapping on to the Altera chip, what sort of glue will you need to take the raw data into the TI firewire chip? When you post the clip, you could say "one small step for an engineer, one giant leap for the creative community." |
Juan and Listmembers,
Does anyone have a service manual and schematics for the DVX100 or DVX100a? Are the service manuals hard to obtain? I am just thinking ahead to when you might like to tap into processed video. |
Juan,
Here is another site with the Delta Clip chip adaptors. A flex ribbon cable out looks like it would work. http://www.adapters.com/catalog/test_clips.pdf |
| All times are GMT -6. The time now is 09:30 PM. |
DV Info Net -- Real Names, Real People, Real Info!
1998-2025 The Digital Video Information Network