LOW SYSTEM MEMORY error message
 

Hi,

I've been having this bug forever witgh PP2 then PP3 with AspectHD v4.0 then v5.0, I've fixed about any Windows settings possible enabled /3GB and /PAE switch, nothing fixes it just make it less bad... I also use a little application to free ram (like the one in AE) when editing in PP3, but when I render it crashes since I can't do that during rendering.

Whenever my PP3 exe gets to 2GB of VM size or to 1GB of Mem usage it crashes with a "sorry PP3 has encountered a serious error" message.

The only thing that tech support at CINEFORM can offer me is to fragment my project file into lots of smaller ones. This I absolutly can't and won't do since I want to have access to all my clips in one project, and in doing so, would totally mess up my workflow.

I had a thought that since I get a low system memory message and that system memory consist of roughly RAM and CACHE; and that my RAM is 4 GB with /3GB and /PAE switches enabled and that the maximum PP3 can handle RAMwise, that the low system memory must be caused then by low cache...!

My CPU is AMD64 Athlon x2 4600+ with 512 MB of L2 cache, which means 256 MB per core... not much when I look at the new XEONs or Core Duo with 4096 MB (2GB per core).

To what extend can Cache impact on this bug, am I right to a certain degree? Or Cache hasn't an impact significant enough...

What about the rize in clocking, would this impact the memory leak or this would just fasten the time it takes my computer to crash upon rendering ?

And what about something like MATROX x2 would this help solve the bug or maybe and probably not ?

Thank you.

Sincerely,

Frédéric Leclair

My project settings are:
CFHD low resolution 1440x1080, 60i
about 1.2 TB of data including wav and CFHD files (low res)


My computer settings:
Windows XP Pro (all upgrapded)
Adobe Premiere CS3 with AspectHD v5.0 (last Build upgraded)
Ultramon with 2 monitor LCD 19" wide
AMD64 Athlon x2 4600+
ASUS M2N-SLI Deluxe
4 GB of DDR2 RAM 733 mghz (4 x 1 GB chips)
MATROX APVe
Soundblaster X-FI
c: 160 GB
Z: RAID 0 of 1 TB (3x320GB)
Y: RAID 0 of 500 GB (2x250GB)

It is inherent in Premiere as a win32 application, CineForm helps lower the memory footprint making it more stable. You should reconsider dividing you project into reels as the high-end workflows do. You can still keep all needed source material in each project reel, just reduce your sequence complexity -- also don't have to many big sequences or complex nesting as they all suck up memory.

I'm currently using "big sequences with complex" as a mean to have my video resynch with my audio (dialogues and ambience) taped separatly on 2 tracks (one is mono and the other is stereo). How could I have those resynch if not in a big nested timeline?? I don't want to have to resynch audio everytime I drop a clip in the timeline... and I can't export these into separate clips with 3 audio channel tied to the clip (stereo for camara and mono for dialogues and stereo for ambiences...) "Or can I !!???"

Should I have all my clips in small individuals nested timeline (one for each) ??

What about the cache suggestion I've made, any thoughts about this ?

Since its inherent to PP3 (which is obvious since it can gives that message even in a non-cineform project file if it's big enough...). A question comes in my mind: Doesn't the sheer size of CFHD detracts its lighter algorythm - when used in a long form project -by been too heavy to bear on the memory with that much of clips ?

All my small size CINEFORM project work fine (that I've tried so far...); why does it crashes when its a bigger project? Is it just of system memory limitation to follow a bigger project needs or is it that PP3 gets to a non-return point with the project size and peak and can't handle it anymore even if memory limitation where bypassed and memory could go high enough to follow the project's size need ?

Thank you.

CFHD is actually lighter than many other formats on memory usage (that what matters not disk space), and yes you can have the same failures with a DV project, so it has nothing to do with CineForm. A native m2t project will have issues with much less complexity, as that codec uses more memory then CineForm.

I don't understand you tone when you describe your timeline -- are you joking? All we recommend is the a 20 minute reel be a maximum project size -- if the does work for you please try Adobe support as CineForm can't fix a limition in the parent application.

Cache size has nothing to do with it. It is simply the application is using too much memory, and 32-bit apps can only address around 1-1.5GB before they die. Premiere just uses a lot of memory to database clips, in the bin, and more data if in a sequence -- doesn't matter even is the clips are unlinked, the memory is used. Check you project size and if it above 25MB you might hitting these limitations.

I'm having trouble understanding the meaning of this phrase since some words seems to be missing in it (I'm native French speaker ... :))

What I understand is that you must have misunderstood something I've wrote about my timelines since you seem to think I'm joking... I'll re-write (better I hope) the segment pertaining to timelines:

You say that that big & complex nested timelines tend to suck up memory. The current project form has 37 nested timeline of 1 hour each to allow a simili timecode since batch capturing with HDlink didn't keep the original timecode, but mostly, having clips on a long timeline like this allowed me to resynch recorded audio (which was done separately on 2 tracks - 1 mono + 1 stereo) with the clip. So all clips have 3 tracks on the nested timelines.

I'm usingthis unlikely workflow because I don't want to have to resynch audio with video everytime I drop clip in the timeline, because the recorded audio and the video = 3 separated files. The only possible way to link these files (that I now of) is to resynch them on a nested timeline. I don't know any way I could export each resynched clips into a new one that would keep intact the audio track information of the 3 original files (the video file + the 2 recorded audio one)...

That's why I've suggested that each clips could to be nested into small sequences, keeping recorded audio synchronized this way and allowing fragmentation of the project file. (I guess this is where you though I was joking...) this possibility seems very tedious but if I was to framented my footage I only see this "solution" or to realign in scenes order my clips unto nested sequences, and to fragment my project file into reels containing smaller amount of big clunky nested timelines...

I hope I was clearer this time ! :)

When you're refering to 20 minutes reel, what do you mean ? Highest possible duration of a timeline OR amount of total footage in a project files OR maximum total timelines duration in the whole project file ? Could you specify please.

BTW, I once did test to try to see to what point my project would start behaving badly with the memory. I've took files out of the project until I arrived at a normal workflow without crashes (my project file is currently 81,1 MB). Upon doing render tests, my PP3 went as high as 2GB without crashes of any sort, but the more I added back clips the less higher it could go without crashing. So the 1,5 GB barrier seems to be relative since I also go pass it with my After Effects without any problems, maybe a slowing down but no more...

Thank you again,

Frédéric

BTW I'm not attacking CINEFORM in any way, but I'm just trying to understand the HOW and WHY the problem occurs.

81 MB Wow! I've been informed that things go bad fast over 50MB (maybe that was a PPro 2 number.)

20 minute reels : When cutting feature film, projects are normally edited at the length that would match one film reel for projection -- i.e. about 20 minute. A 1hour 40min film would have five reels -- we recommend five separate projects.

David,

Have Adobe officially informed users that things go bad after 50MB? Maybe I should ask for my money back for my PPRO CS3 upgrade then!

I tried splitting my film into 20 min project, but the workflow just really sucks. That might be how the 'pros' do it, but it doesn't suit me - The simple task of moving a scene and checking the pacing of the result is made much harder.

My project is around the 80MB mark too - an 80 minute movie with approx 1TB of associated clips.

I'm close to video lock on the film, so I'll prob split it out now in order to Colour Correct and Magic Bullet.

For a piece of software to basically crash is just bad coding, bad memory management - and can be fixed, so when I have some time I'll be bombarding the Adobe support for help! It seemed to improve with CS3, but not altogether gone.

I've only ever edited Aspect HD HDV in PPRO - so I have no experience of large native m2t projects...

Frederic - it might suck, but it looks like your system doesn't like your workflow...

Try not bother rendering! If you're doing colour correction or panning, zooming, titles, fades etc - just scrub the timeline to preview and leave the rendering to when you Export. Rendered previews are not used for Exporting, so you're not saving any time.

Even consider not applying any effects on your timeline, and once you've got video lock, export the timeline out to a High CFHD avi - then import this into a new project and apply FX to that.

Exporting: install DebugMode Frameserver - this serves out your project uncompressed frame by frame. You use it as any other codec, it saves a small .avi - which you would then import into your converstion application like TMPGEnc Xpress - this will create great looking DVDs without having to export a large CFHD avi. Memory seems to be managed better - so you can now export timelines that previoulsy crashed.

I've even exported a High CFHD avi from TMPGEnc using DebugMode Frameserver (served from PPRO CS3) when I've had a complicated title sequence which Premiere Pro kept crashes when exporting! Oh, and then import this avi back into my PPRO project! Talk about a$$ about t1t!!

Good luck.

Cheers, Doug.

PS - some plug-ins are written badly, so if you're using Boris, Magic Bullet etc etc, then it could well be one of them misbehaving also.

Nothing official, but there going be an issue while the OS and application are 32-bit, short of some radical memory optimization on Adobe part. So there will always be a project complex enough to cause problems with a limited memory pool. Reduce complexity however you can.

For many of us, there is a workaround that does help a bit...

I actually run with the task manager open on the performance tab at all times...(yes, i know it's a pain) and I watch the page file usage. Ppro always had a particular size the pagefile would get to and then the app would fall over. For me, once the pagefile got to 1.7 GB, I knew it was time to save and click off the application to change the focus, which would shave it down, or i restart the app, particularly if I had a stressful export to do like a master edit clip of a film or something of that nature.

PProCS3 is better it seems, but i'd also agree that the issue isn't eradicated by any means.

Douglas,

Thank you for your suggestions, glad to see I'm not alone having this nightmarish bug!

I'm not using any pluggin, just lining clip in timeline and putting basic crossfades...

For doing my movie trailer I think not having everything in one project would suck big time, since I've got to acces clip from many scenes out of order and not search and import files everytime I think one scene could fit or not...

But, I think I will bend to some sort of fragmented project, since my movie works in chapters and could be separated on these.

I'm sorry but I didn't understand you, my knowledge of what your describing here is limited. :(

Thank you.

Yes I do that too, and use FreeRAM XP Pro to clear PP3 excesses of Mem Usage when it gets too high. The problem is I can't do that when exporting or rendering and both of them make my memory peak and my PP3 crash.

PP2 didn't used to crash upon renders and crashed less then PP3 during export.

Yes the workflow is better, file size too (I had 127 MB with PP2), but the above described crashes are bad, maybe worse since I could manage the RAM by flushing it very often and could render, now I don't have to clear RAM too much but can't render...so to use ADOBE PREMIERE's box design, by upgrading to CS3, I've traded a bad horse for another... :)

If you're at the rendering stage then breaking the project into 'reels' is easy:

1) start with your 'master' edit that has everything.

2) save as a new project (reel one, or whatever)

3) razor cut and delete at an appropriate point in the timeline, or delete all nested sequences before and after what you have determined to be an appropriate time point to split the reel.

4) open 'master edit and repeat to create reels 2-5 (or whatever works)

This way you have a master edit project which you can still use for edits, just repeat this process whenever you wish to render.

Also, you can import the completed render clips into each reel project and place on the timeline for continuity if you wish to edit in seperate projects for each reel.


Regarding memory usage on 32 bit apps in WinXP - there are apps that render fine in excess of 2gigs. I've rendered projects in Renderman & Nuke that exceeded 2gigs mem usage w/o crashing.

Adobe could have a world-beater on their hands if they made their software more memory robust. Editing in reel segments is ok as a workaround to ancient (in computer terms) bottlenecks, but really in this day and age it's a ridiculous limitation, IMHO.


regards,

john t.

I swallowed my pride and created 5 x 17 min projects (basically cut at a decent scene break).

To reduce Adobe's workload further I removed any video bins that were not used (I don't use the Project Manager to do this, just trash the bins for scenes not in this 17 mins of the film). Projects are now around the 17-24MB ranges (instead of one 81MB).

Did this as I'm about to enter into a film competition down here in Aus, and I need to render out, deinterlace and apply film looks etc.

Gotta say, loading, saving, scrubbing, rendering is pretty much perfect now.

It @%@%$ sucks that Adobe can hide behind the old 'Microsoft are cr@p' excuse and deliver a product that doesn't make allowances for these limitations that other applications have been dealing with for years!

It'll just take clever coding, and a realisation that some people DO want to edit feature films in one project...

- calling it Premiere 'Pro' is a bit of a joke, eh. - maybe call it Premiere 'Short Film Edition'

Frederic... I'll try to explain DebugMode Frameserver for you. This should allow you to render out your trailer.

Install DebugMode Frameserver http://www.debugmode.com/frameserver/
(it's basically a codec, if you like).

Export your trailer - chose DebugMode Frameserver from the Export Settings drop-down list. Save as "frameserver.avi".

Frameserver dialogue box will open up, click Next (check the settings with the documentation on the website above). This will write a small avi file which basically 'points' to each frame in your Premiere Pro project (also contains audio by default).

Keep this Frameserver dialogue open - it should say "Serving".

Open up a conversion application, like Sorenson Sqeeuze or TMPGEnc Xpress and import this "frameserver.avi" as you would any other avi you would convert. (you could even Frameserve directly into Adobe Encore, DVD Architect or similar for a DVD build).

In this application, chose the format you want to convert - if you're a hardcore quality man, then go out to Uncompressed MOV (huge files) - if not, go for an H264 MOV or a high def WMV (1280x720 is pretty good, try 25Mbps video bitrate for high qual) - you can always downconvert to an 'online' or streaming video from this high qual version.

Then start the conversion.

The Frameserver dialogue box will tell you that it's serving frames of your PPRO project out to another application.

Once your conversion has finished, click Stop Serving on the Frameserver dialogue box, and Premiere Pro will return back to normal and the "framserver.avi" should be gone.

Frameserver has worked for me when I've had 'un-exportable" projects.

Good luck.

Thanks Doug and good luck for your festival application.

I currently am cutting my project into separate project and to do so I need to copy/paste my nested clips unto new timelines in scene order (cause obvious I've shoot and digitized the tapes out of scene order...), this is not that long, but rewriting the editting log as I'm copy/pasting is very long and tedious... :(

Also, I was wondering: can we copy/paste the timeline marker on a sequence to another along with the clips ? Cause right now I'm losing all my cues and annotations when copy/pasting. I've checked PP3 help index but found nothing: it looks pretty primitive on marker maleability.