Apple to add a dedicated media processor to Macs?
 

Robert Cringely thinks so.

Moderator note: Content which was copied and pasted from another site has been edited. Please don't copy large blocks of text from other websites as this is a copyright violation; instead just provide a link and a short quote.

Could this be one of the Top Secret features of Leopard? The ability to utilize new hardware encoding/decoding features in the nextgen Macs? Could this by why there are delays in the Macbook Pro and Mac Pros and Mac minis? This would be huge for new computers as the ability for all new computers to efficiently encode/decode AVC content would be a boon to HD movie downloads and a boon to iMovie/FCE/FCP encoding of HD material.

Sure we have the ability to get more power in encode/decode via the GPU in the computer but GPU go from Integrated Chips to full on megawattage firebreathing discrete cards. How does a Apple ensure consistent playback at the very least? They add a media co-processor.

What if Apple were to add a media co-processor like the 3DLabs DMS-02 media IC. This chip consumes as little as 1 watt of power yet contains a 2-core ARM processor. It'll decode 720p and encode D1 at 30fps and encode 720p24. It supports OpenGL ES, audio CODECS and more output channels for RGB 1280x1024 dispay. It plugs into small devices without disrupting the programmatical local model like DSP chips do.

Now consider that with OS X 10.5 Leopard Apple has cleaned out the software stack enroute to adding 64-bit. They are already letting us know that the new QT h.264 encoding engine is "significantly" better and supports Alpha Transparency. It further abstracts the code written against the API from the hardware below.

So with this in mind imagine that Apple announces that Leopard will support this new media co-processor on all new Macs coming. This co-processor will allow much more solid video display and encoding without sapping CPU resources. Suddenly iMovie and iDVD don't take a lifetime to encode the video. But since this co-processor is integral to each Mac ANY application written against the new QTkit API automagically takes advantage. For Pro's this isn't a huge deal as 720p isn't as sexy as 1080p but for quick encodes to test it would likely be a boon.

The real deal, however, is with Apple TV (ATV). Currently downloaded movies will automatically come in SD AVC format. There will be no problems playing on the ATV. The ATV is going to excel at playback of audio, photos and iTunes content that already exists in h.264/AVC format. What about the video content stored already on your computer that isn't in AVC format? The ATV "only" supports MPEG4 based video (which includes h.264/AVC). That means you're SoL if you have AVI, MPEG2 and other CODECS.

Now what if the media co-processor in addition to encoding or decoding video functions equally well as a transcoder? Suddenly we erase the limitation of the ATV being MPEG4 only. If our local computer can transcode most of your non AVC videos into AVC at near realtime speed then we'd only suffer perhaps a small "buffering" period before playback happens on the ATV. The video is transcoded to the efficient AVC prior to delivery over Wifi.

The cost of said media co-processor would be roughly $30 or less in the quantity that Apple would require. The "heat" costs would be minimal as the processor would likely only consume a few more watts at maximum dissipation unlike adding another level of GPU. Cooling subsystems would not have to change most likely.

By doing this Apple ensures a baseline of performance for video decoding and encoding on all Macs with the processor. They instantly make CPU sapping tasks like encoding something that can be done in a background process while the user keeps on computing. This a a rumor that makes a lot of sense. Macs are and have always been known as multimedia machines. Apple could be poised to raise the bar once again. Given just a couple more generations of Media ICs could easily see us happily encoding 1080p video in realtime. I Want to Believe. What say ye?

I haven't read through this all yet ;), but I can answer the question. Years ago, Intel, decided it would be an good idea if all your dedicated processing needs went through the main processor, even though they were far from perfect on those PC systems running at an fraction of the power they run now. To make things worse, then incompetent C programming was done to get this to work, in an non realtime/efficient fashion. If the original team that brought you geoworks Ensemble had programmed it, it would work, maybe if they had been responsible for windows too, windows would be a tenth the size and work ten times better. The wrong people often are in charge of the Chicken Coop ion the computer industry, purely political. So, now we have software dependent USB2.0 rather than hardware oriented Firewire, software modems, software DVD players, etc etc etc, buggering around stuttering all over the place.

You can run an spread sheet on an DVD player, there are even programmable DVD player formats, apart from windows CE one. These players probably use something like an 16 or 64Mhz ARM processor, many times too much to run an spread sheet (those things that they invented for Apple II computers in the seventies that ran 8 bit processors at 1 or 2 Mhz (equivalent to 1-2Ghz in technology trends today). I would think 16-64mhz is also the minimum range you would need to get an software DVD players working with the right instruction set (FPU etc) support (but that is another story) with proper realtime embedded programming.

There is dedicated hardware in PC's to do these things today. Companies like ATI started with hardware co-processing for DVD's, and other companies followed, you will find such type of things in Intel multi-media chip sets, and NVIDIA, and however many others. You will even find full assist. The funny thing nowadays is, that there is no real need for much of this hardware processing, PC's do often have enough power to do it in software multitasking, except for the poor level of programming and system design, compared to what you would need to achieve this :(. I do not know an conventional desktop OS that is good at this (except, maybe, if they managed to make the Japanese TRON OS into an home desktop OS like they planned).


Wayne.

PS (Salt in the Wounds, often DVD players run an, competent, realtime, embedded, operating system, like Taos Intent/Elate Virtual OS (which has an number of Japanese consumer Electronics company as shareholders). I was hoping to bring this level of competence with my VOS design to the PC for many years (as you can see, this subject eggs me, because of all the lost opportunity it has cost society, and the, Feed The Worlds Poor, massive extra cost it has cost the world).

Nvidia last week had a big presser on it's newest cards that allow general purpose code to be run on them.

Apple for the last two years has been slowly moving all graphics processing (not just moving graphics) off to the GPU, ala Aperture and RAW processing. The official release from Nvidia was wink-wink, nudge-nudging that Apple (named specifically) was going to be taking advantage of what the card can offer right away.

I expect that FCP in particular will be using this family of cards for a huge jump in RT processing, especially with regards to color correction work and the ability to do secondaries in real time with 1080 material. Also, I expect that if FCP is to ever gain the ability to do multiple resolution/framesrates in a timeline AND output to HD-SDI simultaneously, it would also need a card like that.

I forgot to mention, that the Intel multimedia chip-sets were supposed to have full hardware acceleration before the Apple Intel were to come out, but this failed for some reason, and the replacement chips have gone way over time (the top of the 965 series) that promised Direct X 10 and full hardware for H264. There is a Ultra Mobile PC version too, and guess what, Apple is realising an tablet version of there Mac Books, so I hope they have it. The mobile 965 chips are due shortly, and there is another version due by end of the year. But, as I pointed out last year or so, that I found an diagram that showed video input being converted to H264 on the fly on an ATI chipset that would have been released by now, unless it was the delayed DirectX 10 part.

Even though I would imagine that Apple's hardware decoding is related to one of these strategies above, there are other alternatives, and one that I am fond of, is the Ambarella H264 encoder for cameras, and other devices. The chip consumes very little power, it is and array of many (maybe something like one hundred to two hundred) Sparc CPU's operating at a couple of hundred Mhz) with some dedicated hardware, and an ARM processing front end. It can do camera functions (among other things) and encode and decode H264 at an professional data-rate (the professional; version costs a lot more, but consumer is cheap and is supposed to be able to do consumer H264 encoding rates). Since the announcement of AVCHD, there has been an delay and I believe the document on their site about the new version, says it supports that format.

Yes, I suspected that NVIDIA was upto the same standard, thanks for the heads up. I do not believe that the NVIDIA GPU architecture is as good for this as the ATI. The ATI (and slow Intel) uses an combined architecture, that Nvidia aims to follow latter. But even without the Nvidia/ATI graphic cards, ATI/NVIDIA/Intel chip sets aim to deliver a lot of power on the motherboard. For the rest of us, it may not matter too much, Apple can ship cards in the desktop series, but for Mini Mac, and portable suers, it should make an significant difference. This is why I have not upgraded my laptop or bought an Mini Mac, as i have been awaiting for the delayed hardware processing products to be incorporated (IE, laptop is half useless to me without).

PS, Sony has talked about video editing on the Sony Playstation PS3, and they have more recently talked about an multi-media version of the Playstation 3. If they can introduce a more complete and power Playstation 3 with upgraded Direct X 10 level GPU (particularly with an combined architecture (my private wish list) and Mac OSX, I would be very happy (if they could keep it down to an $1000). I believe that i could probably run Mac OSX on an PS3 through an emulator (if one becomes available) but I think it might not across the power of the CELL architecture fro precessing, so an officially supported Mac OSX, is the best bet.

Here is one of my older posts that talked about a beefed up PS3.
http://dvinfo.net/conf/showthread.php?t=82176

People have already run Windows on a PS3 and who know, they may have already run Apple’s OS as well.

Using the hardware for acceleration should have been done along time. I'm surprised their thinking about it now. Apple will be making fewer profit per computer sold if they do this but if it means selling a much, much higher percentage of computers, it may actually be a good idea financially.

Now that I think about it, this will benefit Panasonic a lot because there is only one thing that is bad about AVCHD, and that’s editing. This would be huge being able to edit AVCHD natively on a computer with no more than 2 gigs of RAM without any problems what so ever.