Benchmarks: Page (1) of 1 - 11/20/08 Email this story to a friend. email article Print this page (Article printing at MyDmn.com).print page facebook

The Great After Effects Mac/Windows Smackdown: CS4 Edition

Re-running the benchmarks to see which OS runs AE better By Kevin Schmitt

Last summer I got curious. With the advent of Intel Macs, Boot Camp, and Universal builds of After Effects, for the first time it was possible to forget about all the Intel vs. PowerPC benchmarks that had been run to that point and effectively remove hardware from the equation, focusing solely on the OS and how well AE was tuned to it. XP took the prize back then, but I'm curious again, and with Mac OS X Leopard, 64-bit Vista, and After Effects CS4 at my disposal, it's time to find out where things stand today.

Recap and Methodology (again, such as it is)

As I did last year, I'm going to stress that I'm not a big benchmarks guy by nature, and with that the case I offer the caveat that these tests aren't terribly scientific. However, this particular exercise represents an itch I need to scratch, so all I can do is try to level the playing field and see what bubbles up, with the results hopefully being of some broader interest. Fortunately, others have been good enough to provide tests that are good to go, as we'll see below, so all I need to do is to put in the time to actually run everything under the different OSes.

I had to do a few tests last time, owing mostly to the fact that 32-bit XP and Vista weren't able to address the full 3 GB of RAM present on the Mac Pro I had back then. To level the playing field, I did the main set of tests on my first generation MacBook (2.0 gHz Intel Core Duo with 2 GB of RAM). I also ran the Mac Pro tests, with both versions of Windows doing better than Mac OS X 10.4 Tiger, despite the fact that they both ignored a full gigabyte of RAM. Disappointing for Mac users, but hardly surprising.

This time around, I'm only going to deal with the latest and greatest. The spankin' new After Effects CS4 takes center stage here, running on both Mac OS X 10.5 Leopard as well as 64-bit Windows Vista Business. I'm up to date in both cases; the Mac running the latest point release for Leopard (10.5.5 as of this writing), while Vista has been fully patched with SP1 and any hotfixes available through Windows Update. I'm on a different machine as well -- an early 2008 MacBook 2.4 gHz Intel Core 2 Duo with 4 GB of RAM. Each test was run with AE as the only open program (save for the Finder and Windows Explorer), with screensavers and any other background processes disabled. Each render was set to AE's built-in Lossless setting, which I tweaked on the Windows side to render to the QuickTime Animation codec in order to match what the Mac does. Lastly, I ran each test twice, and averaged the scores together for some of results. I'm not sure how much more even to make things, so with that in mind, let's go through the tests.

The Tests

I laid all this out the last time, but it's worth mentioning again that these are the standard tests DMN has used in the past for benchmarking After Effects, and while they all were created some time ago, they are still perfectly valid and useful in After Effects CS4. Once again, here is a rundown of the individual tests:

  • The first test (Simple Animation) involves a basic sequence consisting of a PICT file with tracing paths.
  • Test two (Video Composite) is a relatively standard compositing setup, comprised of green screen layer, a CGI layer, and a background plate originally shot on film.
  • The third test (Data Project) mixes bitmap and vector layers, animating randomly sequenced zeroes and ones.
  • Test four (Gambler) composites vector-based Illustrator files in 2D.
  • The fifth test (Source Shapes) also uses vector images, though moved around in After Effects pseudo-3D environment and "shot" with a 3D camera.
  • Test six (Virtual Set) uses bitmap images constructed into a 3D environment, also making use with a 3D camera.
  • The final test uses Brian Maffit's Total Benchmark for AE6, which is described and available from this link.

If you would like to replicate the first six tests for yourself (which I encourage; I'd love to see some additional numbers on other hardware), pick up the book After Effects 5.5 Magic that includes a CD containing these AE project files (and many more) along with all the media you'll need to reproduce our results. Special thanks to After Effects 5.5 Magic's author Mark Christiansen and the book's editor, Nathan Moody, as well as New Riders Publishing for giving us permission to use materials from this excellent book.

Enough buildup -- let's get to the results.

The Numbers

What may be useful right off the bat is to just give you the raw dump of everything, and then show some side-by-side comparisons of the numbers in subsequent tables. L1 and L2 are the Leopard tests, V1 and V2 are the Vista tests, with S signifying the single process run, and MP the round with After Effects' multi-process rendering enabled. All times are in seconds.

Raw Results

  L1-S L2-S L1-MP L2-MP V1-S V2-S V1-MP V2-MP
Simple Animation 3 3 2 2 2 2 2 2
Video Composite 23 21 23 23 18 16 19 18
Data Project 53 50 50 50 39 36 41 40
Gambler 14 14 11 12 12 12 9 9
Source Shapes 77 77 65 65 58 58 49 49
Virtual Set 76 74 62 62 49 49 47 47
Total Benchmark 1 59 58 41 41 49 47 36 35
Total Benchmark 2 620 622 530 529 517 517 467 467

Do with that what you will. Let's average the times together now and put a few of the columns together for dramatic comparison purposes, with the best results in red. First up, Leopard vs. Vista in a single process render.

Single Process

  Leopard Vista
Simple Animation 3 2
Video Composite 22 17
Data Project 51.5 37.5
Gambler 14 12
Source Shapes 77 58
Virtual Set 75 49
Total Benchmark 1 58.5 48
Total Benchmark 2 621 517

I'll table the obvious comment for the moment. Next up, Leopard vs. Vista with multiprocessing enabled.

Multiple Processes

  Leopard Vista
Simple Animation 2 2
Video Composite 23 18.5
Data Project 50 40.5
Gambler 11.5 9
Source Shapes 65 49
Virtual Set 62 47
Total Benchmark 1 41 35.5
Total Benchmark 2 529.5 467

More of the same. Now, this one should be interesting -- a single process on Vista versus multiple processes on Leopard.

Leopard Multiple Processes/Vista Single Process

  Leopard/MP Vista/S
Simple Animation 2 2
Video Composite 23 17
Data Project 50 37.5
Gambler 11.5 12
Source Shapes 65 58
Virtual Set 62 49
Total Benchmark 1 41 48
Total Benchmark 2 529.5 517

Ouch. Moving on, just for fun, let's compare how single vs. multiple processes play out on Leopard.

Leopard Single/Multiple Processes

  Leopard/S Leopard/MP
Simple Animation 3 2
Video Composite 22 23
Data Project 51.5 50
Gambler 14 11.5
Source Shapes 77 65
Virtual Set 75 62
Total Benchmark 1 58.5 41
Total Benchmark 2 621 529.5

And lastly, single vs. multiple processes on Vista.

Vista Single/Multiple Processes

  Vista-S Vista-MP
Simple Animation 2 2
Video Composite 17 18.5
Data Project 37.5 40.5
Gambler 12 9
Source Shapes 58 49
Virtual Set 49 47
Total Benchmark 1 48 35.5
Total Benchmark 2 517 467

Conclusions

Cripes, the Mac OS X version of After Effects is absolutely smoked again, and the results are slightly worse than last time in places. Either Adobe isn't tuning After Effects on the Mac at all, or tuning the buhjeezus out of the Windows versions. Hell, even single process rendering on Vista generally spanks multiple processes on Leopard, for the love of Pete.

Vista pwnage aside, it's interesting to see how enabling multiprocessing on either platform doesn't necessarily translate to better performance. The time savings bear themselves out on longer renders, but it isn't always a slam dunk to soak up all your system resources by enabling multiprocessing. If your system is beefy enough, it may be a better bet to give AE a single process and then work on something else while the render happens.

In any event, it's clear from these tests that Vista x64 offers significant "pound-for-pound" time savings for your After Effects renders. We're still in disappointing territory for Mac users, to be sure, but I'm actually surprised this time. I figured the situation would be much improved this time around, and, well, not so much. We'll just have to see how the CS5/Snow Leopard/Windows 7 comparison goes.


Page: 1


Though the fame, riches, and notoriety of being a DMN contributor are both tantalizing and substantial, Kevin Schmitt still stubbornly insists on continuing his work as the Director of Interactive Services at EFX Media, a production house located just outside of Washington, D.C. Feel free to follow his updates and contact him through Twitter if you have something to share - he's ready to believe you!
Related Keywords:After Effects, CS4, Mac, Leopard, Windows, Vista, benchmarks, rendering

HOT THREADS on DMN Forums
Content-type: text/html

@ Copyright, 2014 Digital Media Online, All Rights Reserved