Front PageEverythingArticlesEditorialsReviewsContestsQuestion BinDiaries
What's wrong with Apple's SPEC numbers?
 

By Mark Duell

I was watching the keynote last night, and I was amazed by the SPEC scores presented. So amazed in fact, that I had to go to spec.org to see if they were real. While I won't call Apple's numbers fraudulent outright, they had a number of curious details.

1) Selective use of hyperthreading. On the dual processor tests, where HT would help the most due to the program using multiple threads, HT was disabled; but on the single processor tests where spec only uses one thread and HT would be detrimental due to resource contention, Apple conveniently enabled HT.

2) Compiler choice. By choosing gcc for both platforms, Apple puts the Dell at a disadvantage. Better compilers exist for x86 hardware, namely ICC, and gcc has different optimizations for each platform. Similar circumstances on the benchmarks that are written in Fortran, Apple using NAGWare Fortran on the PC instead of the superior Intel Fortran or Compaq Visual Fortran. Note Jobs's error of omission, since he only talked about using gcc for the benchmarks.

3) OS choice. Instead of using a publicly released OS, Apple chose to use a internal beta version of 10.2.7 on their hardware; this is fine, as I'm all in favor of using the best available (to debate the availability of an internal beta is a discussion for another time). But on the Dell they decided to use RedHat Linux. This seems odd, since all of the officially published spec benchmarks for x86 use Windows 2000/XP/2003 (all of which have been publicly released), and Windows is used far more often in homes and businesses.

4) The little things. According to page 27 of VeriTest's report, on they Dell they use -O3 -march=pentium4 -mfpmath=sse. According to the Pentium 4 optimization manual, scalar SSE/SSE2 math is slower than the plain old 387 math on the Pentium 4. Specifically, the 387 math can execute one FP addition per cycle, whereas SSE2 executes one scalar FP addition every two cycles. Incompetence or malice, neither looks good for VeriTest.

Debunking the price myth:

Take a look at the low end

Apple PowerMac G5

Dell Precision 650

Dual 2.0Ghz G5 (fp: 15.7, int: 17.2, average: 16.45) Dual 2.4Ghz Xeon (fp: 14.7, int: 18.7, average: 16.7)
512MB PC3200 RAM 512MB PC2100 RAM
160GB SerialATA HDD Dual 80GB ATA/133 HDD
DVD/CDRW DVD/CDRW
ATI Radeon 9600 Pro nVidia Quadro NVS 280
Built in audio and FireWire Sound Blaster Audigy II with 1394
17" Cinema Display 17" UltraSharp Display
Accessory Kit (Mouse and Keyboard) Mouse and Keyboard
Mac OS X 10.2 Windows XP Professional with SP1
AppleCare 3 year warranty with bring-in service Dell 3 year warranty with onsite labor
Price: $3,718 not including sales tax and shipping Free Palm Zire
Shipping: August Price: $3,242, not including sale tax and shipping
  Shipping: Today

What about the high end?

Apple PowerMac G5

Dell Precision 650

Dual 2.0Ghz G5 (fp: 15.7, int: 17.2, average: 16.45) Dual 2.4Ghz Xeon (fp: 14.7, int: 18.7, average: 16.7)
2.0GB PC3200 RAM 2.0GB PC2100 RAM
250GB SerialATA HDD Dual 120GB ATA/133 HDD
SuperDrive DVD-RW 4x DVD+RW
ATI Radeon 9800 Pro 128MB nVidia QuadroFX 500 128MB
Built in audio and FireWire Sound Blaster Audigy II with 1394
Dual 20" Cinema Displays Dual 20" UltraSharp Displays
Accessory Kit (Mouse and Keyboard) Mouse and Keyboard
Mac OS X 10.2 Windows XP Professional with SP1
AppleCare 3 year warranty with bring-in service Dell 3 year warranty with onsite labor
Price: $7,121 not including sales tax and shipping Free Palm Zire
Shipping: August Price: $5,492, not including sale tax and shipping
  Shipping: Today

The quoted fp/int numbers are from Apple (for the PowerMac), and SPEC (for the Precision). The Dell lacks FireWire 800 and the memory in the Dell is about 33% slower. The graphics cards are different due to options on the BTO sites, but have similar street prices.

The real SPEC numbers:

  Dual 2.0Ghz G5 Dual 3.06Ghz Xeon Dual 1.8Ghz Opteron*
SPECfp_rate_base2000 15.7 15.7** 24.7
SPECin_rate_base2000 17.2 21.7 25.0

* For comparison only, server oriented machine
** Highest score on a workstation oriented machine. A server oriented machine scored 16.7.

Although I advocate using the best software available for each architecture, when introducing the SPEC benchmarks Jobs says "We're going to use gcc 3.3 because we want to use an open source compiler that's the same compiler on all machines, and the compiler we all use, by the way." If he wants "open source" and "the same" why not use Linux on both?

Have a comment? Post it here.


Menu
submit story
create account
about us
faq
search

Login
name
password


Click here to get an account

 

All trademarks and copyrights on this page are owned by their respective companies. Comments are owned by the poster.
All else © 2002 Artificial Cheese. All Rights Reserved.