Least required GPU parallelism for kernel executions

GPUs require a vast number of threads per kernel invocation in order to utilize all execution units. As a first thought one should spawn at least the same number of threads as the number of available shader units (or CUDA cores or Processor Elements). However, this is not enough. The type of scheduling should be taken into account. Scheduling in Compute Units is done by multiple schedulers which in effect restricts the group of shader units in which a thread can execute. For instance the Fermi SMs consist of 32 shader units but at least 64 threads are required because 2 schedulers are evident in which the first can schedule threads only on the first group of 16 shader units and the other on the rest group. Thus a greater number of threads is required. What about the rest GPUs? What is the minimum threading required in order to enable all shader units? The answer lies on schedulers of compute units for each GPU architecture.

NVidia Fermi GPUs

Each SM (Compute Unit) consists of 2 schedulers. Each scheduler handles 32 threads (WARP size), thus 2x32=64 threads are the minimum required per SM. For instance a GTX480 with 15 CUs requires at least 960 active threads.

NVidia Kepler GPUs

Each SM (Compute Unit) consists of 4 schedulers. Each scheduler handles 32 threads (WARP size), thus 4x32=128 threads are the minimum requirement per SM. A GTX660 with 8 CUs requires at least 1024 active threads.

In addition, more independent instructions are required in the instruction stream (instruction level parallelism) in order to utilize the extra 64 shaders of each CU (192 in total).

NVidia Maxwell GPUs

Same as Kepler. A GTX660 with 8 CUs requires at least 1024 active threads. A GTX980 with 16 CUs requires 2048 active threads.

The requirement for instruction independency does not apply here (only 128 threads per CU).

AMD GCN GPUs

Regarding the AMD GCN units the requirement is more evident. This is because each scheduler handles threads in four groups, one for each SIMD unit. This is like having 4 schedulers per CU. Furthermore the unit of thread execution is done per 64 threads instead of 32. Therefore each CU requires the least of 4x64=256 threads. For instance a R9-280X with 32 CUs require a vast amount of 8192 threads! This fact justifies the reason for which in many research papers the AMD GPUs fail to stand against NVidia GPUs for small problem sizes where the amount of active threads is not enough.

Compute performance with OpenCL on AMD A6-1450 (Temash APU)

Being interested about the modern low powered Kabini/Temash APUs from AMD I was searching the internet for information regarding its compute performance on its GPU. I couldn't find almost anything. Their GPU is supposed to be based on GCN architecture but no more information was available. In addition, the AMD's APP SDK documents are outdated and they do not include any information about this APU. In fact they do not even include any information about the Bonaire GPU (HD 7790 & R7 260X branded cards) which is even older. AMD should definitely change it's policy if they want to be taken seriously about GPU computing. I hope an updated reference guide will be released anytime soon covering all recently released GPUs/APUs (Kabini/Temash, Bonaire, Hawai) and what is about to be released (Kaveri APU).

So I recently I got access to a small form laptop based on the A6-1450 APU (Temash) and I would like to share some experience I had with it. After struggling for 1-2 days to install a Linux distro on it I managed to install the Ubuntu 12.04.3. I couldn't install a recent version (i.e. 13.10) as it needed to initiate a graphics mode and with the supplied kernel it was not possible to execute the installer. 12.04.3 installed ok and thereafter I was able to install the catalyst manually. As I already tested with the Ubuntu 14.04 Alpha 1 this seems to be fixed.

In theory this APU features a quad core Jaguar CPU and a 128 shader GPU (HD 8250) operating at 300MHz with an overclock capability (max 400MHz). Unfortunately, memory is clocked at 1066MHz though I hoped it would be 1333MHz.

As all released APUs this one also supports OpenCL. So, I'll provide some information here to anyone who is interested. First, here is a revealing output of the clinfo tool:

Number of platforms:     1
  Platform Profile:     FULL_PROFILE
  Platform Version:     OpenCL 1.2 AMD-APP (1214.3)
  Platform Name:     AMD Accelerated Parallel Processing
  Platform Vendor:     Advanced Micro Devices, Inc.
  Platform Extensions:     cl_khr_icd cl_amd_event_callback cl_amd_offline_devices


  Platform Name:     AMD Accelerated Parallel Processing
Number of devices:     2
  Device Type:      CL_DEVICE_TYPE_GPU
  Device ID:      4098
  Board name:      AMD Radeon HD 8250
  Device Topology:     PCI[ B#0, D#1, F#0 ]
  Max compute units:     2
  Max work items dimensions:    3
    Max work items[0]:     256
    Max work items[1]:     256
    Max work items[2]:     256
  Max work group size:     256
  Preferred vector width char:    4
  Preferred vector width short:    2
  Preferred vector width int:    1
  Preferred vector width long:    1
  Preferred vector width float:    1
  Preferred vector width double:   1
  Native vector width char:    4
  Native vector width short:    2
  Native vector width int:    1
  Native vector width long:    1
  Native vector width float:    1
  Native vector width double:    1
  Max clock frequency:     400Mhz
  Address bits:      32
  Max memory allocation:    136839168
  Image support:     Yes
  Max number of images read arguments:   128
  Max number of images write arguments:   8
  Max image 2D width:     16384
  Max image 2D height:     16384
  Max image 3D width:     2048
  Max image 3D height:     2048
  Max image 3D depth:     2048
  Max samplers within kernel:    16
  Max size of kernel argument:    1024
  Alignment (bits) of base address:   2048
  Minimum alignment (bytes) for any datatype:  128
  Single precision floating point capability
    Denorms:      No
    Quiet NaNs:      Yes
    Round to nearest even:    Yes
    Round to zero:     Yes
    Round to +ve and infinity:    Yes
    IEEE754-2008 fused multiply-add:   Yes
  Cache type:      Read/Write
  Cache line size:     64
  Cache size:      16384
  Global memory size:     370147328
  Constant buffer size:     65536
  Max number of constant args:    8
  Local memory type:     Scratchpad
  Local memory size:     32768
  Kernel Preferred work group size multiple:  64
  Error correction support:    0
  Unified memory for Host and Device:   1
  Profiling timer resolution:    1
  Device endianess:     Little
  Available:      Yes
  Compiler available:     Yes
  Execution capabilities:     
    Execute OpenCL kernels:    Yes
    Execute native function:    No
  Queue properties:     
    Out-of-Order:     No
    Profiling :      Yes
  Platform ID:      0x00007f1d93cc6fc0
  Name:       Kalindi
  Vendor:      Advanced Micro Devices, Inc.
  Device OpenCL C version:    OpenCL C 1.2 
  Driver version:     1214.3 (VM)
  Profile:      FULL_PROFILE
  Version:      OpenCL 1.2 AMD-APP (1214.3)
  Extensions:      cl_khr_fp64 cl_amd_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_gl_sharing cl_ext_atomic_counters_32 cl_amd_device_attribute_query cl_amd_vec3 cl_amd_printf cl_amd_media_ops cl_amd_media_ops2 cl_amd_popcnt cl_khr_image2d_from_buffer 


  Device Type:      CL_DEVICE_TYPE_CPU
  Device ID:      4098
  Board name:      
  Max compute units:     4
  Max work items dimensions:    3
    Max work items[0]:     1024
    Max work items[1]:     1024
    Max work items[2]:     1024
  Max work group size:     1024
  Preferred vector width char:    16
  Preferred vector width short:    8
  Preferred vector width int:    4
  Preferred vector width long:    2
  Preferred vector width float:    8
  Preferred vector width double:   4
  Native vector width char:    16
  Native vector width short:    8
  Native vector width int:    4
  Native vector width long:    2
  Native vector width float:    8
  Native vector width double:    4
  Max clock frequency:     600Mhz
  Address bits:      64
  Max memory allocation:    2147483648
  Image support:     Yes
  Max number of images read arguments:   128
  Max number of images write arguments:   8
  Max image 2D width:     8192
  Max image 2D height:     8192
  Max image 3D width:     2048
  Max image 3D height:     2048
  Max image 3D depth:     2048
  Max samplers within kernel:    16
  Max size of kernel argument:    4096
  Alignment (bits) of base address:   1024
  Minimum alignment (bytes) for any datatype:  128
  Single precision floating point capability
    Denorms:      Yes
    Quiet NaNs:      Yes
    Round to nearest even:    Yes
    Round to zero:     Yes
    Round to +ve and infinity:    Yes
    IEEE754-2008 fused multiply-add:   Yes
  Cache type:      Read/Write
  Cache line size:     64
  Cache size:      32768
  Global memory size:     5670133760
  Constant buffer size:     65536
  Max number of constant args:    8
  Local memory type:     Global
  Local memory size:     32768
  Kernel Preferred work group size multiple:  1
  Error correction support:    0
  Unified memory for Host and Device:   1
  Profiling timer resolution:    1
  Device endianess:     Little
  Available:      Yes
  Compiler available:     Yes
  Execution capabilities:     
    Execute OpenCL kernels:    Yes
    Execute native function:    Yes
  Queue properties:     
    Out-of-Order:     No
    Profiling :      Yes
  Platform ID:      0x00007f1d93cc6fc0
  Name:       AMD A6-1450 APU with Radeon(TM) HD Graphics
  Vendor:      AuthenticAMD
  Device OpenCL C version:    OpenCL C 1.2 
  Driver version:     1214.3 (sse2,avx)
  Profile:      FULL_PROFILE
  Version:      OpenCL 1.2 AMD-APP (1214.3)
  Extensions:      cl_khr_fp64 cl_amd_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_gl_sharing cl_ext_device_fission cl_amd_device_attribute_query cl_amd_vec3 cl_amd_printf cl_amd_media_ops cl_amd_media_ops2 cl_amd_popcnt 

It's good that double precision arithmetic is actually supported on this APU (the brazos APUs did not) and this is actually something I didn't know. I measured the raw performance using FlopsCL (http://olab.is.s.u-tokyo.ac.jp/~kamil.rocki/projects.html) and proved to be 91 GFLOPS on single precision and 6.4 GFLOPS on double precision (which I wasn't sure it supported) arithmetic. It's not the supercomputer you were looking for but think that the whole APU has just 8W TDP.

Next, I measured the effective bandwidth with a custom OpenCL application. This proved to reach near 7GB/sec. It's just ok.

For the last I left the NVidia's nbody simulation (it was included in the CUDA SDKs prior to version 5). With a small modification it can run on AMD GPUs as well (and equally well).

Here is a screenshot:

NVidia's nbody sample OpenCL application on A6-1450

Press here for a larger screenshot.

The results are quite good. For a 16384 body benchmark (parameters: --qatest --n=16384)  the APU performed almost 50GFLOP/S (49.67). Let me note here that my 8600GTS did about the same!

On summary, the APU consists a nice mobile development platform for OpenCL applications which supports double precision maths with minimal power footprint.

AMD Sea Islands instruction set documentation is online

A fresh PDF about the Southern Islands GPU instruction set is now available online. Southern Islands is the architecture of the new AMD GPUs yet to be released. Here are some notes found inside:


Differences Between Southern Islands and Sea Islands Devices

Important differences between S.I. and C.I. GPUs
•Multi queue compute
Lets multiple user-level queues of compute workloads be bound to the device and processed simultaneous. Hardware supports up to eight compute pipelines with up to eight queues bound to each pipeline.
•System unified addressing
Allows GPU access to process coherent address space.
•Device unified addressing
Lets a kernel view LDS and video memory as a single addressable memory. It also adds shader instructions, which provide access to “flat” memory space.
•Memory address watch
Lets a shader determine if a region of memory has been accessed.
•Conditional debug
Adds the ability to execute or skip a section of code based on state bits under control of debugger software. This feature adds two bits of state to each wavefront; these bits are initialized by the state register values set by the debugger, and they can be used in conditional branch instructions to skip or execute debug-only code in the kernel.
•Support for unaligned memory accesses
•Detection and reporting of violations in memory accesses

It seems as the Sea Islands architecture will feature multiple queues similar to the NVidia Kepler's promoted as "Hyper-Q" technology (see GK110 whitepaper).

Link: AMD_Sea_Islands_Instruction_Set_Architecture.pdf

UPDATE: For some reason the referred file is not available anymore.