Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

875 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

oneAPI.jl

Julia support for the oneAPI programming toolkit.

oneAPI.jl provides support for working with the oneAPI unified programming model. The package is verified to work with the (currently) only implementation of this interface that is part of the Intel Compute Runtime, only available on Linux. Windows support is experimental.

Status

oneAPI.jl is actively maintained and contributions are welcome — see the issue tracker for open work.

The current version of oneAPI.jl supports most of the oneAPI Level Zero interface, has good kernel programming capabilities, and as a demonstration of that it fully implements the GPUArrays.jl array interfaces. This results in a full-featured GPU array type. On top of that, the package integrates with vendor libraries through oneMKL, covering dense BLAS and LAPACK, sparse linear algebra, and FFTs, and it provides a KernelAbstractions.jl backend (oneAPIBackend) so portable kernels run unmodified on Intel GPUs.

The package has not been as extensively tested as more mature back-ends, and performance issues might be present. Some operations may still be unavailable or slow.

Quick start

You need to use Julia 1.10 or higher, and it is strongly advised to use the official binaries. For now, only Linux is supported. On Windows, you need to use the second generation Windows Subsystem for Linux (WSL2). If you're using Intel Arc GPUs (A580, A750, A770, etc), you need to use at least Linux 6.2. For other hardware, any recent Linux distribution should work.

Once you have installed Julia, proceed by entering the package manager REPL mode by pressing ] and adding the oneAPI package:

pkg> add oneAPI

This installation will take a couple of minutes to download necessary binaries, such as the oneAPI loader, several SPIR-V tools, etc. For now, the oneAPI.jl package also depends on the Intel implementation of the oneAPI spec. That means you need compatible hardware; refer to the Intel documentation for more details.

Once you have oneAPI.jl installed, perform a smoke test by calling the versioninfo() function:

julia> using oneAPI

julia> oneAPI.versioninfo()
Binary dependencies:
- NEO: 26.18.38308+0
- libigc: 2.34.4+1
- gmmlib: 22.10.0+0
- SPIRV_LLVM_Backend: 22.1.8+1
- SPIRV_Tools: 2025.4.0+0
- oneAPI_Support: 0.10.0+0 (oneMKL v2025.3.0)

Toolchain:
- Julia: 1.12.6
- LLVM: 18.1.7

Julia packages:
- oneAPI.jl: 2.7.2
- GPUArrays: 11.5.8
- GPUCompiler: 2.1.1
- KernelAbstractions: 0.9.42
- LLVM: 9.11.0
- SPIRVIntrinsics: 1.1.0

1 driver:
- 00000000-0000-0000-18c7-da2e010395a4 (v1.3.38308, API v1.15.0)

1 device:
- Intel(R) Arc(TM) A750 Graphics

The drivers and devices that oneAPI.jl found are listed at the end of that output. You can also query them directly:

julia> devices()
ZeDevice iterator for 1 devices:
1. Intel(R) Arc(TM) A750 Graphics

julia> device()
ZeDevice(GPU, vendor 0x8086, device 0x56a1)

If more than one compatible driver or device is listed, use the driver! and device! functions to configure which one to use in the current task, e.g. device!(2) to switch to the second device.

To ensure other functionality works as expected, you can run the test suite from the package manager REPL mode. Note that this will pull and run the test suite for GPUArrays, which takes quite some time:

pkg> test oneAPI

Usage

The functionality of oneAPI.jl is organized as follows:

  • low-level wrappers for the Level Zero library
  • kernel programming capabilities
  • abstractions for high-level array programming

The level zero wrappers are available in the oneL0 submodule, and expose all flexibility of the underlying APIs with user-friendly wrappers:

julia> using oneAPI, oneAPI.oneL0

julia> drv = first(drivers());

julia> ctx = ZeContext(drv);

julia> dev = first(devices(drv))
ZeDevice(GPU, vendor 0x8086, device 0x56a1)

julia> compute_properties(dev)
(maxTotalGroupSize = 1024, maxGroupSizeX = 1024, maxGroupSizeY = 1024, maxGroupSizeZ = 1024, maxGroupCountX = 4294967295, maxGroupCountY = 4294967295, maxGroupCountZ = 4294967295, maxSharedLocalMemory = 65536, subGroupSizes = (8, 16, 32))

julia> queue = ZeCommandQueue(ctx, dev);

julia> execute!(queue) do list
         append_barrier!(list)
       end

Built on top of that, are kernel programming capabilities for executing Julia code on oneAPI accelerators. Device-side intrinsics are provided by SPIRVIntrinsics.jl, and code is compiled to SPIR-V using LLVM's SPIR-V back-end:

julia> function kernel()
         barrier(0)
         return
       end

julia> @oneapi items=1 kernel()

Code reflection macros are available to see the generated code:

julia> @device_code_llvm @oneapi items=1 kernel()
;  @ REPL[2]:1 within `kernel`
define spir_kernel void @_Z6kernel() local_unnamed_addr {
conversion:
  br label %top

top:                                              ; preds = %conversion
;  @ REPL[2]:2 within `kernel`
; ┌ @ SPIRVIntrinsics/src/synchronization.jl:162 within `barrier`
; │┌ @ SPIRVIntrinsics/src/synchronization.jl:154 within `work_group_barrier`
; ││┌ @ SPIRVIntrinsics/src/synchronization.jl:54 within `control_barrier`
     call void @_Z22__spirv_ControlBarrierjjj(i32 2, i32 2, i32 16)
; └└└
;  @ REPL[2]:3 within `kernel`
  ret void
}
julia> @device_code_spirv @oneapi items=1 kernel()
; SPIR-V
; Version: 1.4
; Generator: LLVM LLVM SPIR-V Backend; 22
; Bound: 22
; Schema: 0
               OpCapability Kernel
               OpCapability Addresses
          %1 = OpExtInstImport "OpenCL.std"
               OpMemoryModel Physical64 OpenCL
               OpEntryPoint Kernel %_Z6kernel "_Z6kernel"
               OpExecutionMode %_Z6kernel ContractionOff
               OpSource OpenCL_C 200000
               OpName %_Z6kernel "_Z6kernel"
               OpName %conversion "conversion"
               OpName %top "top"
       %void = OpTypeVoid
          %5 = OpTypeFunction %void
       %uint = OpTypeInt 32 0
    %uint_16 = OpConstant %uint 16
     %uint_2 = OpConstant %uint 2
  %_Z6kernel = OpFunction %void None %5
 %conversion = OpLabel
               OpBranch %top
        %top = OpLabel
               OpControlBarrier %uint_2 %uint_2 %uint_16
               OpReturn
               OpFunctionEnd

Finally, the oneArray type makes it possible to use your oneAPI accelerator without the need to write custom kernels, thanks to Julia's high-level array abstractions:

julia> a = oneArray(rand(Float32, 2,2))
2×2 oneArray{Float32, 2, oneAPI.oneL0.DeviceBuffer}:
 0.879687  0.990588
 0.421241  0.157232

julia> a .+ 1
2×2 oneArray{Float32, 2, oneAPI.oneL0.DeviceBuffer}:
 1.87969  1.99059
 1.42124  1.15723

The oneMKL integration provides extended support for linear algebra operations, including sparse matrix operations that integrate with Julia's standard LinearAlgebra interface:

julia> using oneAPI, oneAPI.oneMKL, SparseArrays, LinearAlgebra

julia> A = sprand(Float32, 100, 100, 0.1);

julia> dA = oneMKL.oneSparseMatrixCSR(A);

julia> x = oneArray(rand(Float32, 100));

julia> y = dA * x;  # matrix-vector multiplication via LinearAlgebra

Sparse matrices are also available in CSC (oneSparseMatrixCSC) and COO (oneSparseMatrixCOO) formats. Note that oneMKL's sparse back-end is CSR-based, so some operations — notably the triangular solves and multiplications — are unavailable for CSC matrices and will throw an ArgumentError.

Writing portable code

While oneAPI.jl provides Intel-specific functionality, it is recommended to write backend-agnostic code where possible, so that it runs on other GPU back-ends without modification:

  • GPUArrays.jl for high-level array abstractions
  • KernelAbstractions.jl for kernels that compile for CPU, CUDA, ROCm, Metal and oneAPI devices. oneAPI.jl provides the oneAPIBackend back-end for this.

Reach for oneAPI-specific macros (like @oneapi) and types (like oneArray) when you need optimizations or features that the generic abstractions do not cover.

Float64 support

Not all oneAPI GPUs support Float64 datatypes. You can test if your GPU does using the following code:

julia> using oneAPI
julia> oneL0.module_properties(device()).fp64flags & oneL0.ZE_DEVICE_MODULE_FLAG_FP64 == oneL0.ZE_DEVICE_MODULE_FLAG_FP64
false

If your GPU doesn't, executing code that relies on Float64 values will result in an error:

julia> oneArray([1.]) .+ 1
┌ Error: Module compilation failed:
│
│ error: Double type is not supported on this platform.

Development

To work on oneAPI.jl, you just need to dev the package. In addition, you may need to build the binary support library that's used to interface with oneMKL and other C++ vendor libraries. This library is normally provided by the oneAPI_Support_jll.jl package, however, we only guarantee to update this package when releasing oneAPI.jl. You can build this library yourself by executing:

$ julia --project=deps deps/build_local.jl

This installs a Conda environment with Intel DPC++ and MKL, builds the library, and writes a LocalPreferences.toml that points oneAPI.jl at the result.

Most of the oneMKL bindings are generated from the oneMKL C++ headers. After changing the generator or updating oneMKL, regenerate them with:

$ julia --project=deps deps/generate_interfaces.jl

Pull requests are checked with Runic.jl; format your changes with git runic main before submitting.

To facilitate development, there are other things you may want to configure:

Enabling the oneAPI validation layer

The oneAPI Level Zero libraries feature a so-called validation layer, which validates the arguments to API calls. This can be useful to spot potential isssues, and can be enabled by setting the following environment variables:

  • ZE_ENABLE_VALIDATION_LAYER=1
  • ZE_ENABLE_PARAMETER_VALIDATION=1
  • EnableDebugBreak=0 (this is needed to work around intel/compute-runtime#639)

Using a debug toolchain

If you're experiencing an issue with the underlying toolchain (NEO, IGC, etc), you may want to use a debug build of these components, which also perform additional validation. This can be done simply by calling oneAPI.set_debug!(true) and restarting your Julia session. This sets a preference used by the respective JLL packages.

Using a local toolchain

To further debug the toolchain, you may need a custom build and point oneAPI.jl towards it. This can also be done using preferences, overriding the paths to resources provided by the various JLLs that oneAPI.jl uses. A helpful script to automate this is provided in the res folder of this repository:

$ julia res/local.jl

Trying to find local IGC...
- found libigc at /usr/local/lib/libigc.so
- found libiga64 at /usr/local/lib/libiga64.so
- found libigdfcl at /usr/local/lib/libigdfcl.so
- found libopencl-clang at /usr/local/lib/libopencl-clang.so.11

Trying to find local gmmlib...
- found libigdgmm at /usr/local/lib/libigdgmm.so

Trying to find local NEO...
- found libze_intel_gpu.so.1 at /usr/local/lib/libze_intel_gpu.so.1
- found libigdrcl at /usr/local/lib/intel-opencl/libigdrcl.so

Trying to find local oneAPI loader...
- found libze_loader at /lib/x86_64-linux-gnu/libze_loader.so
- found libze_validation_layer at /lib/x86_64-linux-gnu/libze_validation_layer.so

Writing preferences...

The discovered paths will be written to a global file with preferences, typically $HOME/.julia/environments/vX.Y/LocalPreferences.toml (where vX.Y refers to the Julia version you are using). You can modify this file, or remove it when you want to revert to default set of binaries.

About

Julia support for the oneAPI programming toolkit.

Topics

Resources

Stars

Watchers

Forks

Releases

Used by

Contributors

Languages