Skip to content

Comment on Guix for Developmentparent

Comments

I tried to change Guix to enable vulkan in pytorch and I get a build error. Since I don't use pytorch I didn't investigate further.

If you do know the build options to make this work, Guix will gladly consider your patch.

The in-progress diff would be,

  diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
  index e702e499fc..73af29487d 100644
  --- a/gnu/packages/machine-learning.scm
  +++ b/gnu/packages/machine-learning.scm
  @@ -101,6 +101,7 @@ (define-module (gnu packages machine-learning)
     #:use-module (gnu packages swig)
     #:use-module (gnu packages tls)
     #:use-module (gnu packages video)
  +  #:use-module (gnu packages vulkan)
     #:use-module (gnu packages web)
     #:use-module (gnu packages xml)
     #:use-module (gnu packages xorg)
  @@ -2897,6 +2898,10 @@ (define-public python-pytorch
       (build-system python-build-system)
       (arguments
        '(#:phases (modify-phases %standard-phases
  +                  (add-before 'build 'use-vulkan
  +                    (lambda _
  +                      (setenv "USE_VULKAN" "1")
  +                      (setenv "USE_VULKAN_SHADERC_RUNTIME" "1")))
                     (add-before 'build 'use-system-libraries
                       (lambda\* (#:key outputs #:allow-other-keys)
                         ;; Tell 'setup.py' to let 'CMakeLists.txt' know that we
  @@ -2973,7 +2978,11 @@ (define-public python-pytorch
              pthreadpool
              protobuf
              pybind11
  +           shaderc
              sleef
  +           glslang spirv-headers ;spirv-tools ; not sure why this is needed
  +           vulkan-headers
  +           vulkan-loader
              xnnpack
              zstd))
       (propagated-inputs
Error messages:
  FAILED: bin/scalar_test 
: && /gnu/store/069aq2v993kpc41yabp5b6vm4wb9jkhg-gcc-10.3.0/bin/c++ -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_QNNPACK -DUSE_PYTORCH_QN NPACK -DUSE_XNNPACK -DUSE_VULKAN -DUSE_VULKAN_API -DUSE_VULKAN_SHADERC_RUNTIME -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing -field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno -error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wn o-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow -DHAVE_AVX512_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION -O3 -DNDEBUG -DNDEBUG -rdynamic - pthread caffe2/CMakeFiles/scalar_test.dir/__/aten/src/ATen/test/scalar_test.cpp.o -o bin/scalar_test -Wl,-rpath,/tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib: -lgtest_main -lgtest /gnu/store/0 94bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib/lib/libgomp.so /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33/lib/libpthread.so -Wl,--no-as-needed,"/tmp/guix-build-python-pytorch-1.12.0.drv-0/source/bui ld/lib/libtorch.so" -Wl,--as-needed -Wl,--no-as-needed,"/tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib/libtorch_cpu.so" -Wl,--as-needed /gnu/store/9pyydl5w9xnz1qm56sxn1zh4qny6fkxz-protobuf-3.17.3 /lib/libprotobuf.so lib/libc10.so -pthread && :
  ld: /tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib/libtorch_cpu.so: undefined reference to `glslang::TShader::setNanMinMaxClamp(bool)'
  ld: /tmp/guix-build-python-pytorch-1.12.0.drv-0/source/build/lib/libtorch_cpu.so: undefined reference to `spvtools::CreateCompactIdsPass()'
  [...]
The reason is because pytorch 1.12.0 depends on shaderc-2020.4/lib/libshaderc_combined.a--and static libraries do not have automatic dependency resolution, so spirv, glsl etc is never linked.

Just tried--same problem with pytorch 1.12.1.

So that's why it's not enabled.

pytorch is using shaderc_combined.a.

There's a shaderc_combined.pc in shaderc, but it doesn't lists glslang or spirv as dependencies, even though shaderc uses glslang and spirv data structures and functions. Errrrr. How's that supposed to work?

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.