Split Arrow CPP libraries on conda

Tuesday, November 28, 2023

It has been a really long time since I have written anything but during the last year I have been involved in the development and releases of Apache Arrow and I have been contributing to the project and the overall ecosystem.

One of the things I have worked during the last months has been to divide the Apache Arrow C++ conda package into subpackages in order to provide a more modular installation.

Before Arrow 14.0.0 on conda we did provide a single libarrow package that contained a full build of Apache Arrow C++. Since 14.0.0 we also provide a single package with everything called libarrow-all which is a metapackage that pulls all the following libraries:

  • libarrow
  • libparquet
  • libarrow-acero
  • libarrow-dataset
  • libarrow-flight
  • libarrow-flight-sql
  • libarrow-gandiva
  • libarrow-substrait

Those are also provided individually with the following dependencies:

libarrow packages