Light Gbm: v4.4.0 Release

Changes

💡 New Features

• [ci] [python-package] add arm64 macOS wheels @jameslamb (#6391)
• [python-package] Add support for NumPy 2.0, test against nightly versions of dependencies (fixes #6454) @jameslamb (#6467)
• [R-package] expose start_iteration to dump/save/lgb.model.dt.tree @mayer79 (#6398)
• [python-package] Allow to pass early stopping min delta in params @borchero (#6274)
• [python-package] Allow to pass Arrow table with boolean columns to dataset @borchero (#6353)
• [c++][fix] Support Quantized Training with Categorical Features on CPU @shiyu1994 (#6301)

🔨 Breaking

• [python-package] remove uses of deprecated NumPy random number generation APIs, require 'numpy>=1.17.0' @jameslamb (#6468)
• [python-package] drop Python 3.6 support, add Python 3.12 @jameslamb (#6459)
• [cmake] [c++] use .dylib extension unconditionally on macOS @jameslamb (#6418)
• ENH more stable gradient of CrossEntropy @lorentzenchr (#6327)
• [R-package] Rename slice() to lgb.slice.Dataset() @david-cortes (#6293)

🚀 Efficiency Improvement

• [R-package] Use ALTREP system to return C++-allocated arrays @david-cortes (#6213)
• [R-package] Speed-up lgb.importance() @mayer79 (#6364)
• Fix single row prediction performance in a multi-threaded environment @Ten0 (#6024)

🐛 Bug Fixes

• [R-package] ensure use of interaction_constraints does not lead to features being ignored @mayer79 (#6377)
• [R-package] skip integer categorical feature check when building dataset subset (fixes #6412) @jmoralez (#6442)
• [cmake] always target Windows SDK v10.x when using Visual Studio generators (fixes #6448) @jameslamb (#6451)
• upgrade CMake in dockerfile-cli (fixes #6420) @narumiruna (#6426)
• remove unnecessary omp single that cause deadlock (fixes #6273) @morokosi (#6394)
• [python-package] allow use of early_stopping_round<=0 to turn off early stopping (fixes #6401) @ddelzell (#6406)
• [c++] Fix memory leak in Arrow table implementation @borchero (#6314)
• Fix calculation of number of bins in FindGroup @CVPaul (#6019)

📖 Documentation

• [docs] ensure pkgdown site index accurately reflects R-package public API (fixes #6444) @jameslamb (#6445)
• [docs] document how to use a custom objective function via the C API @parsiad (#6396)
• [docs] [ci] encourage use of cmake --build @jameslamb (#6368)
• [docs] remove language about CUDA version being experimental (fixes #6340) @jameslamb (#6371)

🧰 Maintenance

• release v4.4.0 @jameslamb (#6439)
• [python-package] add a few type hints in LGBMModel.fit() @jameslamb (#6470)
• [python-package] clarify max_depth warning and limit when it is emitted @jameslamb (#6402)
• [ci] upgrade linters to latest version @jameslamb (#6472)
• [ci] [R-package] add new linters from {lintr} 3.1.1 @jameslamb (#6471)
• [python-package] make LGBMDeprecationWarning inherit from FutureWarning @jameslamb (#6447)
• [ci] further simplify CI configurations @jameslamb (#6463)
• [ci] fix CUDA 11.8 builds (fixes #6466) @jameslamb (#6465)
• [ci] simplify CI configurations, parallelize compilation, test CUDA on Ubuntu 22.04 @jameslamb (#6458)
• [ci]: Bump actions/checkout from 1 to 4 in the ci-dependencies group @dependabot (#6453)
• [cmake] switch to FindCUDAToolkit @characat0 (#6457)
• [R-package] [python-package] deprecate Dataset arguments to cv() and train() @jameslamb (#6446)
• [ci] Add dependabot configuration @borchero (#6449)
• [python-package] upgrade to scikit-build-core 0.9.3 @jameslamb (#6263)
• deprecate HDFS support @jameslamb (#6443)
• [c++] remove uses of '..' in headers @jameslamb (#6409)
• [R-package] always name the shared library 'lightgbm', not 'lib_lightgbm' @jameslamb (#6432)
• [R-package] [ci] Manually install 'Matrix' (fixes #6433) @jameslamb (#6434)
• [ci] Skip Arrow tests on AppVeyor, use Intel macOS runners, upgrade to XCode 14.3 on macOS jobs, disable MacOS MPI jobs @borchero (#6425)
• [ci] raise floors on CI dependencies @jameslamb (#6375)
• [ci] make shell scripts stricter @jameslamb (#6266)
• [python-package] prevent build-python.sh from modifying already-installed dependencies @jameslamb (#6403)
• [ci] reduce hard-coding of host architecture @jameslamb (#6395)
• [c++] remove unused variable 'need_connect_cnt' @jameslamb (#6410)
• [ci] clean source directory at the beginning of every Azure DevOps build @jameslamb (#6416)
• [ci] Fix Azure Pipeline Agents @shiyu1994 (#6407)
• [ci] [R-package] re-enable r-sanitizers jobs (fixes #6367) @jameslamb (#6374)
• [ci] prevent trailing whitespace, ensure files end with newline @jameslamb (#6373)
• [ci] [docs] update some docs and CI dependencies @jameslamb (#6346)
• [ci] [python-package] remove unused flake8 configuration @jameslamb (#6372)
• [ci] put ceiling on graphviz in Python 3.7 jobs @jameslamb (#6370)
• [ci] Fix R 3.6 tests, dask tests, compatibility with dask>=2024.3.1 @borchero (#6357)
• [ci] [python-package] check for untyped definitions with mypy @jameslamb (#6339)
• [ci] ignore ruff-format changes in git blame (fixes #6304) @jameslamb (#6345)
• [c++] remove unnecessary include in CLI @jameslamb (#6342)
• [ci] [python-package] enable ruff-format on all Python code @jameslamb (#6336)
• [python-package] remove unused type alias @jameslamb (#6341)
• [ci] [python-package] enforce 'pylint' checks (fixes #4308) @jameslamb (#6334)
• [ci] [python-package] enable ruff-format on tests and examples @jameslamb (#6317)
• [python-package] fix mypy error about pyarrow data @jameslamb (#6333)
• [ci] remove unnecessary workarounds in R-package macOS jobs @jameslamb (#6331)
• [ci] remove workflow_dispatch from no-response configuration @jameslamb (#6330)
• [ci] fix no-response permissions @jameslamb (#6323)
• [ci] enable ruff-format on some files, add pre-commit config @jameslamb (#6308)
• [ci] ensure shell script exit codes are in range 0-255 @jameslamb (#6305)
• [ci] fix installation of clang-18 (fixes #6306) @jameslamb (#6307)
• [ci] [R-package] pin to older setup-pandoc for R 3.6 jobs (fixes #6298) @jameslamb (#6299)
• bump development version to 4.3.0.99 @jameslamb (#6287)