Skip parquet page when unneeded (#18192)
Improve binview extend/ifthenelse (#18164)
Start on better Parquet delta decoding (#18049)
Tune jemalloc to not create muzzy pages (#18148)
Reduce default async thread count (#18142)
Use single threaded algorithms if only 1 core given (#18101)
Use Arc<Vec<_>> instead of Arc<[_]> for paths and hive partitions (#18066)
SIMD View from FixedSizeBinary (#18059)
Use bitmask to filter Parquet predicate-pushdown items (#17993)
Zerocopy buffers for FixedSizeBinary to BinaryView cast (#18043)
Integer fast path Parquet dict encoding (#18030)
Speedup writing of Parquet primitive values (#18020)
Remove temporary allocations in Parquet (#18013)
Delay selection expansion (#18011)
Optimize strings slices (#17996)
Make .dt.weekday 20x faster (#17992)
Shrink MemSliceInner enum (#17991)
Push down slice with non-zero offset to Parquet (#17972)
Reduce copy in MemSlice (#17983)
Ensure metadata flags are maintained on vertical parallelization (#17804)
Ensure only nodes that are not changed are cached in collapse optimizer (#17791)
Use bitflags for OptState (#17788)
Remove async directory auto-detection (#17779)
Fix accidental quadratic horizontal concat (#17783)
Batch parquet integer decoding (#17734)
Use mmap-ed memory if possible in Parquet reader (#17725)
Use bitflags for function options (#17723)
Introduce MemReader to file buffer in Parquet reader (#17712)
Better GC and push_view for binviews (#17627)
Fix pathological perf issue in window-order-by (#17650)
Cache path resolving of scan functions (#17616)
Add ArrayChunks to optimize codegen of BatchDecoder (#17632)
Rechunk before we go into grouped gathers (#17623)
Cache schema resolve back to DSL (#17610)
Add fastpath for when rounding by single constant durations (#17580)
Improve parallelism in writing hive parquet (#17512)
Support datetime in predicate during hive partition pruning (#17545)
Batch nested embed parquet decoding (#17549)
Batch nested Parquet decoding (#17542)
Collect Parquet dictionary binary as view (#17475)
Keep more parallelism when CSE plan cache hits (#17463)
Batch parquet primitive decoding (#17462)
Respect allow_threading in some more operators (#17450)
Parallelize parquet metadata deserialization (#17399)

✨ Enhancements

Create literals for datetime/date expressions (#18184)
Create literals in 'datetime' expression (#18182)
Add missing impl for Series (#18166)
Raise on invalid 'is_between' and improve error message quality (#18147)
Add boolean Parquet HybridRle encoding (#18022)
Add nested SQL join support (#18006)
Push down slice with non-zero offset to Parquet (#17972)
Add support for binary size method to Expr and Series "bin" namespace (#17924)
Add SQL interface support for PostgreSQL dollar-quoted string literals (#17940)
Allow for parsing parquet file where the time zone is stored as lowercase "utc" (#17925)
Expose binary_elementwise_into_string_amortized for plugin authors, recommend apply_into_string_amortized instead of apply_to_buffer (#17903)
Decompress in CSV / NDJSON scan (#17841)
Ensure unique names in HConcat (#17884)
Support authentication with HuggingFace login (#17881)
Support "BY NAME" qualifier for SQL "INTERSECT" and "EXCEPT" set ops (#17835)
Raise informative error instead of panicking when passing invalid directives to to_string for Date dtype (#17670)
Implement forward/backward fill for all types (#17861)
Implement is_in operation on decimal type (#17832)
Support hf:// in read_(csv|ipc|ndjson) functions (#17785)
Allow literals in sort (#17780)
Cloud support for NDJSON (#17717)
Support API token for scanning hf:// (#17682)
Raise error instead of panic in unsupported serde (#17679)
Include file path option for NDJSON (#17681)
Hugging Face path expansion (#17665)
Add DSL validation for cloud eligible check (#17287)
Raise informative error message if non-IntoExpr is passed by name in *Frame.group_by (#17654)
Change API for writing partitioned Parquet to reduce code duplication (#17586)
Cache schema resolve back to DSL (#17610)
Expose returns_scalar to map_elements (#17613)
Add option to include file path for Parquet, IPC, CSV scans (#17563)
Support describe on decimal (#15092)
Support datetime in predicate during hive partition pruning (#17545)
Raise more informative error message for directories containing files with mixed extensions (#17480)
Exclude empty files from directory/glob expansion (#17478)
Add "future" versioning (#17421)
Apply slice pushdown immediately to in-memory frames (#17459)
Support writing hive partitioned parquet (#17324)
Add right join support (#17441)
Support hive partitioning in scan_ipc (#17434)

🐞 Bug fixes

Fix struct shift and list builder (#18189)
Don't load Parquet nested metadata (#18183)
Throw bigidx error for Parquet row-count (#18154)
Fix unpivot on empty df (#18179)
Don't vertically parallelize cse contexts (#18177)
Properly handle empty Parquet row groups with no dictionary (#18161)
Struct outer nullabillity (#18156)
Fix pyarrow predicate pushdown regression (#18145)
Prevent unwanted supertype cast in 'search_sorted' (#18143)
Parquet with filter=None (#18139)
Don't raise when converting from pandas if index contains duplicate names when include_index=False (the default) (#18133)
Don't remove leading whitespace in read_csv (#18131)
Py-polars compilation with no features (#18129)
String transform to_titlecase was too narrowly defined (#18122)
Reading Parquet with Null dictionary page (#18112)
Incorrect lazy CSV select(len()) for compressed files (#18067)
Fix sink_ipc_cloud panicking with runtime error (#18091)
Properly write Parquet for sliced lists (#18073)
Panic reading multiple CSV files from cloud (#18056)
Fix CloudWriter to use buffer before making requests (#18027)
Fix typos and remove trailing whitespace (#18024)
Handle cfg(feature) for shrink_dtype (#18038)
Subtraction with overflow on negative slice offset in Parquet (#18036)
Add nested SQL join support (#18006)
Allow read_csv schema to take unparsable types (#17765)
Multi-output column expressions in frame sort method (#17947)
Fix Asof join by schema (#17988)
Fix glob resolution for Hugging Face (#17958)
Several parquet reader/writer regressions (#17941)
Incorrect filter on categorical columns from parquet files (#17950)
SQL COUNT(DISTINCT x) should not include NULL values (#17930)
Scanning '%' from cloud (#17890)
Respect glob=False for cloud reads (#17860)
Properly write nest-nulled values in Parquet (#17845)
Allow full-null Object series to be built (#17870)
Fix from_arrow for struct type (#17839)
Infer decimal scales on mixed scale input (#17840)
Raise on unsupported fill strategy dtype (#17837)
Properly write nested NullArray in Parquet (#17807)
Check input type on list.to_struct (#17834)
Fix right join schema (#17833)
Non-compliant Parquet list element name (#17803)
Correctly set should_broadcast flag in HStack CSE rewrite (#17784)
Fix projection pusdhown of literals without names (#17778)
Don't expand HTTP paths (#17774)
Check funtion input len at expansion (#17763)
Don't panic in invalid agg_groups (#17762)
Raise empty struct (#17736)
Fix GC logic in write_ipc (#17752)
Panic in pl.concat_list and list.concat on empty inputs (#17742)
Fix out nullability for structs coming from arrow (#17738)
Percent encode for Hugging Face paths (#17718)
Use bytemuck in slice reinterpret for Parquet ArrayChunks (#17700)
Propagate struct outer nullability eagerly (#17697)
Use ETag for HTTP file cache invalidation (#17684)
Fix type inference failure caused by double transpose (#17663)
Interpret %y consistently with Chrono in to_date/to_datetime/strptime (#17661)
Fix explode invalid check (#17651)
Tighten up error checking on join keys (#17517)
Expand brackets in async glob expansion (#17630)
Fix row index disappearing after projection pushdown in NDJSON (#17631)
Fix struct -> enum is_in (#17622)
Don't needlessly unwrap in pivot_schema (#17611)
Reject literal input in sort_by_exprs() (#17606)
Bitmap collect into safety (#17588)
Method dt.truncate was sometimes returning incorrect results for pre-1970 datetimes (#17582)
Defer path expansion until collect in file scan methods (#17532)
Correct logic for descending sort of BooleanChunked (#17558)
Don't unwrap send attempt to oneshot channel (#17566)
Fix scanning from HTTP cloud paths (#17571)
Properly implement struct (#17522)
Add missing commas in python IR interchange (#17518)
Fix predicate pushdown for .list.(get|gather) (#17511)
Turn panic into error when serializing Object types (#17353)
Fix struct expansion and raise on exclude (#17489)
Fix decimal dyn float supertype (#17464)
Don't rechunk on phys_repr (#17461)
Harden alchemy session for old sqlalchemy versions (#17366)
Fix swapping rename schema (#17458)
Raise on oob decimal precision (#17445)
Don't allow json inference method to be chunked/streaming (#17396)
avoid panic when projecting solitary count into empty frame (#17393)
Set literal nesting to 0 (#17392)
Fix scanning cloud paths with spaces (#17379)
Fix slice length no longer allowing None (#17372)
Cull row index in scan if projection pushdown removes it (#17363)
Fix typo in SchemaError exception message (#17350)

📖 Documentation

Mention 'Array' in data types overview (#18060)
Correct concat rechunk in user guide (#18080)
Fix typo in title of Hugging Face docs page (#18097)
Update pivot docstring for clarity (#18000)
Fix aggregation guide discrepancies (#18003)
Updating the join example input for rust for consistency with python example (#17898)
Rename str.lengths to str.len_bytes in description text (#11577) (#17626)
Create example for polars.Expr.bin.decode (#17508)
Add right join in the user guide (#17608)
Update SQL examples in README (#17568)
Fix typo in Getting Started section of user guide (#17465)

📦 Build system

Update Rust toolchain to nightly-2024-07-26 (#17891)

🛠️ Other improvements

Consistently use PlRandomState, PlHashMap/Set (#18190)
Remove unneeded growable (#18165)
Allow non-scoped tasks to be spawned (#18163)
Update Cargo.lock to fix build error on Linux (#18153)
Remove Nth,Wildcard from ExprIR and make conversion falllible (#18115)
Allow zip node to extend with nulls or broadcast inputs (#18077)
Add @coastalwhite to parquet codeowners (#18032)
Minor bump to comfy-table version (#18028)
Add unit tests for str.contains_any and str.replace_many (#17961)
Remove apply_generic, use unary_elementwise (#17902)
Add general filters in Parquet (#17910)
Fix broken typos command in make pre-commit for py-polars folder (#17897)
Remove HybridRLE iter / batch nested parquet decoding (#17889)
Deal with re-entrant expressions locally (#17885)
Unify Parquet nested decoders (#17816)
Add zip node to streaming engine (#17866)
Fix simple projection in streaming engine (#17871)
Make better use of typos configuration features (#17800)
Remove complicated Parquet traits (#17767)
Rename Unit to Plain in Parquet reader (#17751)
Add missing feature flag for prefetching on arm (#17740)
Add the capability to run new streaming engine on test suite (#17706)
Update CODEOWNERS (#17707)
Allow compute nodes in streaming engine to decide whether they want serial or parallel input/output (#17699)
Name tests so they actually run (#17690)
Add reduce ComputeNode in new streaming engine (#17389)
Add DSL validation for cloud eligible check (#17287)
Rename Pipe to Connector (#17655)
Document and simplify MutableBinView::push_ignore_validity (#17645)
Remove has_validity, use has_nulls (#17519)
Add ArrayChunks to optimize codegen of BatchDecoder (#17632)
Move path logic to from utils to path_utils in polars-io (#17635)
Fix struct gather (#17621)
Back to StructChunked name (#17609)
Re-enable struct related tests (#17597)
Completely redo structure of Parquet decoder (#17589)
Fix struct outer validity;fmt;is_in;cast;cmp (#17590)
Make no-default-features compile (#17516)
No longer blanket allow unused items in polars-stream (#17551)
Smaller filter / valid in Parquet (#17569)
Move path expansion utils to polars-io crate (#17521)
Add streaming slice node to new streaming engine (#17451)
Add ordered union to new streaming engine (#17359)
Add automated check for PR title formatting (#17412)
Remove transmute for object store path (#17395)
Fix Python version resolver in release drafter (#17390)
Remove dead code (#17378)

Thank you to all our contributors for making this release possible! @5j9, @ByteNybbler, @EricTulowetzke, @JamesCE2001, @Julian-J-S, @KDruzhkin, @MarcoGorelli, @Object905, @SandroCasagrande, @Vincenthays, @alexander-beedie, @anergictcell, @arnabanimesh, @atigbadr, @brandon-b-miller, @brunobbaraujo, @cmdlineluser, @coastalwhite, @davanstrien, @deanm0000, @deepyaman, @delsner, @dependabot, @dependabot[bot], @diegoglozano, @eitsupi, @ember91, @flisky, @henryharbeck, @implicit-apparatus, @itamarst, @jonaylor89, @jparag, @knl, @kylebarron, @lukapeschke, @mcrumiller, @moritzwilksch, @nameexhaustion, @orlp, @phi-friday, @philss, @r-brink, @ragyabraham, @rcorty, @ritchie46, @rosstitmarsh, @ruihe774, @sherlockbeard, @stinodego, @szepeviktor, @tylerriccio33, @wangxiaoying and @wence-