Extension Scalar VTable

[connortsui20]

Summary

Tracking Issue: #6618

Adds methods related to extension scalars to the ExtVTable.

This then required updating existing implementors of ExtVTable (only datetime and mock extension test types).

I think this is a nice step before the much more intrusive change to ScalarValue where we add the Extension variant.

API Changes

This is a breaking change to ExtVTable.

Testing

TODO

[gatesn]

Looking nice

[connortsui20]

I added some TODOs and concerns, could you take a look at the most recent commit @gatesn?

[gatesn]

I think in this case it's weird, we only need one Plugin for the ExtVTable, not one for DType and one for Scalar.

[connortsui20]

I still dont actually understand what the plugin is for even after reading the doc comment

[gatesn]

The "plugin" is the thing that lives on the session. It's what allows us to deserialize things into in-memory objects. We lookup the plugin by ID, then call its deserialize function.

It basically encapsulates extension dtype behavior that doesn't have an instance object to operate on.

[connortsui20]

So I decided to keep the validate_scalar_value, but now it just calls unpack_native.map(|_| ()). This way if we see a perf problem later it is way easier to just implement this if we want to.

[connortsui20]

I am still concerned about some of the trait impls on ExtScalarValueRef, right now it basically just delegates things like Eq, Ord, and Hash to the storage value. I don't think we want this? We probably want the implementor to implement this manually right?

[gatesn]

I am still concerned about some of the trait impls on ExtScalarValueRef, right now it basically just delegates things like Eq, Ord, and Hash to the storage value. I don't think we want this? We probably want the implementor to implement this manually right?

Your instinct is correct! But we can do that in the future. We should do it alongside arrays so we can define the vectorized version of this at the same time, otherwise scalar and array semantics would differ.

[codspeed-hq]

Merging this PR will not alter performance

✅ 954 untouched benchmarks
⏩ 1466 skipped benchmarks¹

---

_{Comparing ct/ext-scalar-final (6348fca) with develop (d59d334)}

Footnotes

1. 1466 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩